Spaces:

neuralworm
/

els_journal_lgc_2_2

Sleeping

App Files Files Community

els_journal_lgc_2_2 / repo.txt

neuralworm

initial commit, move to new repo

23f8e38 3 months ago

raw

history blame contribute delete

141 kB

	Repository Documentation
	This document provides a comprehensive overview of the repository's structure and contents.
	The first section, titled 'Directory/File Tree', displays the repository's hierarchy in a tree format.
	In this section, directories and files are listed using tree branches to indicate their structure and relationships.
	Following the tree representation, the 'File Content' section details the contents of each file in the repository.
	Each file's content is introduced with a '[File Begins]' marker followed by the file's relative path,
	and the content is displayed verbatim. The end of each file's content is marked with a '[File Ends]' marker.
	This format ensures a clear and orderly presentation of both the structure and the detailed contents of the repository.

	Directory/File Tree Begins -->

	/
	├── README.md
	├── __pycache__
	├── app.py
	├── bible.py
	├── database-structure.txt
	├── els_cache.db.initial
	├── gematria.py
	├── hindu.py
	├── populate_translations.py
	├── quran.py
	├── requirements-all.txt
	├── requirements.txt
	├── texts
	│ ├── bible
	│ ├── mahabharata
	│ ├── quran
	│ ├── rigveda
	│ ├── torah
	│ └── tripitaka
	├── torah.py
	├── translation_database.db.initial
	├── translation_utils.py
	├── tripitaka.py
	├── util.py
	└── utils.py

	<-- Directory/File Tree Ends

	File Content Begin -->
	[File Begins] README.md
	---
	title: Book of Souls - Search your name+day (journal) oracle with ELS over Torah, Bible, Quran, Rigveda, Tripitaka
	emoji: 📊
	colorFrom: green
	colorTo: pink
	sdk: gradio
	sdk_version: 4.39.0
	app_file: app.py
	pinned: false
	---

	This application searches for equidistant letter sequences (ELS) in the Torah, Bible, Quran, and Rigveda. It also integrates a network search functionality to find related phrases based on gematria.

	Inputs:

	* Target Language for Translation: The language to translate the results into.
	* Date to investigate (optional): A date to include in the gematria calculation.
	* Language of the person/topic (optional) (Date Word Language): The language to use for converting the date to words.
	* Name and/or Topic (required): The text to calculate the gematria for.
	* Jump Width (Steps) (optional) for ELS: The step size for the ELS search.
	* Round (1) / Round (2) (optional): The number of rounds for the ELS search (positive or negative).
	* Include Torah / Include Bible / Include Quran / Include Rigveda: Checkboxes to select which texts to search.
	* Strip Spaces from Books / Strip Text in Braces from Books / Strip Diacritics from Books: Options for text preprocessing.

	Outputs:

	* ELS Results: A dataframe containing the ELS search results.
	* Most Frequent Phrase in Network Search: The most frequent phrase found in the network search.
	* JSON Output: A JSON representation of the search results.


	How to Use:

	1. Enter the name or topic you want to investigate.
	2. Optionally, select a date and the language for its representation.
	3. Set the jump width (steps) and rounds for the ELS search.
	4. Choose which texts to include in the search.
	5. Configure text preprocessing options as needed.
	6. Click "Search with ELS".
	7. The results will be displayed in the output sections. You can copy the JSON output using the provided button.


	Network Search:

	The network search functionality uses the calculated gematria of the ELS results to search a database for phrases with the same gematria. It displays the most frequent matching phrase. If no exact match is found, it attempts to find the closest match based on similarity and word count difference.
	[File Ends] README.md

	[File Begins] app.py
	#TODO: Quran results have numbers

	import logging
	logger = logging.getLogger(__name__)
	logging.basicConfig(level=logging.DEBUG)

	import gradio as gr
	import torah
	import bible
	import quran
	import hindu
	import tripitaka
	from utils import number_to_ordinal_word, custom_normalize, date_to_words, translate_date_to_words
	from gematria import calculate_gematria, strip_diacritics

	import pandas as pd
	from deep_translator import GoogleTranslator
	from gradio_calendar import Calendar
	from datetime import datetime, timedelta
	import math
	import json
	import re
	import sqlite3
	from collections import defaultdict
	from typing import List, Tuple
	import rich
	from fuzzywuzzy import fuzz
	import calendar
	import translation_utils
	import hashlib

	translation_utils.create_translation_table()

	# Create a translator instance once globally
	translator = GoogleTranslator(source='auto', target='auto')
	LANGUAGES_SUPPORTED = translator.get_supported_languages(as_dict=True) # Corrected dictionary name

	LANGUAGE_CODE_MAP = LANGUAGES_SUPPORTED # Use deep_translator's mapping directly

	# --- Constants ---
	DATABASE_FILE = 'gematria.db'
	MAX_PHRASE_LENGTH_LIMIT = 20

	ELS_CACHE_DB = "els_cache.db"
	DATABASE_TIMEOUT = 60

	# --- Database Initialization ---
	def initialize_database():
	global conn
	conn = sqlite3.connect(DATABASE_FILE)
	cursor = conn.cursor()
	cursor.execute('''
	CREATE TABLE IF NOT EXISTS results (
	gematria_sum INTEGER,
	words TEXT,
	translation TEXT,
	book TEXT,
	chapter INTEGER,
	verse INTEGER,
	phrase_length INTEGER,
	word_position TEXT,
	PRIMARY KEY (gematria_sum, words, book, chapter, verse, word_position)
	)
	''')
	cursor.execute('''
	CREATE INDEX IF NOT EXISTS idx_results_gematria
	ON results (gematria_sum)
	''')
	cursor.execute('''
	CREATE TABLE IF NOT EXISTS processed_books (
	book TEXT PRIMARY KEY,
	max_phrase_length INTEGER
	)
	''')
	conn.commit()

	# --- Initialize Database ---
	initialize_database()

	# --- ELS Cache Functions ---
	def create_els_cache_table():
	with sqlite3.connect(ELS_CACHE_DB) as conn:
	conn.execute('''
	CREATE TABLE IF NOT EXISTS els_cache (
	query_hash TEXT PRIMARY KEY,
	results TEXT
	)
	''')

	def get_query_hash(func, args, *kwargs):
	key = (func.__name__, args, tuple(sorted(kwargs.items())))
	return hashlib.sha256(json.dumps(key).encode()).hexdigest()


	def cached_process_json_files(func, args, *kwargs):
	query_hash = get_query_hash(func, args, *kwargs)

	try:
	with sqlite3.connect(ELS_CACHE_DB, timeout=DATABASE_TIMEOUT) as conn:
	cursor = conn.cursor()
	cursor.execute("SELECT results FROM els_cache WHERE query_hash = ?", (query_hash,))
	result = cursor.fetchone()
	if result:
	logger.info(f"Cache hit for query: {query_hash}")
	return json.loads(result[0])
	except sqlite3.Error as e:
	logger.error(f"Database error checking cache: {e}")

	logger.info(f"Cache miss for query: {query_hash}")
	results = func(args, *kwargs)

	try:
	with sqlite3.connect(ELS_CACHE_DB, timeout=DATABASE_TIMEOUT) as conn:
	cursor = conn.cursor()
	cursor.execute("INSERT INTO els_cache (query_hash, results) VALUES (?, ?)", (query_hash, json.dumps(results)))
	conn.commit()
	except sqlite3.Error as e:
	logger.error(f"Database error caching results: {e}")

	return results

	# --- Helper Functions (from Network app.py) ---
	def flatten_text(text: List) -> str:
	if isinstance(text, list):
	return " ".join(flatten_text(item) if isinstance(item, list) else item for item in text)
	return text

	def search_gematria_in_db(gematria_sum: int, max_words: int) -> List[Tuple[str, str, int, int, int, str]]:
	global conn
	with sqlite3.connect(DATABASE_FILE) as conn:
	cursor = conn.cursor()
	cursor.execute('''
	SELECT words, book, chapter, verse, phrase_length, word_position
	FROM results
	WHERE gematria_sum = ? AND phrase_length <= ?
	''', (gematria_sum, max_words))
	results = cursor.fetchall()
	return results

	def get_most_frequent_phrase(results):
	phrase_counts = defaultdict(int)
	for words, book, chapter, verse, phrase_length, word_position in results:
	phrase_counts[words] += 1
	most_frequent_phrase = max(phrase_counts, key=phrase_counts.get) if phrase_counts else None # Handle empty results
	return most_frequent_phrase

	# --- Functions from BOS app.py ---
	def create_language_dropdown(label, default_value='English', show_label=True): # Default value must be in LANGUAGE_CODE_MAP
	return gr.Dropdown(
	choices=list(LANGUAGE_CODE_MAP.keys()), # Correct choices
	label=label,
	value=default_value,
	show_label=show_label
	)

	def calculate_gematria_sum(text, date_words):
	if text or date_words:
	combined_input = f"{text} {date_words}"
	logger.info(f"searching for input: {combined_input}")
	numbers = re.findall(r'\d+', combined_input)
	text_without_numbers = re.sub(r'\d+', '', combined_input)
	number_sum = sum(int(number) for number in numbers)
	text_gematria = calculate_gematria(strip_diacritics(text_without_numbers))
	total_sum = text_gematria + number_sum
	return total_sum
	else:
	return None

	def perform_els_search(step, rounds_combination, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk, include_torah, include_bible, include_quran, include_hindu, include_tripitaka):
	if step == 0 or rounds_combination == "0,0":
	return None

	results = {}
	length = 0

	selected_language_long = tlang # From the Gradio dropdown (long form)
	tlang = LANGUAGES_SUPPORTED.get(selected_language_long) #Get the short code.
	if tlang is None: # Handle unsupported languages
	tlang = "en"
	logger.warning(f"Unsupported language selected: {selected_language_long}. Defaulting to English (en).")

	if include_torah:
	logger.debug(f"Arguments for Torah: {(1, 39, step, rounds_combination, length, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk)}")
	results["Torah"] = cached_process_json_files(torah.process_json_files, 1, 39, step, rounds_combination, length, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk)
	else:
	results["Torah"] = []

	if include_bible:
	results["Bible"] = cached_process_json_files(bible.process_json_files, 40, 66, step, rounds_combination, length, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk)
	else:
	results["Bible"] = []

	if include_quran:
	results["Quran"] = cached_process_json_files(quran.process_json_files, 1, 114, step, rounds_combination, length, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk)
	else:
	results["Quran"] = []

	if include_hindu:
	results["Rig Veda"] = cached_process_json_files(hindu.process_json_files, 1, 10, step, rounds_combination, length, tlang, False, strip_in_braces, strip_diacritics_chk)
	else:
	results["Rig Veda"] = []

	if include_tripitaka:
	results["Tripitaka"] = cached_process_json_files(tripitaka.process_json_files, 1, 52, step, rounds_combination, length, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk)
	else:
	results["Tripitaka"] = []

	return results


	def add_24h_projection(results_dict): #Now takes a dictionary of results
	for book_name, results in results_dict.items(): # Iterate per book
	num_results = len(results)
	if num_results > 0:
	time_interval = timedelta(minutes=24 * 60 / num_results)
	current_time = datetime.min.time()
	for i in range(num_results):
	next_time = (datetime.combine(datetime.min, current_time) + time_interval).time()
	time_range_str = f"{current_time.strftime('%H:%M')}-{next_time.strftime('%H:%M')}"
	results[i]['24h Projection'] = time_range_str
	current_time = next_time
	return results_dict


	def add_monthly_projection(results_dict, selected_date):
	if selected_date is None:
	return results_dict # Return if no date is selected

	for book_name, results in results_dict.items(): # Iterate per book
	num_results = len(results)
	if num_results > 0:
	days_in_month = calendar.monthrange(selected_date.year, selected_date.month)[1]
	total_seconds = (days_in_month - 1) * 24 * 3600
	seconds_interval = total_seconds / num_results
	start_datetime = datetime(selected_date.year, selected_date.month, 1)
	current_datetime = start_datetime


	for i in range(num_results):
	next_datetime = current_datetime + timedelta(seconds=seconds_interval)
	current_date = current_datetime.date() # Moved assignment inside loop
	next_date = next_datetime.date()
	date_range_str = f"{current_date.strftime('%h %d')} - {next_date.strftime('%h %d')}"
	results[i]['Monthly Projection'] = date_range_str
	current_datetime = next_datetime # Add this
	current_date = next_datetime.date() # Add this too
	return results_dict


	def add_yearly_projection(results_dict, selected_date): #Correct name, handle dictionary input
	if selected_date is None:
	return results_dict # Return if no date is selected

	for book_name, results in results_dict.items(): # Iterate per book
	num_results = len(results)
	if num_results > 0:
	days_in_year = 366 if calendar.isleap(selected_date.year) else 365
	total_seconds = (days_in_year - 1) * 24 * 3600
	seconds_interval = total_seconds / num_results
	start_datetime = datetime(selected_date.year, 1, 1)
	current_datetime = start_datetime


	for i in range(num_results):
	next_datetime = current_datetime + timedelta(seconds=seconds_interval)
	current_date = current_datetime.date() # Move assignment inside loop
	next_date = next_datetime.date()
	date_range_str = f"{current_date.strftime('%b %d')} - {next_date.strftime('%b %d')}"
	results[i]['Yearly Projection'] = date_range_str
	current_datetime = next_datetime # Update current datetime for next iteration

	return results_dict


	def sort_results(results):
	def parse_time(time_str):
	try:
	hours, minutes = map(int, time_str.split(':'))
	return hours * 60 + minutes # Convert to total minutes
	except ValueError:
	return 24 * 60 # Sort invalid times to the end

	return sorted(results, key=lambda x: (
	parse_time(x.get('24h Projection', '23:59').split('-')[0]), # Sort by start time first
	parse_time(x.get('24h Projection', '23:59').split('-')[1]) # Then by end time
	))

	# --- Main Gradio App ---
	with gr.Blocks() as app:
	with gr.Column():
	with gr.Row():
	tlang = create_language_dropdown("Target Language for Result Translation", default_value='english')
	selected_date = Calendar(type="datetime", label="Date to investigate (optional)", info="Pick a date from the calendar")
	use_day = gr.Checkbox(label="Use Day", info="Check to include day in search", value=True)
	use_month = gr.Checkbox(label="Use Month", info="Check to include month in search", value=True)
	use_year = gr.Checkbox(label="Use Year", info="Check to include year in search", value=True)
	date_language_input = create_language_dropdown("Language of the person/topic (optional) (Date Word Language)", default_value='english')
	with gr.Row():
	gematria_text = gr.Textbox(label="Name and/or Topic (required)", value="Hans Albert Einstein Mileva Marity-Einstein")
	date_words_output = gr.Textbox(label="Date in Words Translated (optional)")
	gematria_result = gr.Number(label="Journal Sum")
	#with gr.Row():


	with gr.Row():
	step = gr.Number(label="Jump Width (Steps) for ELS")
	float_step = gr.Number(visible=False, value=1)
	half_step_btn = gr.Button("Steps / 2")
	double_step_btn = gr.Button("Steps * 2")

	with gr.Column():
	round_x = gr.Number(label="Round (1)", value=1)
	round_y = gr.Number(label="Round (2)", value=-1)

	rounds_combination = gr.Textbox(label="Combined Rounds", value="1,-1")

	with gr.Row():
	include_torah_chk = gr.Checkbox(label="Include Torah", value=True)
	include_bible_chk = gr.Checkbox(label="Include Bible", value=True)
	include_quran_chk = gr.Checkbox(label="Include Quran", value=True)
	include_hindu_chk = gr.Checkbox(label="Include Rigveda", value=True)
	include_tripitaka_chk = gr.Checkbox(label="Include Tripitaka", value=True)
	merge_results_chk = gr.Checkbox(label="Merge Results (Torah-Bible-Quran)", value=True)

	strip_spaces = gr.Checkbox(label="Strip Spaces from Books", value=True)
	strip_in_braces = gr.Checkbox(label="Strip Text in Braces from Books", value=True)
	strip_diacritics_chk = gr.Checkbox(label="Strip Diacritics from Books", value=True)

	translate_btn = gr.Button("Search with ELS")

	# --- Output Components ---
	markdown_output = gr.Dataframe(label="ELS Results")
	most_frequent_phrase_output = gr.Textbox(label="Most Frequent Phrase in Network Search")
	json_output = gr.JSON(label="JSON Output")

	# --- Event Handlers ---

	def update_date_words(selected_date, date_language_input, use_day, use_month, use_year):
	if selected_date is None:
	return ""

	if not use_year and not use_month and not use_day:
	return translate_date_to_words(selected_date, date_language_input)

	year = selected_date.year if use_year else None
	month = selected_date.month if use_month else None
	day = selected_date.day if use_day else None

	if year is not None and month is not None and day is not None:
	date_obj = selected_date
	elif year is not None and month is not None:
	date_obj = str(f"{year}-{month}")
	elif year is not None:
	date_obj = str(f"{year}")
	else: # Return empty string if no date components are selected
	return ""


	date_in_words = date_to_words(date_obj)


	translator = GoogleTranslator(source='auto', target=date_language_input)
	translated_date_words = translator.translate(date_in_words)
	return custom_normalize(translated_date_words)

	def update_journal_sum(gematria_text, date_words_output):
	sum_value = calculate_gematria_sum(gematria_text, date_words_output)
	return sum_value, sum_value, sum_value

	def update_rounds_combination(round_x, round_y):
	return f"{int(round_x)},{int(round_y)}"

	def update_step_half(float_step):
	new_step = math.ceil(float_step / 2)
	return new_step, float_step / 2

	def update_step_double(float_step):
	new_step = math.ceil(float_step * 2)
	return new_step, float_step * 2


	def find_closest_phrase(target_phrase, phrases):
	best_match = None
	best_score = 0

	logging.debug(f"Target phrase for similarity search: {target_phrase}") # Log target phrase

	for phrase, _, _, _, _, _ in phrases:
	word_length_diff = abs(len(target_phrase.split()) - len(phrase.split()))
	similarity_score = fuzz.ratio(target_phrase, phrase)
	combined_score = similarity_score - word_length_diff

	logging.debug(f"Comparing with phrase: {phrase}") # Log each phrase being compared
	logging.debug(
	f"Word Length Difference: {word_length_diff}, Similarity Score: {similarity_score}, Combined Score: {combined_score}") # Log scores

	if combined_score > best_score:
	best_score = combined_score
	best_match = phrase

	logging.debug(f"Closest phrase found: {best_match} with score: {best_score}") # Log the best match
	return best_match

	def perform_search(step, rounds_combination, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk, include_torah, include_bible, include_quran, include_hindu, include_tripitaka, gematria_text, date_words_output, selected_date):
	# Inside perform_search
	els_results = perform_els_search(step, rounds_combination, tlang, strip_spaces, strip_in_braces,
	strip_diacritics_chk, include_torah, include_bible, include_quran,
	include_hindu,
	include_tripitaka)

	# --- Network Search Integration ---
	most_frequent_phrases = {}
	combined_and_sorted_results = [] # Combined list to hold all results

	for book_name, book_results in els_results.items():
	if book_results: # Add this check to ensure book_results is not empty
	most_frequent_phrases[book_name] = "" # Default value

	for result in book_results:
	try:
	gematria_sum = calculate_gematria(result['result_text']) # Calculate gematria
	max_words = len(result['result_text'].split())
	matching_phrases = search_gematria_in_db(gematria_sum, max_words)
	max_words_limit = 20
	while not matching_phrases and max_words < max_words_limit: # Increase max_words for more results
	max_words += 1
	matching_phrases = search_gematria_in_db(gematria_sum, max_words)

	if matching_phrases:
	most_frequent_phrase = get_most_frequent_phrase(matching_phrases)
	most_frequent_phrases[book_name] = most_frequent_phrase
	else:
	closest_phrase = find_closest_phrase(result['result_text'],
	search_gematria_in_db(gematria_sum, max_words_limit))
	most_frequent_phrases[
	book_name] = closest_phrase or "" # Update most frequent phrases even if no phrase found

	result['Most Frequent Phrase'] = most_frequent_phrases[book_name]
	if 'book' in result:
	if isinstance(result['book'], int): # Torah, Bible, Quran case
	result['book'] = f"{book_name} {result['book']}."
	combined_and_sorted_results.append(result)

	except KeyError as e:
	print(f"DEBUG: KeyError - Key '{e.args[0]}' not found in result. Skipping this result.")
	continue

	# --- Batch Translation ---
	selected_language_long = tlang
	tlang_short = LANGUAGES_SUPPORTED.get(selected_language_long)
	if tlang_short is None:
	tlang_short = "en"
	logger.warning(f"Unsupported language selected: {selected_language_long}. Defaulting to English (en).")

	# Prepare lists for batch translation, including source language
	phrases_to_translate = []
	phrases_source_langs = [] # Source languages for phrases
	results_to_translate = []
	results_source_langs = [] # Source languages for results
	for result in combined_and_sorted_results:
	phrases_to_translate.append(result.get('Most Frequent Phrase', ''))
	# Always use 'iw' as the source language for "Most Frequent Phrase"
	phrases_source_langs.append("iw")
	results_to_translate.append(result.get('result_text', ''))
	results_source_langs.append(result.get("source_language", "auto"))

	translated_phrases = translation_utils.batch_translate(phrases_to_translate, tlang_short, phrases_source_langs)
	translated_result_texts = translation_utils.batch_translate(results_to_translate, tlang_short, results_source_langs)


	for i, result in enumerate(combined_and_sorted_results):
	result['translated_text'] = translated_result_texts.get(results_to_translate[i], None)
	result['Translated Most Frequent Phrase'] = translated_phrases.get(phrases_to_translate[i], None)

	# Time Projections (using els_results dictionary)
	updated_els_results = add_24h_projection(els_results) # Use original els_results dictionary
	updated_els_results = add_monthly_projection(updated_els_results, selected_date) # Call correct functions with correct params
	updated_els_results = add_yearly_projection(updated_els_results, selected_date)

	combined_and_sorted_results = []
	for book_results in updated_els_results.values(): # Combine results for dataframe and json
	combined_and_sorted_results.extend(book_results)
	combined_and_sorted_results = sort_results(combined_and_sorted_results) # sort combined results

	df = pd.DataFrame(combined_and_sorted_results)
	df.index = range(1, len(df) + 1)
	df.reset_index(inplace=True)
	df.rename(columns={'index': 'Result Number'}, inplace=True)

	for i, result in enumerate(combined_and_sorted_results): # Iterate through the combined list
	result['Result Number'] = i + 1

	search_config = {
	"step": step,
	"rounds_combination": rounds_combination,
	"target_language": tlang,
	"strip_spaces": strip_spaces,
	"strip_in_braces": strip_in_braces,
	"strip_diacritics": strip_diacritics_chk,
	"include_torah": include_torah,
	"include_bible": include_bible,
	"include_quran": include_quran,
	"include_hindu": include_hindu,
	"include_tripitaka": include_tripitaka,
	"gematria_text": gematria_text,
	"date_words": date_words_output
	}

	output_data = {
	"search_configuration": search_config,
	"results": combined_and_sorted_results # Use the combined list here
	}

	json_data = output_data

	# --- Return results ---
	combined_most_frequent = "\n".join(
	f"{book}: {phrase}" for book, phrase in most_frequent_phrases.items()) # Combine phrases
	return df, combined_most_frequent, json_data



	# --- Event Triggers ---
	round_x.change(update_rounds_combination, inputs=[round_x, round_y], outputs=rounds_combination)
	round_y.change(update_rounds_combination, inputs=[round_x, round_y], outputs=rounds_combination)

	selected_date.change(update_date_words, inputs=[selected_date, date_language_input, use_day, use_month, use_year], outputs=[date_words_output])
	date_language_input.change(update_date_words, inputs=[selected_date, date_language_input, use_day, use_month, use_year], outputs=[date_words_output])

	gematria_text.change(update_journal_sum, inputs=[gematria_text, date_words_output], outputs=[gematria_result, step, float_step])
	date_words_output.change(update_journal_sum, inputs=[gematria_text, date_words_output], outputs=[gematria_result, step, float_step])

	half_step_btn.click(update_step_half, inputs=[float_step], outputs=[step, float_step])
	double_step_btn.click(update_step_double, inputs=[float_step], outputs=[step, float_step])

	translate_btn.click(
	perform_search,
	inputs=[step, rounds_combination, tlang, strip_spaces, strip_in_braces, strip_diacritics_chk, include_torah_chk, include_bible_chk, include_quran_chk, include_hindu_chk, include_tripitaka_chk, gematria_text, date_words_output, selected_date],
	outputs=[markdown_output, most_frequent_phrase_output, json_output]
	)

	app.load(
	update_date_words,
	inputs=[selected_date, date_language_input, use_day, use_month, use_year], # Include all 5 inputs
	outputs=[date_words_output]
	)

	use_day.change(
	update_date_words,
	inputs=[selected_date, date_language_input, use_day, use_month, use_year],
	outputs=[date_words_output]
	)
	use_month.change(
	update_date_words,
	inputs=[selected_date, date_language_input, use_day, use_month, use_year],
	outputs=[date_words_output]
	)
	use_year.change(
	update_date_words,
	inputs=[selected_date, date_language_input, use_day, use_month, use_year],
	outputs=[date_words_output]
	)

	def checkbox_behavior(use_day_value, use_month_value):
	if use_day_value: # Tick month and year automatically when day is ticked.
	return True, True

	return use_month_value, True # return month value unchanged and automatically tick year if month is checked

	use_day.change(checkbox_behavior, inputs=[use_day, use_month], outputs=[use_month, use_year])
	use_month.change(checkbox_behavior, inputs=[use_day, use_month], outputs=[use_month, use_year]) #No need for use_day here, day won't be changed by month


	if __name__ == "__main__":
	app.launch(share=False)

	[File Ends] app.py

	[File Begins] bible.py
	import logging
	import json
	import os
	import re
	from deep_translator import GoogleTranslator
	from gematria import calculate_gematria
	import math
	import csv

	# Configure the logger
	# You can uncomment the next line to enable debugging logs
	# logging.basicConfig(level=logging.DEBUG, format='%(levelname)s:%(message)s')
	logger = logging.getLogger(__name__)

	def process_json_files(start=40, end=66, step=1, rounds="1", length=0, tlang="en", strip_spaces=True,
	strip_in_braces=True, strip_diacritics=True, translate=False):
	"""
	Process a CSV file containing biblical texts and perform various text manipulations.

	Parameters:
	- start (int): Starting book number.
	- end (int): Ending book number.
	- step (int): Step value for character selection.
	- rounds (str): Comma-separated string of round values (can include floats).
	- length (int): Maximum length of the result text.
	- tlang (str): Target language for translation.
	- strip_spaces (bool): Whether to remove spaces from the text.
	- strip_in_braces (bool): Whether to remove text within braces.
	- strip_diacritics (bool): Whether to remove diacritics from the text.
	- translate (bool): Whether to translate the result text.

	Returns:
	- list: A list of dictionaries containing processed data or error messages.
	"""
	file_name = "texts/bible/OpenGNT_version3_3.csv"
	translator = GoogleTranslator(source='auto', target=tlang) if translate else None
	results = []

	# Dictionary for the 27 books of the New Testament (English names)
	nt_books = {
	40: "Matthew",
	41: "Mark",
	42: "Luke",
	43: "John",
	44: "Acts",
	45: "Romans",
	46: "1 Corinthians",
	47: "2 Corinthians",
	48: "Galatians",
	49: "Ephesians",
	50: "Philippians",
	51: "Colossians",
	52: "1 Thessalonians",
	53: "2 Thessalonians",
	54: "1 Timothy",
	55: "2 Timothy",
	56: "Titus",
	57: "Philemon",
	58: "Hebrews",
	59: "James",
	60: "1 Peter",
	61: "2 Peter",
	62: "1 John",
	63: "2 John",
	64: "3 John",
	65: "Jude",
	66: "Revelation"
	}

	try:
	with open(file_name, 'r', encoding='utf-8') as file:
	reader = csv.DictReader(file, delimiter='\t')

	book_texts = {}
	current_book = None
	for row_num, row in enumerate(reader, start=1):
	try:
	# Parse the book number from '〔Book｜Chapter｜Verse〕' field
	book_field = row['〔Book｜Chapter｜Verse〕']
	book_str = book_field.split('｜')[0] # e.g., '〔40'
	book_num_str = book_str.lstrip('〔') # Remove leading '〔'
	book = int(book_num_str)

	if book < start or book > end:
	continue

	if current_book != book:
	current_book = book
	book_texts[book] = ""

	# Parse the Greek text from '〔OGNTk｜OGNTu｜OGNTa｜lexeme｜rmac｜sn〕' field
	greek_field = row['〔OGNTk｜OGNTu｜OGNTa｜lexeme｜rmac｜sn〕']
	# Extract the first part before '〔' and split by '｜'
	if '〔' in greek_field:
	greek_text = greek_field.split('〔')[1]
	greek_text = greek_text.split('｜')[0]
	else:
	greek_text = greek_field.split('｜')[0]

	book_texts[book] += greek_text + " "

	except (KeyError, IndexError, ValueError) as e:
	logger.error(f"Error parsing row {row_num}: {e}")
	continue # Skip this row and continue

	for book, full_text in book_texts.items():
	logger.debug(f"Processing book {book}")
	clean_text = full_text
	if strip_in_braces:
	clean_text = re.sub(r"\[.?\]\|\{.?\}\|\<.*?\>", "", clean_text, flags=re.DOTALL)

	if strip_diacritics:
	# Adjusted regex for Greek diacritics
	clean_text = re.sub(r"[^\u0370-\u03FF\u1F00-\u1FFF ]+", "", clean_text)
	# Optionally, remove specific diacritics or punctuation if needed
	# clean_text = re.sub(r'[additional patterns]', '', clean_text)

	# Normalize spaces
	clean_text = clean_text.replace("\n\n ", " ")
	clean_text = clean_text.replace("\n", " ")
	clean_text = re.sub(r'\s+', ' ', clean_text).strip()

	if strip_spaces:
	clean_text = clean_text.replace(" ", "")

	text_length = len(clean_text)
	logger.debug(f"Clean text for book {book}: Length = {text_length}")

	if text_length == 0:
	logger.warning(f"No text available for book {book} after cleaning.")
	continue # Skip processing if there's no text

	try:
	rounds_list = list(map(float, rounds.split(','))) # Allow floats
	except ValueError as e:
	logger.error(f"Invalid rounds parameter: {e}")
	return [{"error": f"Invalid rounds parameter: {e}"}]

	result_text = ""

	for r in rounds_list:
	abs_r = abs(r)

	# Determine the number of full passes and the remainder.
	full_passes = math.floor(abs_r)
	remainder = abs_r - full_passes

	# Base number of characters per pass
	base_chars = text_length // step

	if base_chars == 0:
	if abs_r > 1: # Changed from >=1 to >1
	# When step > text_length and rounds >1, pick 1 character per full pass
	chars_per_full_pass = 1
	logger.debug(f"Book {book}: step > text_length ({step} > {text_length}), selecting 1 character per full pass.")
	else:
	# No characters to pick
	chars_per_full_pass = 0
	logger.debug(f"Book {book}: step > text_length ({step} > {text_length}) and rounds <=1, no characters selected.")
	# For remainder, since base_chars=0, no remainder characters
	chars_for_remainder = 0
	else:
	# Normal case
	chars_per_full_pass = base_chars
	chars_for_remainder = math.floor(base_chars * remainder) # Partial pass
	logger.debug(f"Book {book}: Normal case, chars_per_full_pass = {chars_per_full_pass}, chars_for_remainder = {chars_for_remainder}")

	if r > 0:
	current_index = (step - 1) % text_length
	direction = 1
	else:
	current_index = (text_length - step) % text_length
	direction = -1

	pass_result = ""

	# Full passes, keep only the last pass
	for pass_num in range(1, full_passes + 1):
	current_pass_chars = ""
	for _ in range(chars_per_full_pass):
	if chars_per_full_pass == 0:
	break
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length

	# Keep only the last full pass
	if pass_num == full_passes:
	pass_result = current_pass_chars
	logger.debug(f"Book {book}: Pass {pass_num}, pass_result = {pass_result}")

	# Remainder pass for fractional rounds
	if remainder > 0 and chars_for_remainder > 0:
	current_pass_chars = ""
	for _ in range(chars_for_remainder):
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length
	pass_result += current_pass_chars
	logger.debug(f"Book {book}: Remainder pass_result = {pass_result}")

	# Handle cases where step > text_length and chars_per_full_pass=1
	if base_chars == 0 and chars_per_full_pass == 1 and full_passes > 0:
	# pass_result already contains the last character picked
	pass
	elif base_chars == 0 and chars_per_full_pass == 0 and full_passes > 0:
	# When no characters are picked, skip appending
	pass

	result_text += pass_result

	logger.debug(f"Result text for book {book}: {result_text}")

	if length != 0:
	result_text = result_text[:length]
	logger.debug(f"Book {book}: Result text truncated to length {length}.")

	# Translate the result text if required
	try:
	translated_text = translator.translate(result_text) if translator and result_text else ""
	except Exception as e:
	logger.error(f"Book {book}: Translation error: {e}")
	translated_text = ""

	# Calculate the Gematria sum
	try:
	result_sum = calculate_gematria(result_text)
	except Exception as e:
	logger.error(f"Book {book}: Gematria calculation error: {e}")
	result_sum = None

	if result_text:
	result = {
	'book': f"Bible {book}.",
	'title': nt_books.get(book, "Unknown Book"),
	'result_text': result_text,
	'result_sum': result_sum,
	'translated_text': translated_text
	}
	results.append(result)

	except FileNotFoundError:
	logger.error(f"File {file_name} not found.")
	results.append({"error": f"File {file_name} not found."})
	except Exception as e:
	logger.error(f"An unexpected error occurred: {e}")
	results.append({"error": f"An unexpected error occurred: {e}"})

	return results if results else None


	# Tests
	test_results = [
	(process_json_files(40,40,386,rounds="1,0.5,-1,-0.5"), "τωιεοννναυοοαμπυρααυοιξοηαϲοιιωομκνοοουομρυοιεχοοδεαλαννοτοκϲααυϲϲτεαδαϲαιεευαιηαεηαμαλκμαιακιγνμυνετνυυθθεγιεδρεαοαντηοκεοατϲνπναολαεοοφεηεϲωμκουμερρυοϲαοοαϲλτιηιωδυνϲυτυιχοονεαηωντολθβοτεαιαυοηετιτεαννυεινϲεενωκωξρυρηρνϲξεαγεαϲατωιεοννναυοοαμπυρααυοιξοηαϲοιιωομκνοοουομρυοιεχοοδεαλαννοτοκϲααυϲϲτεαδαϲαιεευαιηαεηαμαλκμαιακιγνμυνετνυυθθεγιεδρεαοϲτοαϲωθειιυνδνδξλονυταολαϲαττμτννοερτεοροανιεκτεεαιιϲωεαμϲωικμτποκϲϲιορϲπμοκαιουτωτδοωαξεαγωεφτωυπμαλυττταεομττλυαεπατονεαξτομυχαωηνυοβωπτυυκξαπιτααπυενεροοτϲαααυηοανηλταιθεαντινοιλκιβπγοδαιοηωαδαετακυϲοηυουαυνωαεαοττυαεεωτανεκβγεϲτοαϲωθειιυνδνδξλονυταολαϲαττμτννοερτεοροανιεκτεεαιιϲωεαμϲωικμτποκϲϲιορϲπμοκαιουτωτδοωαξεαγωεφτωυπμαλυττταεομττλυαε"),
	(process_json_files(40,40,386,rounds="1,-1"), "τωιεοννναυοοαμπυρααυοιξοηαϲοιιωομκνοοουομρυοιεχοοδεαλαννοτοκϲααυϲϲτεαδαϲαιεευαιηαεηαμαλκμαιακιγνμυνετνυυθθεγιεδρεαοαντηοκεοατϲνπναολαεοοφεηεϲωμκουμερρυοϲαοοαϲλτιηιωδυνϲυτυιχοονεαηωντολθβοτεαιαυοηετιτεαννυεινϲεενωκωξρυρηρνϲξεαγεαϲαϲτοαϲωθειιυνδνδξλονυταολαϲαττμτννοερτεοροανιεκτεεαιιϲωεαμϲωικμτποκϲϲιορϲπμοκαιουτωτδοωαξεαγωεφτωυπμαλυττταεομττλυαεπατονεαξτομυχαωηνυοβωπτυυκξαπιτααπυενεροοτϲαααυηοανηλταιθεαντινοιλκιβπγοδαιοηωαδαετακυϲοηυουαυνωαεαοττυαεεωτανεκβγε"),
	#(process_json_files(1, 1, 21, rounds="3", length=0), ""),
	#(process_json_files(1, 1, 22, rounds="1", length=0), ""),
	#(process_json_files(1, 1, 22, rounds="3", length=0), ""),
	#(process_json_files(1, 1, 23, rounds="3", length=0), ""),
	#(process_json_files(1, 1, 11, rounds="1", length=0), ""),
	#(process_json_files(1, 1, 2, rounds="1", length=0), ""),
	#(process_json_files(1, 1, 23, rounds="1", length=0), None), # Expect None, when no results
	#(process_json_files(1, 1, 23, rounds="-1", length=0), None), # Expect None, when no results
	#(process_json_files(1, 1, 22, rounds="-1", length=0), ""),
	#(process_json_files(1, 1, 22, rounds="-2", length=0), ""),
	#(process_json_files(1, 1, 1, rounds="-1", length=0), ""), # Reversed Hebrew alphabet
	#(process_json_files(1, 1, 1, rounds="1,-1", length=0), ""), # Combined rounds
	#(process_json_files(1, 1, 22, rounds="1,-1", length=0, average_compile=True), ""), # average compile test (400+1) / 2 = math.ceil(200.5)=201=200+1="רא"
	]

	all_tests_passed = True
	for result, expected in test_results:
	if expected is None: # Check if no result is expected
	if not result:
	logger.warning(f"Test passed: Expected no results, got no results.")
	else:
	logger.error(f"Test failed: Expected no results, but got: {result}")
	all_tests_passed = False
	else:
	# Check if result is not empty before accessing elements
	if result:
	result_text = result[0]['result_text']
	if result_text == expected:
	logger.warning(f"Test passed: Expected '{expected}', got '{result_text}'")
	else:
	logger.error(f"Test failed: Expected '{expected}', but got '{result_text}'")
	all_tests_passed = False
	else:
	logger.error(f"Test failed: Expected '{expected}', but got no results")
	all_tests_passed = False

	if all_tests_passed:
	logger.info("All round tests passed.")


	[File Ends] bible.py

	[File Begins] database-structure.txt
	Gematria Sum, Words, Translation, Book, Chapter, Verse, Phrase Length, Phrase Position
	913 בראשית Genesis 1 1 1 1-1
	1116 בראשית ברא Genesis 1 1 2 1-2
	1762 בראשית ברא אלהים Genesis 1 1 3 1-3
	2163 בראשית ברא אלהים את Genesis 1 1 4 1-4
	3118 בראשית ברא אלהים את השמים Genesis 1 1 5 1-5
	3525 בראשית ברא אלהים את השמים ואת Genesis 1 1 6 1-6

	[File Ends] database-structure.txt

	[File Begins] els_cache.db.initial
	SQLite format 3@ .zp
	33mtableels_cacheels_cacheCREATE TABLE els_cache (
	query_hash TEXT PRIMARY KEY,
	results TEXT
	)1Eindexsqlite_autoindex_els_cache_1els_cache


	[File Ends] els_cache.db.initial

	[File Begins] gematria.py
	import unicodedata
	import logging

	logger = logging.getLogger(__name__)

	def strip_diacritics(text):
	"""
	Entfernt Diakritika von Unicode-Zeichen, um den Basisbuchstaben zu erhalten, und gibt Warnungen
	für tatsächlich unbekannte Zeichen aus.
	"""
	stripped_text = ''
	for char in unicodedata.normalize('NFD', text):
	if unicodedata.category(char) not in ['Mn', 'Cf']:
	stripped_text += char
	else:
	logger.debug(f"Info: Diakritisches Zeichen '{char}' wird ignoriert.")
	return stripped_text

	def letter_to_value(letter):
	"""
	Konvertiert einen einzelnen Buchstaben in seinen Gematria-Wert, ignoriert Leerzeichen
	und Nicht-Buchstaben-Zeichen.
	"""
	# Dein vorhandenes Wörterbuch bleibt unverändert
	values = {
	# Lateinische Buchstaben
	'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f': 6, 'g': 7, 'h': 8, 'i': 9, 'j': 600,
	'k': 10, 'l': 20, 'm': 30, 'n': 40, 'o': 50, 'p': 60, 'q': 70, 'r': 80, 's': 90,
	't': 100, 'u': 200, 'v': 700, 'w': 900, 'x': 300, 'y': 400, 'z': 500,

	'A': 1, 'B': 2, 'C': 3, 'D': 4, 'E': 5, 'F': 6, 'G': 7, 'H': 8, 'I': 9, 'J': 600,
	'K': 10, 'L': 20, 'M': 30, 'N': 40, 'O': 50, 'P': 60, 'Q': 70, 'R': 80, 'S': 90,
	'T': 100, 'U': 200, 'V': 700, 'W': 900, 'X': 300, 'Y': 400, 'Z': 500,

	# Basisbuchstaben und einige bereits genannte Varianten
	'ا': 1, 'أ': 1, 'إ': 1, 'آ': 1, 'ب': 2, 'ج': 3, 'د': 4, 'ه': 5, 'و': 6, 'ز': 7, 'ح': 8, 'ط': 9,
	'ي': 10, 'ى': 10, 'ك': 20, 'ک': 20, 'ل': 30, 'م': 40, 'ن': 50, 'س': 60, 'ع': 70, 'ف': 80,
	'ص': 90, 'ق': 100, 'ر': 200, 'ش': 300, 'ت': 400, 'ث': 500, 'خ': 600, 'ذ': 700, 'ض': 800, 'ظ': 900, 'غ': 1000,
	'ٱ': 1, # Alif Wasla
	'ـ': 0, # Tatweel

	# Zusätzliche Varianten und Sonderzeichen
	'ة': 400, # Taa Marbuta
	'ؤ': 6, # Waw mit Hamza darüber
	'ئ': 10, # Ya mit Hamza darüber
	'ء': 1, # Hamza
	'ى': 10, # Alif Maqsurah
	'ٹ': 400, # Taa' marbuta goal
	'پ': 2, # Pe (Persisch/Urdu)
	'چ': 3, # Che (Persisch/Urdu)
	'ژ': 7, # Zhe (Persisch/Urdu)
	'گ': 20, # Gaf (Persisch/Urdu)
	'ڭ': 20, # Ngaf (Kazakh, Uyghur, Uzbek, and in some Arabic dialects)
	'ں': 50, # Noon Ghunna (Persisch/Urdu)
	'ۀ': 5, # Heh with Yeh above (Persisch/Urdu)
	'ے': 10, # Barree Yeh (Persisch/Urdu)
	'؋': 0, # Afghani Sign (wird als Währungssymbol verwendet, nicht für Gematria relevant, aber hier zur Vollständigkeit aufgeführt)

	# Anmerkung: Das Währungssymbol und ähnliche Zeichen sind in einem Gematria-Kontext normalerweise nicht relevant,
	# werden aber der Vollständigkeit halber aufgeführt. Es gibt noch viele weitere spezifische Zeichen in erweiterten
	# arabischen Schriftsystemen (z.B. für andere Sprachen wie Persisch, Urdu, Pashto usw.), die hier nicht vollständig
	# abgedeckt sind.

	# Grund- und Schlussformen hebräischer Buchstaben

	'א': 1, 'ב': 2, 'ג': 3, 'ד': 4, 'ה': 5, 'ו': 6, 'ז': 7, 'ח': 8, 'ט': 9, 'י': 10,
	'כ': 20, 'ך': 500, 'ל': 30, 'מ': 40, 'ם': 600, 'נ': 50, 'ן': 700, 'ס': 60, 'ע': 70, 'פ': 80, 'ף': 800,
	'צ': 90, 'ץ': 900, 'ק': 100, 'ר': 200, 'ש': 300, 'ת': 400,

	# Griechische Buchstaben
	'α': 1, 'β': 2, 'γ': 3, 'δ': 4, 'ε': 5, 'ϝ': 6, 'ζ': 7, 'η': 8, 'θ': 9, 'ι': 10,
	'κ': 20, 'λ': 30, 'μ': 40, 'ν': 50, 'ξ': 60, 'ο': 70, 'π': 80, 'ϟ': 90, 'ρ': 100,
	'σ': 200, 'τ': 300, 'υ': 400, 'φ': 500, 'χ': 600, 'ψ': 700, 'ω': 800, 'ϡ': 900,

	# Griechische Großbuchstaben
	'Α': 1, 'Β': 2, 'Γ': 3, 'Δ': 4, 'Ε': 5, 'Ϝ': 6, 'Ζ': 7, 'Η': 8, 'Θ': 9, 'Ι': 10,
	'Κ': 20, 'Λ': 30, 'Μ': 40, 'Ν': 50, 'Ξ': 60, 'Ο': 70, 'Π': 80, 'Ϟ': 90, 'Ρ': 100,
	'Σ': 200, 'Τ': 300, 'Υ': 400, 'Φ': 500, 'Χ': 600, 'Ψ': 700, 'Ω': 800, 'Ϡ': 900,
	'σ': 200, # Sigma
	'ς': 200, # Final Sigma
	'ϲ': 200, # Lunate Sigma (Greek)
	'Ϲ': 200, # Uppercase Lunate Sigma (Greek)

	# Katapayadi System (Comprehensive with variants)
	'क': 1, 'ख': 2, 'ग': 3, 'घ': 4, 'ङ': 5,
	'च': 6, 'छ': 7, 'ज': 8, 'झ': 9, 'ञ': 0, # Or placeholder for zero if appropriate
	'ट': 1, 'ठ': 2, 'ड': 3, 'ढ': 4, 'ण': 5,
	'त': 6, 'थ': 7, 'द': 8, 'ध': 9, 'न': 0, # Or placeholder for zero if appropriate
	'प': 1, 'फ': 2, 'ब': 3, 'भ': 4, 'म': 5,
	'य': 1, 'र': 2, 'ल': 3, 'व': 4, 'श': 5, 'ष': 6, 'स': 7, 'ह': 8,

	# Half forms (same values)
	'क्': 1, 'ख्': 2, 'ग्': 3, 'घ्': 4, 'ङ्': 5,
	'च्': 6, 'छ्': 7, 'ज्': 8, 'झ्': 9, 'ञ्': 0,
	'ट्': 1, 'ठ्': 2, 'ड्': 3, 'ढ्': 4, 'ण्': 5,
	'त्': 6, 'थ्': 7, 'द्': 8, 'ध्': 9, 'न्': 0,
	'प्': 1, 'फ्': 2, 'ब्': 3, 'भ्': 4, 'म्': 5,
	'य्': 1, 'र्': 2, 'ल्': 3, 'व्': 4, 'श्': 5, 'ष्': 6, 'स्': 7, 'ह्': 8,

	# Nukta forms (assuming same values - verify)
	'क़': 1, 'ख़': 2, 'ग़': 3, 'ज़': 8, 'ड़': 3, 'ढ़': 4, 'फ़': 2,

	# Kannada, Telugu, Malayalam equivalents
	'ಕ': 1, 'ಖ': 2, 'ಗ': 3, 'ಘ': 4, 'ಙ': 5, 'ಚ': 6, 'ಛ': 7, 'ಜ': 8, 'ಝ': 9, 'ಞ': 0,
	'ಟ': 1, 'ಠ': 2, 'ಡ': 3, 'ಢ': 4, 'ಣ': 5, 'ತ': 6, 'ಥ': 7, 'ದ': 8, 'ಧ': 9, 'ನ': 0,
	'ಪ': 1, 'ಫ': 2, 'ಬ': 3, 'ಭ': 4, 'ಮ': 5, 'ಯ': 1, 'ರ': 2, 'ಲ': 3, 'ವ': 4, 'ಶ': 5, 'ಷ': 6, 'ಸ': 7, 'ಹ': 8,

	# Malayalam
	'ക': 1, 'ഖ': 2, 'ഗ': 3, 'ഘ': 4, 'ങ': 5,
	'ച': 6, 'ഛ': 7, 'ജ': 8, 'ഝ': 9, 'ഞ': 0,
	'ട': 1, 'ഠ': 2, 'ഡ': 3, 'ഢ': 4, 'ണ': 5,
	'ത': 6, 'ഥ': 7, 'ദ': 8, 'ധ': 9, 'ന': 0,
	'പ': 1, 'ഫ': 2, 'ബ': 3, 'ഭ': 4, 'മ': 5,
	'യ': 1, 'ര': 2, 'ല': 3, 'വ': 4, 'ശ': 5, 'ഷ': 6, 'സ': 7, 'ഹ': 8,


	# Vokale (Svara)
	'अ': 0, 'आ': 0, 'इ': 0, 'ई': 0, 'उ': 0,
	'ऊ': 0, 'ऋ': 0, 'ॠ': 0, 'ऌ': 0, 'ॡ': 0,
	'ए': 0, 'ऐ': 0, 'ओ': 0, 'औ': 0,
	# Zusätzliche Zeichen
	'क्ष': 1, 'त्र': 6, 'ज्ञ': 8,

	# Anusvaras und Visargas
	'ं': 0, 'ः': 0,
	# Halante (Virama)
	'्': 0,
	# Ziffern
	'०': 0, '१': 1, '२': 2, '३': 3, '४': 4,
	'५': 5, '६': 6, '७': 7, '८': 8, '९': 9,
	# Sonderzeichen
	'ॐ': 0, # Om-Symbol
	# Vokalzeichen (Matra)
	'ा': 0, 'ि': 0, 'ी': 0, 'ु': 0, 'ू': 0,
	'ृ': 0, 'ॄ': 0, 'ॢ': 0, 'ॣ': 0, 'े': 0,
	'ै': 0, 'ो': 0, 'ौ': 0,
	# Zusätzliche Zeichen für vollständige Abdeckung
	'ॅ': 0, 'ॆ': 0, 'ॉ': 0, 'ॊ': 0, 'ऍ': 0,
	'ऎ': 0, 'ऑ': 0, 'ऒ': 0, 'ॎ': 0, 'ॏ': 0,
	# Vedische Erweiterungen
	'ᳵ': 0, 'ᳶ': 0, 'ॽ': 0,
	}

	# Stelle sicher, dass Diakritika entfernt werden, bevor auf das Wörterbuch zugegriffen wird
	letter_no_diacritics = strip_diacritics(letter)

	if letter_no_diacritics in values:
	return values[letter_no_diacritics.lower()]
	elif letter.strip() == "": # Ignoriere Leerzeichen und leere Zeilen
	return 0
	else:
	# Gib eine spezifische Warnung aus, wenn das Zeichen unbekannt ist
	logger.debug(f"Warnung: Unbekanntes Zeichen '{letter}' ignoriert.")
	return 0


	def calculate_gematria(text):
	"""Calculate the Gematria value of a given Hebrew text, ignoring spaces and non-Hebrew characters."""
	return sum(letter_to_value(letter) for letter in text if letter.strip() != "")

	[File Ends] gematria.py

	[File Begins] hindu.py
	import logging
	import json
	import re
	import math
	from deep_translator import GoogleTranslator
	from gematria import calculate_gematria # Stelle sicher, dass diese Funktion korrekt definiert ist

	# Konfiguriere den Logger
	#logging.basicConfig(level=logging.DEBUG, format='%(levelname)s:%(message)s')
	logger = logging.getLogger(__name__)

	def process_json_files(start=1, end=10, step=1, rounds="1", length=0, tlang="en", strip_spaces=True,
	strip_in_braces=True, strip_diacritics=True, translate=False):
	"""
	Processes Rigveda JSON files and performs various text manipulations.

	Parameters:
	- start (int): Start number of the Mandala.
	- end (int): End number of the Mandala.
	- step (int): Step size for character selection.
	- rounds (str): Comma-separated list of round numbers (can include floats).
	- length (int): Maximum length of the result text.
	- tlang (str): Target language for translation.
	- strip_spaces (bool): Whether to remove spaces.
	- strip_in_braces (bool): Whether to remove text within brackets.
	- strip_diacritics (bool): Whether to remove diacritics.
	- translate (bool): Whether to translate the result text.

	Returns:
	- list: A list of dictionaries with the processed data or error messages.
	"""
	base_path = "texts/rigveda"
	translator = GoogleTranslator(source='sa', target=tlang) if translate else None # Correct source language for Sanskrit
	results = []

	for i in range(start, end + 1):
	file_name = f"{base_path}/rigveda_mandala_{i:02}.json"
	# Optional, if needed:
	# file_name = f"{base_path}/mahabharata_book_{i:02}.json"
	try:
	with open(file_name, 'r', encoding='utf-8') as file:
	data = json.load(file)
	# Concatenate all Suktas for the current Mandala
	full_text = " ".join(sukta.get("text", "") for sukta in data)

	clean_text = full_text
	if strip_in_braces:
	# Remove content within [], {}, <> brackets
	clean_text = re.sub(r"\[.?\]\|\{.?\}\|\<.*?\>", "", clean_text, flags=re.DOTALL)

	if strip_diacritics:
	# Keep only Devanagari characters and spaces
	clean_text = re.sub(r'[^\u0900-\u097F\s]', '', clean_text)
	# Remove specific diacritics and punctuation
	clean_text = re.sub(r'[\u0951-\u0954\u0964\u0965]+', '', clean_text)
	# Remove Devanagari numerals
	clean_text = re.sub(r'[०१२३४५६७८९]+', '', clean_text)
	# Remove additional unwanted characters
	clean_text = re.sub(r'[:?!\'\-]', '', clean_text)

	# Normalize spaces
	clean_text = clean_text.replace("\n\n ", " ")
	clean_text = clean_text.replace("\n", " ")
	clean_text = re.sub(r'\s+', ' ', clean_text).strip()

	if strip_spaces:
	clean_text = clean_text.replace(" ", "")

	text_length = len(clean_text)
	logger.debug(f"Mandala {i}: Clean text length = {text_length}")

	if text_length == 0:
	logger.warning(f"Mandala {i}: No text available after cleaning.")
	continue # Skip processing if no text is available

	try:
	rounds_list = list(map(float, rounds.split(',')))
	except ValueError as e:
	logger.error(f"Mandala {i}: Invalid 'rounds' format: {rounds}")
	results.append({"error": f"Mandala {i}: Invalid 'rounds' format: {rounds}"})
	continue # Skip this Mandala due to invalid 'rounds' input

	result_text = ""

	for r in rounds_list:
	abs_r = abs(r)

	# Determine the number of full passes and the remainder.
	full_passes = math.floor(abs_r)
	remainder = abs_r - full_passes

	# Base number of characters per pass
	base_chars = text_length // step

	if base_chars == 0:
	if abs_r > 1: # Changed from >=1 to >1
	# When step > text_length and rounds >1, pick 1 character per full pass
	chars_per_full_pass = 1
	logger.debug(f"Mandala {i}: step > text_length ({step} > {text_length}), selecting 1 character per full pass.")
	else:
	# No characters to pick
	chars_per_full_pass = 0
	logger.debug(f"Mandala {i}: step > text_length ({step} > {text_length}) and rounds <=1, no characters selected.")
	# For remainder, since base_chars=0, no remainder characters
	chars_for_remainder = 0
	else:
	# Normal case
	chars_per_full_pass = base_chars
	chars_for_remainder = math.floor(base_chars * remainder) # Partial pass
	logger.debug(f"Mandala {i}: Normal case, chars_per_full_pass = {chars_per_full_pass}, chars_for_remainder = {chars_for_remainder}")

	if r > 0:
	current_index = (step - 1) % text_length
	direction = 1
	else:
	current_index = (text_length - step) % text_length
	direction = -1

	pass_result = ""

	# Full passes, keep only the last pass
	for pass_num in range(1, full_passes + 1):
	current_pass_chars = ""
	for _ in range(chars_per_full_pass):
	if chars_per_full_pass == 0:
	break
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length

	# Keep only the last full pass
	if pass_num == full_passes:
	pass_result = current_pass_chars
	logger.debug(f"Mandala {i}: Pass {pass_num}, pass_result = {pass_result}")

	# Remainder pass for fractional rounds
	if remainder > 0 and chars_for_remainder > 0:
	current_pass_chars = ""
	for _ in range(chars_for_remainder):
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length
	pass_result += current_pass_chars
	logger.debug(f"Mandala {i}: Remainder pass_result = {pass_result}")

	# Handle cases where step > text_length and chars_per_full_pass=1
	if base_chars == 0 and chars_per_full_pass == 1 and full_passes > 0:
	# pass_result already contains the last character picked
	pass
	elif base_chars == 0 and chars_per_full_pass == 0 and full_passes > 0:
	# When no characters are picked, skip appending
	pass

	result_text += pass_result

	if length != 0:
	result_text = result_text[:length]
	logger.debug(f"Mandala {i}: Result text truncated to length {length}.")

	# Translate the result text if required
	try:
	translated_text = translator.translate(result_text) if translator and result_text else ""
	except Exception as e:
	logger.error(f"Mandala {i}: Translation error: {e}")
	translated_text = ""

	# Calculate the Gematria sum
	try:
	result_sum = calculate_gematria(result_text)
	except Exception as e:
	logger.error(f"Mandala {i}: Gematria calculation error: {e}")
	result_sum = None

	if result_text:
	results.append({
	"book": f"Rigveda {i}.",
	"title": f"Mandala {i}",
	"result_text": result_text,
	"result_sum": result_sum,
	"translated_text": translated_text,
	"source_language": "sa",
	})

	except FileNotFoundError:
	logger.error(f"Mandala {i}: File not found: {file_name}")
	results.append({"error": f"Mandala {i}: File not found: {file_name}"})
	except json.JSONDecodeError as e:
	logger.error(f"Mandala {i}: JSON error in file {file_name}: {e}")
	results.append({"error": f"Mandala {i}: JSON error in file {file_name}: {e}"})
	except KeyError as e:
	logger.error(f"Mandala {i}: Missing key in file {file_name}: {e}")
	results.append({"error": f"Mandala {i}: Missing key in file {file_name}: {e}"})
	except Exception as e:
	logger.error(f"Mandala {i}: Unexpected error in file {file_name}: {e}")
	results.append({"error": f"Mandala {i}: Unexpected error in file {file_name}: {e}"})

	return results if results else None

	[File Ends] hindu.py

	[File Begins] populate_translations.py
	import sqlite3
	import logging
	from deep_translator import GoogleTranslator, exceptions
	from tqdm import tqdm
	import threading
	import time
	from queue import Queue

	# Constants
	DATABASE_FILE = 'gematria.db' # Use your actual database file name
	BATCH_SIZE = 1000
	NUM_THREADS = 10 # Number of parallel translation threads

	# Set up logging
	logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

	# Initialize the translator
	translator = GoogleTranslator(source='yi', target='en')
	logging.info("Translator initialized.")

	# Separate Queue and tqdm
	translation_queue = Queue() # Regular queue
	translation_queue_tqdm = tqdm(total=0, dynamic_ncols=True, desc="Translation Queue") # tqdm for the queue
	total_translations_tqdm = tqdm(total=0, dynamic_ncols=True, desc="Total Translations") # tqdm for overall progress

	# Lock for database access
	db_lock = threading.Lock()
	translations_completed = 0 # Counter for completed translations


	def translate_and_store(phrase: str) -> str:
	"""Translates a Hebrew phrase to English using Google Translate."""
	global translator
	max_retries = 3
	retries = 0
	while retries < max_retries:
	try:
	translation = translator.translate(phrase)
	return translation
	except (exceptions.TranslationNotFound, exceptions.NotValidPayload,
	exceptions.ServerException, exceptions.RequestError) as e:
	retries += 1
	logging.warning(f"Error translating phrase '{phrase}': {e}. Retrying... ({retries}/{max_retries})")
	logging.error(f"Failed to translate phrase '{phrase}' after {max_retries} retries.")
	return None


	def translation_worker():
	"""Worker thread to process translations from the queue."""
	global conn, translator, translation_queue, db_lock, translation_queue_tqdm, translations_completed, total_translations_tqdm

	while True:
	phrase = translation_queue.get() # Get from the actual queue
	translation_queue_tqdm.update() # Update the tqdm progress bar
	if phrase is None: # Sentinel value to stop the thread
	break

	translation = translate_and_store(phrase)

	# Acquire the lock before any database interaction for this phrase
	with db_lock:
	with sqlite3.connect(DATABASE_FILE) as conn:
	cursor = conn.cursor()
	if translation is not None:
	cursor.execute("UPDATE results SET translation = ? WHERE words = ?", (translation, phrase))
	translations_completed += 1 # Increment the global counter
	total_translations_tqdm.update() # Update the overall progress bar
	conn.commit()

	translation_queue.task_done()


	def populate_translations():
	"""Populates translations for all Hebrew phrases in the database."""
	global conn, translator, translation_queue, translation_queue_tqdm, total_translations_tqdm

	with sqlite3.connect(DATABASE_FILE) as conn:
	cursor = conn.cursor()

	# Get the total count of distinct phrases needing translation
	cursor.execute("SELECT COUNT(DISTINCT words) FROM results WHERE translation IS NULL")
	total_phrases = cursor.fetchone()[0]

	logging.info(f"Found {total_phrases} distinct phrases to translate.")

	# Get distinct Hebrew phrases that need translation using a generator
	cursor.execute("SELECT DISTINCT words FROM results WHERE translation IS NULL")
	phrases_generator = (phrase for phrase, in cursor) # Use a generator for tqdm

	# Set the total for both tqdm progress bars
	translation_queue_tqdm.total = total_phrases
	total_translations_tqdm.total = total_phrases

	# Build the translation queue first
	for phrase in phrases_generator:
	translation_queue.put(phrase) # Put into the actual queue
	translation_queue_tqdm.update() # Update tqdm progress bar

	# Close the translation queue tqdm after it's fully populated
	translation_queue_tqdm.close()

	# Start worker threads AFTER the queue is built
	threads = []
	for _ in range(NUM_THREADS):
	thread = threading.Thread(target=translation_worker)
	thread.start()
	threads.append(thread)

	# Wait for all tasks to be completed
	translation_queue.join()

	# Stop worker threads
	for _ in range(NUM_THREADS):
	translation_queue.put(None) # Sentinel value to stop threads
	for thread in threads:
	thread.join()

	logging.info("All translations completed.")



	def save_translations_periodically():
	"""Saves translations to the database every minute."""
	while True:
	time.sleep(60) # Wait for 1 minute
	logging.info("Saving translations to the database...")
	with db_lock: # Acquire the lock before saving
	with sqlite3.connect(DATABASE_FILE) as conn:
	conn.commit()
	logging.info("Translations saved.")


	if __name__ == "__main__":
	# Start the translation process in a separate thread
	translation_thread = threading.Thread(target=populate_translations)
	translation_thread.start()

	# Start the periodic saving thread
	save_thread = threading.Thread(target=save_translations_periodically)
	save_thread.start()

	# Keep the main thread alive
	while True:
	time.sleep(1)
	[File Ends] populate_translations.py

	[File Begins] quran.py
	import logging
	logger = logging.getLogger(__name__)

	import json
	import os
	import re
	from deep_translator import GoogleTranslator
	from gematria import calculate_gematria
	import math

	import logging
	import json
	import re
	import math
	import glob
	from deep_translator import GoogleTranslator
	from gematria import calculate_gematria # Ensure this function is correctly defined
	import csv

	# Configure the logger
	# You can uncomment the next line to enable debugging logs
	# logging.basicConfig(level=logging.DEBUG, format='%(levelname)s:%(message)s')
	logger = logging.getLogger(__name__)

	def process_json_files(start=1, end=114, step=1, rounds="1", length=0, tlang="en", strip_spaces=True,
	strip_in_braces=True, strip_diacritics=True, translate=False):
	"""
	Processes Quran JSON files and performs various text manipulations.

	Parameters:
	- start (int): Start number of the Quran sura (1-114).
	- end (int): End number of the Quran sura (1-114).
	- step (int): Step size for character selection.
	- rounds (str): Comma-separated list of round numbers (can include negative values).
	- length (int): Maximum length of the result text.
	- tlang (str): Target language for translation.
	- strip_spaces (bool): Whether to remove spaces from the text.
	- strip_in_braces (bool): Whether to remove text within braces.
	- strip_diacritics (bool): Whether to remove diacritics from the text.
	- translate (bool): Whether to translate the result text.

	Returns:
	- list: A list of dictionaries containing processed data or error messages.
	"""
	base_path = "texts/quran"
	translator = GoogleTranslator(source='ar', target=tlang) if translate else None
	results = []

	for i in range(start, end + 1):
	file_name = f"{base_path}/{i:03}.json" # Updated file name formatting
	try:
	with open(file_name, 'r', encoding='utf-8') as file:
	data = json.load(file)

	# Extract text from verses
	full_text = ""
	for verse_key, verse_text in data.get("verse", {}).items():
	full_text += verse_text + " "

	# Remove BOM if present
	full_text = full_text.replace("\ufeff", "")

	clean_text = full_text
	if strip_in_braces:
	# Remove content within [], {}, <> brackets
	clean_text = re.sub(r"\[.?\]\|\{.?\}\|\<.*?\>", "", clean_text, flags=re.DOTALL)

	if strip_diacritics:
	# Remove Arabic diacritics and punctuation
	clean_text = re.sub(
	r"[\u0610-\u061A\u064B-\u065F\u0670\u06D6-\u06DC\u06DF-\u06E4\u06E7\u06E8\u06EA-\u06ED]+", "",
	clean_text)
	# Optionally, remove additional unwanted characters
	clean_text = re.sub(r'[:?!\'\-]', '', clean_text)

	# Normalize spaces
	clean_text = clean_text.replace("\n\n ", " ")
	clean_text = clean_text.replace("\n", " ")
	clean_text = re.sub(r'\s+', ' ', clean_text).strip()

	if strip_spaces:
	clean_text = clean_text.replace(" ", "")

	text_length = len(clean_text)
	logger.debug(f"Sura {i}: Clean text length = {text_length}")

	if text_length == 0:
	logger.warning(f"Sura {i}: No text available after cleaning.")
	continue # Skip processing if no text is available

	# Parse 'rounds', allow floats
	try:
	rounds_list = list(map(float, rounds.split(',')))
	except ValueError as e:
	logger.error(f"Sura {i}: Invalid 'rounds' format: {rounds}")
	results.append({"error": f"Sura {i}: Invalid 'rounds' format: {rounds}"})
	continue # Skip this Sura due to invalid 'rounds' input

	result_text = ""

	for r in rounds_list:
	abs_r = abs(r)

	# Determine the number of full passes and the remainder.
	full_passes = math.floor(abs_r)
	remainder = abs_r - full_passes

	# Base number of characters per pass
	base_chars = text_length // step

	if base_chars == 0:
	if abs_r > 1: # Changed from >=1 to >1
	# When step > text_length and rounds >1, pick 1 character per full pass
	chars_per_full_pass = 1
	logger.debug(f"Sura {i}: step > text_length ({step} > {text_length}), selecting 1 character per full pass.")
	else:
	# No characters to pick
	chars_per_full_pass = 0
	logger.debug(f"Sura {i}: step > text_length ({step} > {text_length}) and rounds <=1, no characters selected.")
	# For remainder, since base_chars=0, no remainder characters
	chars_for_remainder = 0
	else:
	# Normal case
	chars_per_full_pass = base_chars
	chars_for_remainder = math.floor(base_chars * remainder) # Partial pass
	logger.debug(f"Sura {i}: Normal case, chars_per_full_pass = {chars_per_full_pass}, chars_for_remainder = {chars_for_remainder}")

	if r > 0:
	current_index = (step - 1) % text_length
	direction = 1
	else:
	current_index = (text_length - step) % text_length
	direction = -1

	pass_result = ""

	# Full passes, keep only the last pass
	for pass_num in range(1, full_passes + 1):
	current_pass_chars = ""
	for _ in range(chars_per_full_pass):
	if chars_per_full_pass == 0:
	break
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length

	# Keep only the last full pass
	if pass_num == full_passes:
	pass_result = current_pass_chars
	logger.debug(f"Sura {i}: Pass {pass_num}, pass_result = {pass_result}")

	# Remainder pass for fractional rounds
	if remainder > 0 and chars_for_remainder > 0:
	current_pass_chars = ""
	for _ in range(chars_for_remainder):
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length
	pass_result += current_pass_chars
	logger.debug(f"Sura {i}: Remainder pass_result = {pass_result}")

	# Handle cases where step > text_length and chars_per_full_pass=1
	if base_chars == 0 and chars_per_full_pass == 1 and full_passes > 0:
	# pass_result already contains the last character picked
	pass
	elif base_chars == 0 and chars_per_full_pass == 0 and full_passes > 0:
	# When no characters are picked, skip appending
	pass

	result_text += pass_result

	# Translate the result text if required
	try:
	translated_text = translator.translate(result_text) if translator and result_text else ""
	except Exception as e:
	logger.error(f"Translation error: {e}")
	translated_text = ""

	if length != 0:
	result_text = result_text[:length]
	logger.debug(f"Sura {i}: Result text truncated to length {length}.")

	# Calculate the Gematria sum
	try:
	result_sum = calculate_gematria(result_text)
	except Exception as e:
	logger.error(f"Sura {i}: Gematria calculation error: {e}")
	result_sum = None

	if result_text:
	results.append({
	"book": f"Quran {i}.",
	"title": data.get("name", "Unknown Title"), # Use "name" instead of "title"
	"result_text": result_text,
	"result_sum": result_sum,
	"translated_text": translated_text,
	"source_language": "ar"
	})

	except FileNotFoundError:
	logger.error(f"File {file_name} not found.")
	results.append({"error": f"File {file_name} not found."})
	except json.JSONDecodeError as e:
	logger.error(f"File {file_name} could not be read as JSON: {e}")
	results.append({"error": f"File {file_name} could not be read as JSON: {e}"})
	except KeyError as e:
	logger.error(f"Expected key 'verse' is missing in {file_name}: {e}")
	results.append({"error": f"Expected key 'verse' is missing in {file_name}: {e}"})
	except Exception as e:
	logger.error(f"An unexpected error occurred while processing {file_name}: {e}")
	results.append({"error": f"An unexpected error occurred while processing {file_name}: {e}"})

	return results if results else None



	# Tests
	test_results = [
	#(process_json_files(0, 0, 21, rounds="3", length=0), "שרק"),
	]

	all_tests_passed = True
	for result, expected in test_results:
	if expected is None: # Check if no result is expected
	if not result:
	logger.info(f"Test passed: Expected no results, got no results.")
	else:
	logger.error(f"Test failed: Expected no results, but got: {result}")
	all_tests_passed = False
	else:
	# Check if result is not empty before accessing elements
	if result:
	#result_text = result[0]['result_text']
	result_text = None
	if result_text == expected:
	logger.info(f"Test passed: Expected '{expected}', got '{result_text}'")
	else:
	logger.error(f"Test failed: Expected '{expected}', but got '{result_text}'")
	all_tests_passed = False
	else:
	logger.error(f"Test failed: Expected '{expected}', but got no results")
	all_tests_passed = False

	if all_tests_passed:
	logger.info("All round tests passed.")

	[File Ends] quran.py

	[File Begins] requirements-all.txt
	aiofiles==23.2.1
	annotated-types==0.7.0
	anyio==4.6.2.post1
	beautifulsoup4==4.12.3
	certifi==2024.8.30
	charset-normalizer==3.4.0
	click==8.1.7
	deep-translator==1.11.4
	exceptiongroup==1.2.2
	fastapi==0.115.4
	ffmpy==0.4.0
	filelock==3.16.1
	fsspec==2024.10.0
	fuzzywuzzy==0.18.0
	gradio==5.5.0
	gradio_calendar==0.0.6
	gradio_client==1.4.2
	h11==0.14.0
	httpcore==1.0.6
	httpx==0.27.2
	huggingface-hub==0.26.2
	idna==3.10
	inflect==7.4.0
	Jinja2==3.1.4
	Levenshtein==0.26.1
	markdown-it-py==3.0.0
	MarkupSafe==2.1.5
	mdurl==0.1.2
	more-itertools==10.5.0
	numpy==2.1.3
	orjson==3.10.11
	packaging==24.2
	pandas==2.2.3
	pillow==11.0.0
	pydantic==2.9.2
	pydantic_core==2.23.4
	pydub==0.25.1
	Pygments==2.18.0
	python-dateutil==2.9.0.post0
	python-Levenshtein==0.26.1
	python-multipart==0.0.12
	pytz==2024.2
	PyYAML==6.0.2
	RapidFuzz==3.10.1
	requests==2.32.3
	rich==13.9.4
	ruff==0.7.3
	safehttpx==0.1.1
	semantic-version==2.10.0
	shellingham==1.5.4
	six==1.16.0
	sniffio==1.3.1
	soupsieve==2.6
	starlette==0.41.2
	tabulate==0.9.0
	tomlkit==0.12.0
	tqdm==4.67.0
	transliterate==1.10.2
	typeguard==4.4.1
	typer==0.13.0
	typing_extensions==4.12.2
	tzdata==2024.2
	urllib3==2.2.3
	uvicorn==0.32.0
	websockets==12.0

	[File Ends] requirements-all.txt

	[File Begins] requirements.txt
	gradio==5.5.0
	git+https://github.com/bartman081523/deep-translator
	tabulate==0.9.0
	gradio_calendar==0.0.6
	inflect==7.4.0
	fuzzywuzzy==0.18.0
	python-Levenshtein==0.26.1
	transliterate==1.10.2

	[File Ends] requirements.txt

	[File Begins] torah.py
	import logging
	logger = logging.getLogger(__name__)

	import json
	import os
	import re
	from deep_translator import GoogleTranslator
	from gematria import calculate_gematria
	import math


	def process_json_files(start, end, step, rounds="1", length=0, tlang="en", strip_spaces=True, strip_in_braces=True,
	strip_diacritics=True, translate=False):
	"""
	Processes Tripitaka JSON files and performs various text manipulations.

	Parameters:
	- start (int): Start number of the Tripitaka book.
	- end (int): End number of the Tripitaka book.
	- step (int): Step size for character selection.
	- rounds (str): Comma-separated list of round numbers (can include negative values).
	- length (int): Maximum length of the result text.
	- tlang (str): Target language for translation.
	- strip_spaces (bool): Whether to remove spaces from the text.
	- strip_in_braces (bool): Whether to remove text within braces.
	- strip_diacritics (bool): Whether to remove diacritics from the text.
	- translate (bool): Whether to translate the result text.

	Returns:
	- list: A list of dictionaries containing processed data or error messages.
	"""
	base_path = "texts/torah"
	translator = GoogleTranslator(source='auto', target=tlang)
	results = []

	for i in range(start, end + 1):
	file_name = f"{base_path}/{i:02}.json"
	try:
	with open(file_name, 'r', encoding='utf-8') as file:
	data = json.load(file)
	text_blocks = data["text"]

	full_text = ""
	for block in text_blocks:
	full_text += ' '.join(block)

	clean_text = full_text
	if strip_in_braces:
	clean_text = re.sub(r"\[.*?\]", "", clean_text, flags=re.DOTALL)
	if strip_diacritics:
	clean_text = re.sub(r"[^\u05D0-\u05EA ]+", "", clean_text)
	if strip_spaces:
	clean_text = clean_text.replace(" ", "")
	else:
	# Replace multiple spaces with a single space
	clean_text = re.sub(r'\s+', ' ', clean_text)

	text_length = len(clean_text)
	if text_length == 0:
	# If after cleaning, there's no text, skip processing
	continue

	rounds_list = list(map(float, rounds.split(','))) # Allow floats
	result_text = ""

	for r in rounds_list:
	abs_r = abs(r)

	# Determine the number of full passes and the remainder.
	full_passes = math.floor(abs_r)
	remainder = abs_r - full_passes

	# Base chars per pass
	base_chars = text_length // step

	if base_chars == 0:
	if abs_r > 1: # Changed from >=1 to >1
	# When step > text_length and rounds >1, pick 1 character per full pass
	chars_per_full_pass = 1
	else:
	# No characters to pick
	chars_per_full_pass = 0
	# For remainder, since base_chars=0, no remainder characters
	chars_for_remainder = 0
	else:
	# Normal case
	chars_per_full_pass = base_chars
	chars_for_remainder = math.floor(base_chars * remainder) # Partial pass

	if r > 0:
	current_index = (step - 1) % text_length
	direction = 1
	else:
	current_index = (text_length - step) % text_length
	direction = -1

	pass_result = ""

	# Full passes, we only keep the last pass
	for pass_num in range(1, full_passes + 1):
	current_pass_chars = ""
	for _ in range(chars_per_full_pass):
	if chars_per_full_pass == 0:
	break
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length

	# Keep only the last full pass
	if pass_num == full_passes:
	pass_result = current_pass_chars

	# Remainder pass, for fractional rounds
	if remainder > 0 and chars_for_remainder > 0:
	current_pass_chars = ""
	for _ in range(chars_for_remainder):
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length
	pass_result += current_pass_chars

	# Handle cases where step > text_length and chars_per_full_pass=1
	if base_chars == 0 and chars_per_full_pass == 1 and full_passes > 0:
	# Append the last character picked in the full passes
	# Since only one character is picked per full pass, pass_result is already set
	pass
	elif base_chars ==0 and chars_per_full_pass ==0 and full_passes >0:
	# When no characters are picked, and rounds >=1, we might want to skip
	pass

	result_text += pass_result

	# Only translate if needed
	translated_text = translator.translate(result_text) if (result_text and translate) else ""

	if length != 0:
	result_text = result_text[:length]

	if result_text:
	results.append({
	"book": f"Torah {i}.",
	"title": data.get("title", "Unknown Title"),
	"result_text": result_text,
	"result_sum": calculate_gematria(result_text),
	"translated_text": translated_text,
	"source_language": "iw",
	})

	except FileNotFoundError:
	results.append({"error": f"File {file_name} not found."})
	except json.JSONDecodeError as e:
	results.append({"error": f"File {file_name} could not be read as JSON: {e}"})
	except KeyError as e:
	results.append({"error": f"Expected key 'text' is missing in {file_name}: {e}"})

	return results if results else None




	# Tests
	test_results = [
	(process_json_files(0, 0, 21, rounds="3", length=0), "ק"),
	(process_json_files(0, 0, 22, rounds="1", length=0), "ת"),
	(process_json_files(0, 0, 22, rounds="3", length=0), "ת"),
	(process_json_files(0, 0, 23, rounds="3", length=0), "ג"),
	(process_json_files(0, 0, 11, rounds="1", length=0), "כת"),
	(process_json_files(0, 0, 2, rounds="1", length=0), "בדוחילנעצרת"),
	(process_json_files(0, 0, 23, rounds="1", length=0), None), # Expect None, when no results
	(process_json_files(0, 0, 23, rounds="-1", length=0), None), # Expect None, when no results
	(process_json_files(0, 0, 22, rounds="-1", length=0), "א"),
	(process_json_files(0, 0, 22, rounds="-2", length=0), "א"),
	(process_json_files(0, 0, 1, rounds="1,-1", length=0), "אבגדהוזחטיכלמנסעפצקרשתתשרקצפעסנמלכיטחזוהדגבא"), # Combined rounds
	(process_json_files(0, 0, 1, rounds="-1", length=0), "תשרקצפעסנמלכיטחזוהדגבא"), # Reversed Hebrew alphabet
	(process_json_files(0, 0, 1, rounds="-1.5", length=0), "תשרקצפעסנמלכיטחזוהדגבאתשרקצפעסנמל"), # Fractional rounds
	]

	all_tests_passed = True
	for result, expected in test_results:
	result_text = result[0]['result_text'] if result else None
	if expected is None: # Check if no result is expected
	if not result:
	logger.warning(f"Test passed: Expected no results, got no results.")
	else:
	logger.error(f"Test failed: Expected no results, but got: {result_text}")
	all_tests_passed = False
	else:
	# Check if result is not empty before accessing elements
	if result:
	if result_text == expected:
	logger.warning(f"Test passed: Expected '{expected}', got '{result_text}'")
	else:
	logger.error(f"Test failed: Expected '{expected}', but got '{result_text}'")
	all_tests_passed = False
	else:
	logger.error(f"Test failed: Expected '{expected}', but got no results")
	all_tests_passed = False

	if all_tests_passed:
	logger.info("All round tests passed.")

	[File Ends] torah.py

	[File Begins] translation_database.db.initial
	SQLite format 3@ iii.zp


	x%%3tabletranslationstranslationsCREATE TABLE translations (
	phrase TEXT PRIMARY KEY
	, `mi` TEXT, `sm` TEXT, `fa` TEXT, `ko` TEXT, `bg` TEXT, `ht` TEXT, `gu` TEXT, `sv` TEXT, `yi` TEXT, `ny` TEXT, `xh` TEXT, `ga` TEXT, `hi` TEXT, `ne` TEXT, `is_` TEXT, `sd` TEXT, `bn` TEXT, `yo` TEXT, `ha` TEXT, `uk` TEXT, `ky` TEXT, `si` TEXT, `hu` TEXT, `ta` TEXT, `ar` TEXT, `da` TEXT, `zh_CN` TEXT, `ka` TEXT, `eu` TEXT, `lo` TEXT, `co` TEXT, `pa` TEXT, `sl` TEXT, `ru` TEXT, `jw` TEXT, `st` TEXT, `pl` TEXT, `lb` TEXT, `hmn` TEXT, `sw` TEXT, `gd` TEXT, `sn` TEXT, `th` TEXT, `ps` TEXT, `pt` TEXT, `vi` TEXT, `mt` TEXT, `ceb` TEXT, `kn` TEXT, `hy` TEXT, `fy` TEXT, `mn` TEXT, `sr` TEXT, `fi` TEXT, `id` TEXT, `haw` TEXT, `te` TEXT, `mg` TEXT, `zh_TW` TEXT, `su` TEXT, `iw` TEXT, `ro` TEXT, `am` TEXT, `ml` TEXT, `mr` TEXT, `kk` TEXT, `en` TEXT, `lv` TEXT, `el` TEXT, `la` TEXT, `no` TEXT, `ur` TEXT, `tg` TEXT, `sk` TEXT, `ig` TEXT, `it` TEXT, `sq` TEXT, `eo` TEXT, `mk` TEXT, `tl` TEXT, `bs` TEXT, `ca` TEXT, `km` TEXT, `ku` TEXT, `nl` TEXT, `et` TEXT, `gl` TEXT, `cy` TEXT, `be` TEXT, `my` TEXT, `uz` TEXT, `so` TEXT, `fr` TEXT, `hr` TEXT, `de` TEXT, `az` TEXT, `af` TEXT, `cs` TEXT, `lt` TEXT, `tr` TEXT, `ms` TEXT, `ja` TEXT, `zu` TEXT, `es` TEXT)7K%indexsqlite_autoindex_translations_1translations


	[File Ends] translation_database.db.initial

	[File Begins] translation_utils.py
	# translation_utils.py

	import logging
	import sqlite3
	from concurrent.futures import ThreadPoolExecutor
	import functools
	from deep_translator import GoogleTranslator, exceptions

	# Constants
	TRANSLATION_DATABASE_FILE = 'translation_database.db'
	SUPPORTED_LANGUAGES = {"af", "sq", "am", "ar", "hy", "az", "eu", "be", "bn", "bs", "bg", "ca", "ceb", "ny", "zh-CN", "zh-TW", "co", "hr", "cs", "da", "nl", "en", "eo", "et", "tl", "fi", "fr", "fy", "gl", "ka", "de", "el", "gu", "ht", "ha", "haw", "iw", "hi", "hmn", "hu", "is", "ig", "id", "ga", "it", "ja", "jw", "kn", "kk", "km", "ko", "ku", "ky", "lo", "la", "lv", "lt", "lb", "mk", "mg", "ms", "ml", "mt", "mi", "mr", "mn", "my", "ne", "no", "ps", "fa", "pl", "pt", "pa", "ro", "ru", "sm", "gd", "sr", "st", "sn", "sd", "si", "sk", "sl", "so", "es", "su", "sw", "sv", "tg", "ta", "te", "th", "tr", "uk", "ur", "uz", "vi", "cy", "xh", "yi", "yo", "zu"}

	# Initialize logger
	logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
	logger = logging.getLogger(__name__)


	def create_translation_table():
	"""Creates the translation table if it doesn't exist."""
	try:
	with sqlite3.connect(TRANSLATION_DATABASE_FILE) as conn:
	cursor = conn.cursor()
	cursor.execute('''
	CREATE TABLE IF NOT EXISTS translations (
	phrase TEXT PRIMARY KEY
	)
	''')

	# Dynamically add language columns
	cursor.execute("PRAGMA table_info(translations)")
	existing_columns = {col[1] for col in cursor.fetchall()}
	for lang_code in SUPPORTED_LANGUAGES:
	column_name = lang_code.replace('-', '_')
	if column_name == "is": # Handle reserved keywords in SQLite
	column_name = "is_"
	if column_name not in existing_columns:
	try:
	cursor.execute(f"ALTER TABLE translations ADD COLUMN `{column_name}` TEXT")
	logger.info(f"Added column '{column_name}' to translations table.")
	except sqlite3.OperationalError as e:
	if "duplicate column name" in str(e).lower():
	logger.debug(f"Column '{column_name}' already exists. Skipping.")
	else:
	logger.error(f"Error adding column '{column_name}': {e}") # More specific error
	conn.commit()
	except Exception as e: # Broad exception handling to catch any other potential issues
	logger.error(f"An unexpected error occurred in create_translation_table: {e}")


	@functools.lru_cache(maxsize=1000) # Use the correct decorator name
	def translate_cached(text, target_language, source_language="auto"): # Renamed to avoid conflicts
	"""Translates text using Google Translate with caching."""
	if not text:
	return ""

	try:
	translator = GoogleTranslator(source=source_language, target=target_language)
	translated = translator.translate(text)
	return translated
	except exceptions.TranslationNotFound:
	logger.error(f"Translation not found for: {text}")
	except Exception as e: # Catch generic exceptions
	logger.exception(f"Translation error: {e}") # Log with traceback
	return None


	def get_translation(phrase, target_language, source_language="auto"):
	"""Retrieves a translation from the database or translates and stores it."""
	if target_language not in SUPPORTED_LANGUAGES:
	logger.error(f"Unsupported target language: {target_language}")
	return None, False # Return None and False for failure

	try:
	with sqlite3.connect(TRANSLATION_DATABASE_FILE) as conn:
	cursor = conn.cursor()
	column_name = target_language.replace('-', '_')
	if column_name == "is":
	column_name = "is_"

	cursor.execute(f"SELECT `{column_name}` FROM translations WHERE phrase=?", (phrase,))
	result = cursor.fetchone()

	if result and result[0]: # Check that the result is not empty and has a translated value
	return result[0], True

	translated_text = translate_cached(phrase, target_language, source_language)
	if translated_text:
	cursor.execute(f"""
	INSERT INTO translations (phrase, `{column_name}`) VALUES (?, ?)
	ON CONFLICT(phrase) DO UPDATE SET `{column_name}`=excluded.`{column_name}`
	""", (phrase, translated_text))

	conn.commit()
	return translated_text, True
	else:
	return None, False # Explicitly return False when translation fails


	except sqlite3.Error as e:
	logger.error(f"Database error: {e}")
	return None, False # Return explicit failure indicator
	except Exception as e:
	logger.exception(f"Unexpected error in get_translation: {e}") # Generic Exception Catch-All
	return None, False # Return explicit failure indicator


	def batch_translate(phrases, target_language, source_languages): # Takes a list of source languages
	"""Translates multiple phrases concurrently, respecting source language."""

	with ThreadPoolExecutor() as executor:
	# Use source language for each phrase
	futures = [executor.submit(get_translation, phrase, target_language, source_languages[i])
	for i, phrase in enumerate(phrases) if phrase] # Only translate if phrase is not empty

	results = [future.result() for future in futures] # Only includes results from phrases that were translated


	translations = {}
	j=0 #index for results
	for i, phrase in enumerate(phrases):
	if phrase: #if phrase was translated
	translations[phrase] = results[j][0] #Store translation (results[j] is a tuple (translation, success_flag), take the first element)
	j+=1
	else: # If phrase is empty
	translations[phrase] = None #Or translations[phrase] = "" # Alternatively assign empty string if not translating empty phrases

	return translations

	[File Ends] translation_utils.py

	[File Begins] tripitaka.py
	import logging
	import json
	import re
	import math
	import glob
	from deep_translator import GoogleTranslator
	from gematria import calculate_gematria # Ensure this function is correctly defined
	import csv

	# Configure the logger
	# You can uncomment the next line to enable debugging logs
	# logging.basicConfig(level=logging.DEBUG, format='%(levelname)s:%(message)s')
	logger = logging.getLogger(__name__)

	def process_json_files(start=1, end=52, step=1, rounds="1,-1", length=0, tlang="en", strip_spaces=True,
	strip_in_braces=True, strip_diacritics=True, translate=False):
	"""
	Processes Tripitaka JSON files and performs various text manipulations.

	Parameters:
	- start (int): Start number of the Tripitaka book.
	- end (int): End number of the Tripitaka book.
	- step (int): Step size for character selection.
	- rounds (str): Comma-separated list of round numbers (can include negative values).
	- length (int): Maximum length of the result text.
	- tlang (str): Target language for translation.
	- strip_spaces (bool): Whether to remove spaces from the text.
	- strip_in_braces (bool): Whether to remove text within braces.
	- strip_diacritics (bool): Whether to remove diacritics from the text.
	- translate (bool): Whether to translate the result text.

	Returns:
	- list: A list of dictionaries containing processed data or error messages.
	"""
	base_path = "texts/tripitaka"
	translator = GoogleTranslator(source='sa', target=tlang) if translate else None # Correct source language for Sanskrit
	results = []

	for i in range(start, end + 1):
	file_pattern = f"{base_path}/{i:02}*.json" # Adjust to Tripitaka file naming
	matched_files = glob.glob(file_pattern)
	if not matched_files:
	logger.warning(f"Mandala {i}: No files found for pattern {file_pattern}.")
	results.append({"error": f"Mandala {i}: No files found for pattern {file_pattern}."})
	continue

	for file_name in matched_files:
	try:
	with open(file_name, 'r', encoding='utf-8') as file:
	data = json.load(file)
	full_text = ""
	for gatha in data.get("gathas", []): # Safe access to 'gathas'
	full_text += gatha + " "

	clean_text = full_text
	if strip_in_braces:
	# Remove content within [], {}, <> brackets
	clean_text = re.sub(r"\[.?\]\|\{.?\}\|\<.*?\>", "", clean_text, flags=re.DOTALL)

	if strip_diacritics:
	# Keep only Devanagari characters and spaces
	clean_text = re.sub(r'[^\u0900-\u097F\s]', '', clean_text)
	# Remove specific diacritics and punctuation
	clean_text = re.sub(r'[\u0951-\u0954\u0964\u0965]+', '', clean_text)
	# Remove Devanagari numerals
	clean_text = re.sub(r'[०१२३४५६७८९]+', '', clean_text)
	# Remove additional unwanted characters in one step
	clean_text = re.sub(r'[:?!\'\-]', '', clean_text)

	# Normalize spaces
	clean_text = clean_text.replace("\n\n ", " ")
	clean_text = clean_text.replace("\n", " ")
	clean_text = re.sub(r'\s+', ' ', clean_text).strip()

	if strip_spaces:
	clean_text = clean_text.replace(" ", "")

	text_length = len(clean_text)
	logger.debug(f"Mandala {i}: Clean text length = {text_length}")

	if text_length == 0:
	logger.warning(f"Mandala {i}: No text available after cleaning.")
	continue # Skip processing if no text is available

	# Parse 'rounds', allow floats
	try:
	rounds_list = list(map(float, rounds.split(',')))
	except ValueError as e:
	logger.error(f"Mandala {i}: Invalid 'rounds' format: {rounds}")
	results.append({"error": f"Mandala {i}: Invalid 'rounds' format: {rounds}"})
	continue # Skip this Mandala due to invalid 'rounds' input

	result_text = ""

	for r in rounds_list:
	abs_r = abs(r)

	# Determine the number of full passes and the remainder.
	full_passes = math.floor(abs_r)
	remainder = abs_r - full_passes

	# Base number of characters per pass
	base_chars = text_length // step

	if base_chars == 0:
	if abs_r > 1: # Changed from >=1 to >1
	# When step > text_length and rounds >1, pick 1 character per full pass
	chars_per_full_pass = 1
	logger.debug(f"Mandala {i}: step > text_length ({step} > {text_length}), selecting 1 character per full pass.")
	else:
	# No characters to pick
	chars_per_full_pass = 0
	logger.debug(f"Mandala {i}: step > text_length ({step} > {text_length}) and rounds <=1, no characters selected.")
	# For remainder, since base_chars=0, no remainder characters
	chars_for_remainder = 0
	else:
	# Normal case
	chars_per_full_pass = base_chars
	chars_for_remainder = math.floor(base_chars * remainder) # Partial pass
	logger.debug(f"Mandala {i}: Normal case, chars_per_full_pass = {chars_per_full_pass}, chars_for_remainder = {chars_for_remainder}")

	if r > 0:
	current_index = (step - 1) % text_length
	direction = 1
	else:
	current_index = (text_length - step) % text_length
	direction = -1

	pass_result = ""

	# Full passes, keep only the last pass
	for pass_num in range(1, full_passes + 1):
	current_pass_chars = ""
	for _ in range(chars_per_full_pass):
	if chars_per_full_pass == 0:
	break
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length

	# Keep only the last full pass
	if pass_num == full_passes:
	pass_result = current_pass_chars
	logger.debug(f"Mandala {i}: Pass {pass_num}, pass_result = {pass_result}")

	# Remainder pass for fractional rounds
	if remainder > 0 and chars_for_remainder > 0:
	current_pass_chars = ""
	for _ in range(chars_for_remainder):
	current_pass_chars += clean_text[current_index]
	current_index = (current_index + direction * step) % text_length
	pass_result += current_pass_chars
	logger.debug(f"Mandala {i}: Remainder pass_result = {pass_result}")

	# Handle cases where step > text_length and chars_per_full_pass=1
	if base_chars == 0 and chars_per_full_pass == 1 and full_passes > 0:
	# pass_result already contains the last character picked
	pass
	elif base_chars == 0 and chars_per_full_pass == 0 and full_passes > 0:
	# When no characters are picked, skip appending
	pass

	result_text += pass_result

	logger.debug(f"Result text for Mandala {i}: {result_text}")

	if length != 0:
	result_text = result_text[:length]
	logger.debug(f"Mandala {i}: Result text truncated to length {length}.")

	# Translate the result text if required
	try:
	translated_text = translator.translate(result_text) if translator and result_text else ""
	except Exception as e:
	logger.error(f"Translation error: {e}")
	translated_text = ""

	# Calculate the Gematria sum
	try:
	result_sum = calculate_gematria(result_text)
	except Exception as e:
	logger.error(f"Mandala {i}: Gematria calculation error: {e}")
	result_sum = None

	if result_text:
	result = {
	'book': f"Tripitaka {i}.",
	'title': f'{data.get("title", "Unknown Title")} {data.get("book_name", "Unknown Book")} {data.get("chapter", "")}',
	'result_text': result_text,
	'result_sum': result_sum,
	'translated_text': translated_text,
	'source_language': "sa",
	}
	results.append(result)

	except (FileNotFoundError, json.JSONDecodeError, KeyError) as e:
	logger.error(f"Mandala {i}: Error processing {file_name}: {e}")
	results.append({"error": f"Mandala {i}: Error processing {file_name}: {e}"})
	except Exception as e:
	logger.error(f"Mandala {i}: Unexpected error processing {file_name}: {e}")
	results.append({"error": f"Mandala {i}: Unexpected error processing {file_name}: {e}"})

	return results if results else None

	# Tests
	test_results = [
	(process_json_files(1,1,386,"1,-1", translate=True),"अअमुोच्टेकोसचण")
	]

	# Function to run tests
	def run_tests(test_results):
	all_tests_passed = True
	for idx, (result, expected) in enumerate(test_results):
	if expected is None: # Check if no result is expected
	if not result:
	logger.warning(f"Test {idx}: Passed (Expected no results, got no results.)")
	else:
	# Extract result_text from the first result
	result_text = result[0].get('result_text', None)
	logger.error(f"Test {idx}: Failed (Expected no results, but got: {result_text})")
	all_tests_passed = False
	else:
	# Check if result is not empty before accessing elements
	if result:
	result_text = result[0].get("result_text")
	if result_text == expected:
	logger.warning(f"Test {idx}: Passed (Expected: '{expected}', Got: '{result_text}')")
	else:
	logger.error(f"Test {idx}: Failed (Expected: '{expected}', but got: '{result_text}')")
	all_tests_passed = False
	else:
	logger.error(f"Test {idx}: Failed (Expected: '{expected}', but got no results)")
	all_tests_passed = False

	if all_tests_passed:
	logger.info("All round tests passed.")
	else:
	logger.info("Some tests failed. Please check the error messages above.")

	# Run the tests
	run_tests(test_results)

	[File Ends] tripitaka.py

	[File Begins] util.py
	import json
	import re

	def process_json_files(start, end):
	"""
	Processes JSON files containing Tanach text and returns a dictionary
	mapping book IDs to their data.

	Args:
	start: The starting book ID (inclusive).
	end: The ending book ID (inclusive).

	Returns:
	A dictionary where keys are book IDs and values are dictionaries
	containing 'title' and 'text' fields.
	"""
	base_path = "texts"
	results = {} # Use a dictionary to store results

	for i in range(start, end + 1):
	file_name = f"{base_path}/{i:02}.json"
	try:
	with open(file_name, 'r', encoding='utf-8') as file:
	data = json.load(file)
	if data:
	# Store book ID as key and book data as value
	results[i] = {"title": data.get("title", "No title"), "text": data.get("text", [])}

	except FileNotFoundError:
	logging.warning(f"File {file_name} not found.")
	except json.JSONDecodeError as e:
	logging.warning(f"File {file_name} could not be read as JSON: {e}")
	except KeyError as e:
	logging.warning(f"Expected key 'text' is missing in {file_name}: {e}")

	return results
	[File Ends] util.py

	[File Begins] utils.py
	import logging
	logger = logging.getLogger(__name__)
	logging.basicConfig(level=logging.INFO)

	import inflect
	from datetime import datetime, date
	from deep_translator import GoogleTranslator

	# Custom function to convert number to ordinal words
	def number_to_ordinal_word(number):
	ordinal_dict = {
	1: "first", 2: "second", 3: "third", 4: "fourth", 5: "fifth",
	6: "sixth", 7: "seventh", 8: "eighth", 9: "ninth", 10: "tenth",
	11: "eleventh", 12: "twelfth", 13: "thirteenth", 14: "fourteenth",
	15: "fifteenth", 16: "sixteenth", 17: "seventeenth", 18: "eighteenth",
	19: "nineteenth", 20: "twentieth", 21: "twentyfirst", 22: "twentysecond",
	23: "twentythird", 24: "twentyfourth", 25: "twentyfifth",
	26: "twentysixth", 27: "twentyseventh", 28: "twentyeighth",
	29: "twentyninth", 30: "thirtieth", 31: "thirtyfirst"
	}
	return ordinal_dict.get(number, "")

	def custom_normalize(text):
	mappings = {
	'ü': 'ue', 'ö': 'oe', 'ä': 'ae', 'ß': 'ss', 'Ü': 'Ue', 'Ö': 'Oe', 'Ä': 'Ae',
	'á': 'a', 'à': 'a', 'â': 'a', 'ã': 'a', 'å': 'aa', 'ā': 'a', 'ă': 'a', 'ą': 'a',
	'Á': 'A', 'À': 'A', 'Â': 'A', 'Ã': 'A', 'Å': 'Aa', 'Ā': 'A', 'Ă': 'A', 'Ą': 'A',
	'é': 'e', 'è': 'e', 'ê': 'e', 'ë': 'e', 'ē': 'e', 'ĕ': 'e', 'ė': 'e', 'ę': 'e', 'ě': 'e',
	'É': 'E', 'È': 'E', 'Ê': 'E', 'Ë': 'E', 'Ē': 'E', 'Ĕ': 'E', 'Ė': 'E', 'Ę': 'E', 'Ě': 'E',
	'í': 'i', 'ì': 'i', 'î': 'i', 'ï': 'i', 'ī': 'i', 'ĭ': 'i', 'į': 'i', 'ı': 'i',
	'Í': 'I', 'Ì': 'I', 'Î': 'I', 'Ï': 'I', 'Ī': 'I', 'Ĭ': 'I', 'Į': 'I', 'I': 'I',
	'ó': 'o', 'ò': 'o', 'ô': 'o', 'õ': 'o', 'ø': 'oe', 'ō': 'o', 'ŏ': 'o', 'ő': 'o',
	'Ó': 'O', 'Ò': 'O', 'Ô': 'O', 'Õ': 'O', 'Ø': 'Oe', 'Ō': 'O', 'Ŏ': 'O', 'Ő': 'O',
	'ú': 'u', 'ù': 'u', 'û': 'u', 'ū': 'u', 'ŭ': 'u', 'ů': 'u', 'ű': 'u', 'ų': 'u',
	'Ú': 'U', 'Ù': 'U', 'Û': 'U', 'Ü': 'Ue', 'Ū': 'U', 'Ŭ': 'U', 'Ů': 'U', 'Ű': 'U', 'Ų': 'U',
	'ç': 'c', 'ć': 'c', 'ĉ': 'c', 'ċ': 'c', 'č': 'c',
	'Ç': 'C', 'Ć': 'C', 'Ĉ': 'C', 'Ċ': 'C', 'Č': 'C',
	'ñ': 'n', 'ń': 'n', 'ņ': 'n', 'ň': 'n', 'ŋ': 'n',
	'Ñ': 'N', 'Ń': 'N', 'Ņ': 'N', 'Ň': 'N', 'Ŋ': 'N',
	'ý': 'y', 'ÿ': 'y', 'ŷ': 'y',
	'Ý': 'Y', 'Ÿ': 'Y', 'Ŷ': 'Y',
	'ž': 'zh', 'ź': 'z', 'ż': 'z',
	'Ž': 'Zh', 'Ź': 'Z', 'Ż': 'Z',
	'ð': 'd', 'Ð': 'D', 'þ': 'th', 'Þ': 'Th', 'ł': 'l', 'Ł': 'L', 'đ': 'd', 'Đ': 'D',
	'æ': 'ae', 'Æ': 'Ae', 'œ': 'oe', 'Œ': 'Oe',
	'ś': 's', 'ŝ': 's', 'ş': 's', 'š': 's',
	'Ś': 'S', 'Ŝ': 'S', 'Ş': 'S', 'Š': 'S',
	'ť': 't', 'ţ': 't', 'ŧ': 't', 'Ť': 'T', 'Ţ': 'T', 'Ŧ': 'T',
	'ŕ': 'r', 'ř': 'r', 'Ŕ': 'R', 'Ř': 'R',
	'ľ': 'l', 'ĺ': 'l', 'ļ': 'l', 'ŀ': 'l',
	'Ľ': 'L', 'Ĺ': 'L', 'Ļ': 'L', 'Ŀ': 'L',
	'ē': 'e', 'Ē': 'E',
	'ň': 'n', 'Ň': 'N',
	'ğ': 'g', 'Ğ': 'G',
	'ġ': 'g', 'Ġ': 'G',
	'ħ': 'h', 'Ħ': 'H',
	'ı': 'i', 'İ': 'I',
	'ĵ': 'j', 'Ĵ': 'J',
	'ķ': 'k', 'Ķ': 'K',
	'ļ': 'l', 'Ļ': 'L',
	'ņ': 'n', 'Ņ': 'N',
	'ŧ': 't', 'Ŧ': 'T',
	'ŭ': 'u', 'Ŭ': 'U'
	}
	for key, value in mappings.items():
	text = text.replace(key, value)
	return text




	# Convert a date to words with an ordinal day
	def date_to_words(date_obj):
	inf_engine = inflect.engine()

	if isinstance(date_obj, (date, datetime)):
	year = date_obj.year
	month = date_obj.strftime("%B") if hasattr(date_obj, 'month') else None
	day_ordinal = number_to_ordinal_word(date_obj.day) if hasattr(date_obj, 'day') else None
	elif isinstance(date_obj, str):
	try:
	# Attempt to parse full date first
	date_obj = datetime.strptime(date_obj, "%Y-%m-%d")
	year = date_obj.year
	month = date_obj.strftime("%B")
	day_ordinal = number_to_ordinal_word(date_obj.day)
	except ValueError:
	try:
	# Try year-month
	date_obj = datetime.strptime(date_obj, "%Y-%m")
	year = date_obj.year
	month = date_obj.strftime("%B")
	day_ordinal = None
	except ValueError:
	try:
	# Try just year
	date_obj = datetime.strptime(date_obj, "%Y")
	year = date_obj.year
	month = None
	day_ordinal = None
	except ValueError:
	raise ValueError("Invalid date format. Use YYYY-MM-DD, YYYY-MM, or YYYY.")
	else:
	raise TypeError("date_obj must be a datetime, date, or string object.")


	if 1900 <= year <= 1999:
	year_words = f"{inf_engine.number_to_words(year // 100, andword='')} hundred"
	if year % 100 != 0:
	year_words += f" {inf_engine.number_to_words(year % 100, andword='')}"
	else:
	year_words = inf_engine.number_to_words(year, andword='')
	year_formatted = year_words.replace(',', '')

	parts = []
	if day_ordinal:
	parts.append(day_ordinal)
	if month:
	parts.append(month)
	parts.append(year_formatted)

	return " ".join(parts)



	def translate_date_to_words(date, lang='en'):
	"""Converts a date to words in the specified language."""
	if date is None:
	return "No date selected"

	date_string = date.strftime("%Y-%m-%d")
	logger.info(f"Date string: {date_string}")

	date_in_words = date_to_words(date_string)
	logger.info(f"Date in words: {date_in_words}")

	translator = GoogleTranslator(source='auto', target=lang)
	translated_date_words = translator.translate(date_in_words)
	logger.info(f"Translated date words: {translated_date_words}")

	# Normalize the text if it contains any special characters
	translated_date_words = custom_normalize(translated_date_words)
	logger.info(f"Normalized date words: {translated_date_words}")

	return translated_date_words

	[File Ends] utils.py


	<-- File Content Ends