feat: add dictionary CSV export tools (Jastrow, BDB, Klein)

[smargolis]

Adds CLI script and browser-based explorer for exporting Sefaria's dictionary/lexicon data (Jastrow, BDB, Klein) as structured CSV files.

Dictionary entries use a DictionaryNode schema and aren't included in the standard GCS text export. These tools crawl /api/words/ following next_hw linked-list pointers to enumerate every entry.

New files:

• scripts/export_dictionaries_csv.py: CLI for bulk CSV export with resume, rate limiting, HTML stripping
• • examples/dictionary-explorer/index.html: zero-dependency browser app with search, letter nav, CSV download
• • examples/export_dictionary_example.py: quick demo script
• • tests/test_export_dictionaries_csv.py: 30 unit tests (all passing alongside existing 24)

[yodem]

Code review

Found 2 issues:

1. Broken regex in default output filename. re.sub(r'[^w-]', '_', args.lexicon) is missing the backslash before w. The character class [^w-] means "not the literal letter w or hyphen" rather than "not a word character", so "Jastrow Dictionary" becomes __r_s_ow__ic_io_ar___sample.csv instead of Jastrow_Dictionary_sample.csv. The main script gets this right with r"[^\w\-]" — the example was clearly meant to mirror it but dropped the escape.

Sefaria-Export/examples/export_dictionary_example.py

Lines 135 to 139 in 262e5f3

	print(f" {e['definition']}")
	else:
	output_file = args.output or f"{re.sub(r'[^w-]', '_', args.lexicon)}_sample.csv"
	with open(output_file, "w", newline="", encoding="utf-8") as f:
	writer = csv.DictWriter(f, fieldnames=["headword", "rid", "transliteration",

2. Example crawl terminates silently on cross-lexicon headwords. In the while current_hw and len(entries) < args.limit: loop, next_hw is initialized to None and only assigned inside for entry in matches. If a headword exists in the API but has no entry in the target lexicon (a known cross-lexicon case), matches is empty, next_hw stays None, and the loop ends — producing a truncated CSV well below --limit with no warning. scripts/export_dictionaries_csv.py handles this by falling back to any entry's next_hw; the example omits that fallback.

Sefaria-Export/examples/export_dictionary_example.py

Lines 104 to 120 in 262e5f3

	entries = []
	while current_hw and len(entries) < args.limit:
	resp = requests.get(f"{SEFARIA_API}/words/{urllib.parse.quote(current_hw)}")
	resp.raise_for_status()
	data = resp.json()
	# Filter to target lexicon
	matches = [e for e in data if e.get("parent_lexicon") == args.lexicon]
	next_hw = None
	for entry in matches:
	senses = entry.get("content", {}).get("senses", [])
	row = {
	"headword": entry.get("headword", ""),
	"definition": flatten_senses(senses),
	"morphology": entry.get("content", {}).get("morphology", ""),
	"transliteration": entry.get("transliteration", ""),

---

On the linked gist (Arithmomaniac/924ef9e00ff2cabf72142d75c9263da0): it's a parallel implementation of the same goal — exporting Jastrow/BDB/Klein — but reads raw MongoDB BSON dumps (lexicon.bson, lexicon_entry.bson, word_form.bson) instead of crawling the public /api/words/ HTTP endpoint. It uses BeautifulSoup for HTML stripping (vs. regex here), unicodedata headword normalization, a two-stage JSONL→CSV/DSL pipeline, and produces a richer 25-column schema plus ABBYY Lingvo DSL output. The two approaches are complementary: the gist requires DB-dump access (insider-only); this PR works against the public API (anyone can run it). Worth deciding whether to cross-link them in the README, and whether the PR's CSV schema should expand to match the gist's columns where the API exposes them.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[yodem]

@smargolis Thanks for the PR!
I added some notes using Claude.
2 things from my side:

1. there is a small problem with the html - Since its under our repo we might want it to be under our own design system and might go under product/ux teams. I think we should keep the html out of this scope and preserve only the cli tool/scripts. we can create a "powered-by" project for this html, you can have it on vercel for free and I would be happy to help with it.
2. I didnt quite get why is there an export_dictionary_example.py? I mean isnt the example is the actual script? we can perhaps get rid of the example suffix.

I will also say that i didnt really get into the 2000+ lines but it is important for me to get this tool inside the repo.