Hebrew bakes the speaker's gender directly into first-person verbs and adjectives, sometimes through different letters and sometimes only through unwritten vowels. To solve this for language learners and text-to-speech pipelines, a learner named Daniel spent roughly ten dollars building a two-model system: a fine-tuned Gemma 4 E4B model that generates unpointed Hebrew sentences natively in a chosen gender, and a planned second model to add vowels and route voices. Using synthetic sentences from Claude Sonnet filtered by Dicta's joint morphological analyser, the one-pass generator avoids the subtle errors of rewriting masculine text into feminine. Across three iterations, the fine-tune outperformed prompting on JSON formatting and naturalness, though achieving reliable female-speaker accuracy remains an ongoing challenge when inflecting masculine dictionary terms.
Training a $10 Model for Hebrew Speaker Gender
A $10 LoRA fine-tune on Gemma 4 E4B learns to write gendered Hebrew in 32 minutes, but a filter bug initially made it worse for female speakers.
29:18 Transcript
Episode Details
- Episode ID
- MWP-6868
- Published
- Duration
- 29:18
- Audio
- Direct link
- Pipeline
- V5.35
- TTS Engine
- omnivoice
- Script Writing Agent
- GLM-5.3
AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.
The prompt
The prompt that started this episode, as sent in (production notes for the hosts removed).
Hebrew marks the speaker's gender in the words themselves: a man says "ani holekh" and a woman "ani holekhet" for "I go", and sometimes only the vowels differ: "ani rotse" and "ani rotsa" for "I want" are spelled identically. For a Hebrew learner like me, or for a text-to-speech voice, that matters: a male voice should read sentences written for a man. No tool I could find handles this, so today I started on two small models. The first generates Hebrew sentences in the right gender for the speaker; it's a Gemma 4 fine-tune now in its third version. The second will be a simple classifier that works in reverse: give it Hebrew with nikud and it says whether the speaker is male or female, so the sentence goes to the right voice. The whole thing has cost about ten dollars. Talk through these works in progress, what each version taught me, and the bigger question: when filling gaps like this has become so cheap and easy, how is it that nobody has built these small Hebrew models already?
Sources
What the research for this episode read before the script was written. Primary sources first.
- Hugging Face model card, v3, created/modified 2026-11-18, retrieved 2026-11-18. primary
- GitHub README, status dated 2026-11-18, retrieved 2026-11-18. primary
- Hugging Face model repo, 305M params, CC-BY-4.0, created 2026-11-18; card currently the inherited DictaBERT-large-char-menaked card (SOTA claim dated 2025-03), retrieved 2026-11-18. primary
- training dataset, MIT licence, created 2026-11-18, retrieved 2026-11-18. primary
- base model, 8.0B params, Apache-2.0, created 2026-03-02, retrieved 2026-11-18. primary
- step-up base model, 12.0B params, created 2026-05-23, retrieved 2026-11-18. primary
- and https://huggingface.co/google/gemma-4-26B-A4B-it primary Gemma 4 siblings, created 2026-03-11, retrieved 2026-11-18.
- Wikipedia, last edited 2026-10-21, retrieved 2026-11-18 (background on Modern Hebrew inflection).
- Wikipedia, last edited 2026-11-16, retrieved 2026-11-18 (background on the Tiberian vocalization system behind nikud).
Context
Daniel's own project notes, written on 18 November 2026. Treat them as ground truth for his models and numbers. V3 was still training when this episode was made, so its results are unknown: say so, and don't invent them. The classifier is planned, with nothing trained; its design is undecided. Costs marked as estimates are estimates. The "why the gap exists" view at the end is a working view, not a sourced claim: research that question properly. # Brief: Daniel's two Hebrew gendered models (as of 2026-11-18, 23:15 Israel time) Written by the session that built model 1. Confirmed facts come from the repo `~/repos/github/audio-dictation/Hebrew-Gendered-Sentence-Creator` (`docs/training-plan.md` has the full numbers). **V3 is still training**, so its results are not in this brief. ## The two models (Daniel's own description, 18 Nov 2026) 1. **Generator (model 1, now at V3):** term + context + speaker gender (m/f) in, Hebrew sentences out, with every first-person form agreeing with the speaker's gender. 2. **Classifier (model 2, planned, nothing trained yet):** the same job in reverse. Hebrew WITH nikud in, the speaker's gender out: is this sentence spoken by a man or a woman? Nikud matters because some forms are gendered only in the vowels: "ani rotse" (m) and "ani rotsa" (f), "I want", share the same letters. Use: Hebrew TTS, so a sentence is sent to a male or a female voice (and a learner hears his examples in a male voice), and as a check on the generator's output. Its design, base model and data are not decided yet: don't invent them. Hebrew marks the speaker's gender in two ways. Sometimes in the **letters**: ani holekh / ani holekhet, "I go". Sometimes only in the **vowels**: ani rotse / ani rotsa, "I want", spelled the same for both. ## Model 1: Hebrew-Gendered-Sentence-Creator (confirmed unless marked otherwise) - **Repos.** - GitHub: `Hebrish-ASR/Hebrew-Gendered-Sentence-Creator` (public). - Weights so far are private: `danielrosehill/hebrew-gendered-sentence-creator-gemma-4-e4b-v1` and `-v2`. - Planned public release once V3 is scored: model `Hebrish/Hebrew-Gendered-Sentence-Creator` and dataset `Hebrish/Hebrew-Gendered-Sentences`. Not published yet. - **What it does.** - Input: JSON `{"term", "context", "n", "speaker_gender": "m"|"f"}`. - Output: a JSON array of n unpointed Hebrew first-person sentences, 6 to 18 words each. Each uses the term in the given sense, and every first-person form agrees with the requested gender. Other people in the sentence keep their own gender. - Example, term מדוכא (meduka, "depressed"), f: "אני מרגישה מדוכאה כבר שבועות ולא יודעת למי לפנות", ani margisha medukhet... roughly "I've been feeling (f.) depressed (f.) for weeks and don't know (f.) who to turn to". This sentence is from the training data, written by Claude Sonnet 5.5. - Use case named by Daniel: Hebrew podcast generation for TTS. A male and a female character each need lines spelled for their gender before the text reaches the TTS voice. Gender is a required deterministic switch, m or f, with no neutral option (Daniel's decision, 2026-11-18). - **Base model.** `google/gemma-4-E4B-it`: Apache-2.0, about 8B parameters including embeddings (about 4B "effective"). Daniel chose it; Hebrew-specific models (DictaLM 3.0 1.7B, DictaLM 2.0 7B) were on the shortlist. LoRA rank 32 on the language-model layers only, 69.8M trainable parameters (0.87%). - **Training data.** - Distilled from Claude Sonnet 5.5. Claude also generated 924 everyday Hebrew terms with English context lines, across 20 categories, plus 10 hand-picked pilot terms. - For each term and each gender, Sonnet wrote 10 sentences **from scratch in that gender**. Nothing was written in one gender and then edited into the other; Daniel rejected that as bad design. - One Anthropic batch, `msgbatch_01PoCafFGsoKAuBRB1JkazQ8`: 1,848 requests, US$5.01. - Automatic filters: - a format gate (length, term present, no nikud); - a gender check using Dicta's `dictabert-joint` morphological analyser on Modal, which rejects any sentence whose letter-marked first-person form has the wrong gender. - Even Sonnet slipped on about 1% of female-speaker sentences (wrote ani hared, "I'm anxious", in the masculine for a woman). - Result: 17,725 sentences kept, giving 6,884 training examples (V2/V3) over 834 terms, with 100 terms held out for evaluation. - **Hardware and time.** Modal, one NVIDIA L40S. V1 trained in 30.5 min, V2 in 37 min, then about 6 to 10 min of evaluation generation each. - **Cost (estimate).** - Whole project so far: about US$10.80 = US$5.53 Anthropic + about US$5.30 Modal, plus about US$1.50 for V3. The Modal part is estimated from run times, not read from the bill. - A single clean training run is about US$1.20 to 1.50. - The total was inflated by two retrains: one caused by a bug, one by a killed client. ### Version history The baseline is the same Gemma with only a detailed prompt and no training. Everything is evaluated on the same 100 held-out terms x m/f, 10 sentences each. | | Prompt-only Gemma | V1 | V2 | |---|---|---|---| | broken JSON (of 200 calls) | 6 | 0 | 1 | | female speaker: wrong gender | 19 (3.6%) | 41 (5.8%) | 26 (3.7%) | | male speaker: wrong gender | 2 | 3 | **0** | | sentences where the letters show the gender | 1,019 | 1,485 | 1,446 | - **Before V1, a viability test.** On 10 pilot terms, prompt-only Gemma made 0 gender errors in 200 sentences. That raised Daniel's question: is a fine-tune needed at all? The answer, after the full evaluation: prompting gets gender close. The fine-tune buys reliability: - format that does not break; - far more sentences that actually show the gender; - more natural, varied Hebrew. The prompt-only output is flat and repeats the pronoun ("ani hekhanti", "ani kaniti"). This is Claude's reading, not Daniel's. - **V1 (2026-11-18).** The female errors came from a bug in the data filter. It required the term to appear exactly as given, so it threw out the correct feminine forms of adjective terms: - asirat toda, "grateful (f.)", אסירת תודה; - medukhet, "depressed (f.)", מדוכאה. The female training data for those terms kept only masculine uses. 39 of V1's 41 female errors were "ani asir toda" / "ani meduka" said by a woman. - **V2.** The filter now accepts inflected terms; the data was rebuilt with no new generation and retrained. - Male errors went to 0. - Of the 26 female flags, 15 are still the term left masculine on held-out terms (meduka x10, sokher "renter" x5). The model's own instruction still said "use the term as given". - 5 are genuine slips (er "awake", gar "live", matkhil "start", all in the masculine). - 6 are analyser false alarms. - **V3 (training now, started 22:47).** One change: the instruction now says the term may be inflected to agree with the "I". Expected: most of the 15 term errors disappear. **Not measured yet.** ### Status and next steps 1. Score V3. 2. Write the model and dataset cards. 3. Publish to the Hebrish org on Hugging Face (Daniel approved, 2026-11-18). The model card will recommend pairing the model with the analyser check (reject and regenerate). A fine-tune is still probabilistic, so the guarantee comes from the check. ### Prior work (searched 2026-11-18) Nothing found for Hebrew that generates text with speaker gender as an input. The closest work is Arabic: - first-person gender reinflection, Habash et al. 2019; - the Arabic Parallel Gender Corpus; - User-Centric Gender Rewriting, 2022. Those rewrite existing text; they do not generate. MORPHOGEN (2026) tests LLM first-person gender rewriting in French, Arabic and Hindi. It finds that models also flip the gender of other people in the sentence, which this project's checks are built to catch. ## Why Daniel needs these (from the repos) - **Generating Hebrew to be read aloud.** Two cases: ASR training-data sentences that a male or female reader records, and TTS (podcast characters). A male voice reading feminine forms, or the reverse, is wrong. Unpointed text cannot even tell a TTS engine whether רוצה is rotse or rotsa. - **Why the gap exists (this session's view, not a sourced claim).** Hebrew NLP tooling tags or points existing text; it does not control generation. General LLMs are good at the letters when prompted but weak at nikud.
Mentions
- Arabic Parallel Gender Corpus Gender-labelled subtitle corpus with opposite-gender twins
- Claude Sonnet AI model used for script generation
- Dicta Hebrew NLP project and tools
- DictaBERT Hebrew BERT for nikud and morphological analysis
- Gemma Google's open model family, base for fine-tune
- Hebrew-Gendered-Sentences MIT-licensed dataset of natively gendered Hebrew sentences
- Hebrish Project name for the gendered Hebrew model repos
- Hugging Face Platform for AI models and datasets
- MORPHOGEN 2025 paper on LLM first-person gender rewriting
- User-Centric Gender Rewriting 2022 paper separating speaker and listener gender
Downloads
Plain text transcript file
The episode's record — date, duration, models, sources — with the full transcript
Never miss an episode
New episodes drop daily — subscribe on your favorite platform
New to the show? Start here#6664: Training a $10 Model for Hebrew Speaker Gender
This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities. It may contain errors: disclaimer · report an error