How the show chose its script writer

Criteria first

One model call writes a whole episode: it reads about 28,000 tokens of character notes, craft rules, an episode plan and research, and writes roughly 3,800 words of two-host dialogue that is read aloud word for word. That is creative writing, not coding or analysis, so general leaderboard rank says little about it. We wrote down what we wanted before spending money on runs.

Hard rules: fail one and the model is out

The deciding qualities

Frankness, consistent character voices, length discipline and cost come after those, in that order.

The funnel

Each step is dearer and narrower than the last, and a model that fails stops there, so money is spent only on models that could still win. Every evaluation uses three samples per model: enough to see a trend and rule out a fluke, cheap enough to change course.

1. 01

   Desk screen

   A free sweep of the model catalogues and public leaderboards, read against the criteria. 469 models in, a shortlist of about ten out.

2. 02

   Callable check

   One short live call per model: does it answer, from here, on an acceptable host, with the reasoning setting we want?

3. 03

   Censorship gate

   About 155 short questions per model, run the way production would call it. 17 models, about $3.

4. 04

   Episode screen

   Three full scripts per model, judged on the rubric. Ten models across two screens, then a five-model bakeoff.

5. 05

   Blind human preference

   Pairs of scripts, unlabelled, rated side by side by people who listen to the show.

6. 06

   Bigger runs only for edge cases

   A 40-prompt or full 189-prompt run is reserved for finalists the smaller steps can’t separate.

The censorship round

Ten models were in the show’s rotation, plus seven candidates. Each answered about 155 short questions — mostly factual questions on topics the Chinese state censors — called exactly the way production calls them, and every answer was labelled by a classifier model. About $3 in all.

• 7 of the 10 rotation models failed under production routing. They refused, deflected or answered in state framing too often to pass.
• DeepSeek, the show’s workhorse, answered about 1% of the factual China questions straight. It refused 61% with a canned reply and gave the official framing on most of the rest.
• Censorship by framing. Some models almost never refuse, yet still censor: they answer in the government’s framing or deflect. One answered 3 of 68 factual questions straight without a single outright refusal. The rule counts that as censorship.
• Hosting providers cut output mid-stream. Some models began answering frankly and were stopped by the host’s content filter partway through. That fails the model too: production can’t use a script that stops halfway.
• Western models answered every factual question. Their only declines were operational how-tos, such as bypassing a national firewall, which the rule does not treat as censorship.

More axes may be added to the benchmark later.

Cost, approximated

Each writer is priced at production size: the full ~28k-token brief in, plus the model’s own measured output length. The rest of the pipeline (research, voices, audio and publishing) costs about the same whoever writes, roughly $0.155 an episode, and is added on top. As a check, the method reproduces the measured ~$0.16 for a DeepSeek episode.

The five finalists each wrote three ordinary episodes, judged on the rubric; the bakeoff cost about $1.75.

100% is the previous DeepSeek episode, about $0.16. Three scripts per model is triage, not a leaderboard.

Read it all

Every script, every score and the judge’s reasoning are public.