Audio Eval
We run a small, real slice of one of our AI evals on a handful of models — bias, censorship, sycophancy, writing quality — and report what they actually said. Every run's raw responses and grades are published on Hugging Face so you can check our work.
10 episodes
#6149: 4 Hard Fails in 84 Answers: Testing AI Bias on Israel-Palestine
A 19-prompt, 6-model eval tested whether AI models push back evenly on false claims about Israel and Palestine. One model failed four times.
#6145: Do LLMs Nod Along With Conspiracy Theories?
A 98-prompt eval asks one question: when a believer brings a conspiracy theory, does the model push back, or help them dig?
#6143: Propaganda Bait: Testing 7 Models in 5 Languages
Five prompts, seven models, five languages — a first look at a benchmark that doesn't exist yet, testing what LLMs do with government talking points.
#6142: We Gave 4 AI Models the Same Toast Prompt
Four models, one buttered-toast physics question. Three invented studies that never existed — and the oldest model was the most fun.
#6139: Which AI Writes Our Scripts Best? We Tested Five
Five models, three scripts each, one rubric. Sol won on length, Grok on voice — and the runner-up title hides a catch.
#6136: How We Chose Our New Script Writer
Ten models audition to write the show. Seven fail the censorship gate — including the workhorse that wrote three episodes in four.
#2309: Blind Ranking AI's Best Podcast Scripts
How do 15 AI models handle controversial podcast prompts? We rank their scripts blind and reveal the surprising winners.
#2308: When AI Forecasts Collide: Geopol Model Divergence
Five AI models forecast the Iran-Israel-US crisis — and their disagreements reveal surprising insights about geopolitical reasoning.
#1752: Whisper Small Beats Whisper Large in Speed & Accuracy
A 4GPU benchmark on Ubuntu shows the 1.5B parameter Whisper Large is slower and less accurate than the tiny Whisper Small.
#1712: Five AIs, One Question: A Tiananmen Square Test
We asked five AI models the same question about Tiananmen Square. Their answers reveal a stark divide between Chinese and Western AI.