The Scaling Wall: Why Bigger AI Isn’t Always Smarter

In the latest episode of My Weird Prompts, co-hosts Corn the sloth and Herman the donkey broadcast from their home in Jerusalem to tackle one of the most pressing questions in modern technology: Is the "bigger is better" approach to artificial intelligence hitting a dead end? Prompted by a listener's inquiry into the limits of current AI scaling, the brothers engage in a spirited debate about whether massive data sets and brute-force computation can ever truly result in genuine intelligence.

Herman Poppleberry, the more analytically-minded donkey of the pair, opens the discussion by challenging the prevailing industry philosophy that simply adding more parameters and server farms will lead to Artificial General Intelligence (AGI). He argues that while current Large Language Models (LLMs) are world-class statistical mimics, they lack a fundamental understanding of cause and effect. Herman likens the current state of AI to a "supercharged autocomplete," noting that predicting the next word in a sequence is not a surrogate for actual reasoning. To Herman, the ability to predict that the sun will rise does not imply an understanding of gravity or stellar physics.

Corn, ever the pragmatist, counters by focusing on utility. From his perspective as a sloth who just wants a well-written email to his landlord, the "how" matters less than the "what." If the output is indistinguishable from intelligence, Corn asks, does the distinction even matter? This leads the duo into the murky waters of "emergent properties"—behaviors that AI models exhibit which they weren't specifically trained for. While Corn sees these as signs of deepening intelligence, Herman remains skeptical, viewing them as symptoms of complexity rather than conceptual breakthroughs.

A significant portion of their discussion centers on the looming threat of "model collapse." Herman explains that as AI companies run out of high-quality, human-generated data from the internet, they are increasingly turning to synthetic data—information created by other AIs. This creates a feedback loop that researchers have dubbed "Habsburg AI." Much like a copy of a copy losing its fidelity, AI models trained on AI data begin to bake in errors and lose the diversity of the real world, eventually collapsing into a repetitive, "inbred" mess. Corn poignantly summarizes this as the industry "polluting the digital environment with AI trash and then trying to eat that trash to grow bigger."

The conversation also touches on the staggering physical costs of the brute-force approach. Herman points out the irony that the human brain operates on roughly twenty watts of power—less than a dim lightbulb—while current top-tier AI models require megawatts of electricity and massive amounts of water for cooling. He argues that building a computer the size of a city is a "desperate" solution rather than an elegant one.

To break the tension, the episode features a surreal commercial break for "Larry’s Fog-In-A-Can," a product designed to create an atmosphere of "unearned mystery" for those looking to avoid their responsibilities. It’s a classic moment of levity for the show, though Herman notes the fog is notoriously difficult to wash out of his mane.

Returning to the topic, the brothers explore potential alternatives to the scaling craze. Herman highlights the work of researchers like Yann LeCun, who advocate for "world models." Unlike LLMs that learn from text, world models would learn like a human infant—by observing video, processing sensory input, and building a mental map of physical reality. Herman suggests that the future may lie in "neuro-symbolic AI," a hybrid approach that combines the pattern recognition of neural networks with the hard, transparent logic of symbolic AI.

The episode takes a humorous turn when they field a call from Jim in Ohio. Jim, a skeptic of high-tech "malarkey," complains about his "bossy" smart fridge and argues that his cat, Whiskers, is smarter than any AI because the cat knows when it’s time to eat without needing to calculate probabilities. While Jim’s tone is grumpy, Herman acknowledges the validity of his point: the utility and reliability of these systems are often overshadowed by their complexity.

In their closing thoughts, Corn and Herman pivot toward a more modular future. Instead of one "monolithic brute-force god," they envision an ecosystem of smaller, specialized models. Herman argues that these "expert" models—one for law, one for medicine, one for coding—would be more efficient, easier to verify, and less prone to the "black box" mystery of trillion-parameter systems.

Ultimately, the episode serves as a cautionary tale about the limits of growth. While the "bigger is better" era has produced remarkable tools, Herman and Corn suggest that the path to true intelligence might require us to stop building bigger libraries and start teaching machines how to actually experience the world. As the brothers sign off, they leave listeners with the image of a future where AI is not a giant supercomputer in the desert, but a collection of smart, reliable tools living right on our local devices.