RoBERTa
Meta's robustly optimized BERT variant
Episodes
-
#5438: Why Linguists Left the AI RoomLarge language models grew out of linguistics — so why aren't linguists in the room where they're built? -
#5431: The Other Half of Hugging Face: Why BERT Still Out-Downloads LlamaEncoder models pull over a billion downloads a month. Decoder models pull 397 million. The AI conversation and the download counter are describing ... -
#5408: When Small NLP Models Beat the LLMFeature extraction, fill-mask, token classification — the classic NLP tasks still have a job. Here's when a small model beats a frontier API. -
#2487: Why AI Calls Everything a "Prediction" (Even Images)Machine learning calls everything a "prediction" — even generated images. Here's why the terminology matters more than you think. -
#2066: The Transformer Trinity: Why Three Architectures Rule AIWhy did decoder-only models like GPT dominate AI, while encoders and encoder-decoders still hold critical niches? -
#1799: The Original AI Blueprints: BERT & CLIPBefore GPT, two models changed everything. Discover how BERT and CLIP taught machines to read and see the world.