Daniel's been staring at the compact button and getting philosophical about it. He writes — okay, let me read this properly, because he's asking something genuinely sharp. "Claude Code's compact command is impressive, and it raises a question I've never seen answered properly: are there standalone specialist models whose only job is text compaction? Not a frontier model summarising its own conversation with a clever prompt — an actual small, dedicated compressor that you point at a transcript. Does that thing exist? Does the research literature have one? And if it does, why doesn't my coding agent use it instead of spending a full frontier-model inference on summarising itself?"
That's already three questions, and he's not done.
He's not done. "And then the part that actually worries me: whichever route you take — big model writing a summary, or a purpose-built compressor throwing tokens away — how would you ever know that the thing it dropped wasn't the important bit? Is there any way to verify fidelity, or are we all just trusting that the summary is good because the agent keeps working afterwards?"
That last sentence is the one that should keep people up at night.
It really is. This prompt lands right at the intersection of engineering pragmatism and a uncomfortable open research problem. So let's unpack it. The first question is straightforward: does a dedicated compaction model exist? The second is the one that has no clean answer: how would you ever verify the thing didn't drop what mattered?
And the answer to the first question is actually yes — the research literature has produced several purpose-built compressors, some of them remarkably small. Production coding agents use essentially none of them. That gap is the story.
Start with what exists. Give me the taxonomy.
Three families, and a fourth that's a different thing entirely that people keep confusing with this. Family one is token pruning — black-box methods, text in, text out. The headline act here is something called LLMLingua, from Microsoft researchers, published at EMNLP twenty twenty-three. The idea is you use a small language model — GPT-two small class, or a LLaMA seven billion — to score the perplexity of every token in the conversation, then delete the low-information ones. Coarse-to-fine compression. They claimed up to about twenty-x on certain benchmarks.
Twenty-x compression. That's not nothing.
It's aggressive. And then there's LLMLingua-two, which is the closest thing to a literal answer to Daniel's question. Published at ACL twenty twenty-four Findings. They distilled GPT-four's compression behavior into a small encoder-only token classifier — BERT-base or XLM-RoBERTa class, the kind of thing that fits on a laptop. It labels every token keep-or-drop. Task-agnostic, several times faster than version one. A genuine standalone specialist compaction model.
So the thing exists. It's sitting in a paper. Why isn't it sitting in my terminal?
Because an agent transcript is not prose. This is the part the benchmarks don't capture. A Claude Code conversation is a structured message array with paired tool use and tool result blocks, and those pairs have IDs that must stay consistent. Token-level pruning does not know what a JSON argument looks like. It will happily corrupt a function signature, break the pairing contract, and leave you with ungrammatical fragments that the agent can't parse. LLMLingua-class methods were evaluated on prose and question-answering, not on API-shaped conversation state.
So the compressor that exists was never tested on the thing we'd actually need it to compress.
Family two is soft or latent compression — this is where you get into methods that need white-box access to the model weights. Gist tokens, from Mu and Li and Goodman at NeurIPS twenty twenty-three — you fine-tune the model to squeeze an entire prompt into a handful of activation tokens. AutoCompressors, same year, recursive compression into summary vectors. There's an in-context autoencoder approach from ICLR twenty twenty-four that uses a LoRA encoder to compress context into memory slots.
And these compress hardest of all, I'm guessing.
They do. And they're the least usable in production. Behind a hosted API, you cannot inject someone else's activation vectors. The vectors aren't portable across model versions — your compressed memory dies at the next upgrade. And they defeat prompt caching, which is where the real cost economics live.
That's the thing people miss about compaction economics. It's not just about how many tokens you remove.
Prompt-cache economics invert the whole argument. Preserving a stable, cache-friendly prefix can beat aggressive compression on cost. That's exactly why the cheap tiers run first in Claude Code's architecture — tool-result trimming, then session-memory compaction — and the full summarization call is a last resort. If you replace that with a compressor that breaks the cache, you might save on the compaction inference and lose more on the cache misses.
So family one can't handle the data format, family two can't operate behind an API and kills your cache economics. You mentioned a third family.
RAG-adjacent compressors. RECOMP, from Xu and Shi and Choi, ICLR twenty twenty-four — paired extractive and abstractive compressors, the abstractive one a small T-five-class model distilled from a bigger language model. Purpose-built, small, trained for exactly this job. And then there's the whole lineage of recursive summarization — OpenAI's recursive book summarization work from twenty twenty-one is basically the ancestor of what compact does today.
So RECOMP sounds closer to the mark. Small model, purpose-built.
It does, and it's interesting work. But it was evaluated on retrieval-augmented generation tasks, not on coding agent transcripts. The gap between "compressing documents for retrieval" and "compressing a debugging session where the agent needs to remember that the race condition only triggers on ARM chips" is...
What's the fourth family you said people confuse with this?
KV-cache compression. Methods like H-two-O, StreamingLLM, SnapKV, PyramidKV. These shrink the attention cache, not the text. There's no human-readable artifact, no portability, and they sit below the API boundary. It's a different axis entirely — compressing the attention state rather than compressing the words. Daniel's prompt is about compressing the words.
And that distinction matters because a human-readable summary is something you can inspect, correct, or override. Latent memory is unauditable.
Which is actually one of the reasons production systems prefer the frontier-model approach. A summary you can read is worth a lot of compression ratio. If the agent compacted your three-hour debugging session into a paragraph, you can glance at it and go "wait, you dropped the part about the database connection pool" — and restore it. With latent vectors, you can't do any of that.
So we've got specialists in the literature, some of them small and clever, and none of them in production. The reasons are partly about data format, partly about API boundaries, partly about cache economics, and partly about inspectability. But there's a deeper reason too, isn't there?
The deep one. For a coding agent, this is not a compression-ratio problem. It's a selection problem. What matters is retaining task state and constraints — not maximizing tokens removed per unit of perplexity. The benchmarks in the compression literature reward the wrong objective. They ask "did the answer survive?" They don't ask "did the constraint survive?"
Explain that distinction.
Imagine you're three hours into debugging a race condition. The agent has read through log files, traced call stacks, identified a timing dependency. You compact. The summary preserves what the agent was about to do next — "investigate the mutex lock in the request handler." But it drops the log line that originally revealed the timing dependency in the first place. The agent proceeds confidently, investigates the mutex, finds nothing wrong, and suggests a fix that doesn't address the actual bug. Nothing errors. The code compiles. The tests pass — the tests don't simulate the race condition. You ship it.
And the failure is invisible.
Completely invisible. The compaction preserved the what-to-do-next and dropped the why. And the why was the evidence that the what was even correct.
This is the fidelity problem. Daniel's second question. How would you ever know?
Let me tell you the state of the art on summarization faithfulness evaluation. There are standard benchmarks — SummaC from twenty twenty-two, FactCC from twenty twenty. Newer frameworks like AlignScore and TRUE from twenty twenty-four and twenty twenty-five. These are systems designed to detect when a summary has hallucinated or dropped critical information. And across the board, on adversarial examples, the ceiling accuracy is around seventy to eighty percent.
Seventy to eighty percent. That's the ceiling.
That's the best we can do at detecting whether a summary is faithful. And that's on general summarization benchmarks — news articles, Wikipedia, the kind of text these systems were trained on. Nobody has a faithfulness detector tuned for coding agent transcripts. Nobody.
So we're flying blind.
We're flying with a faulty instrument that we know is faulty and we don't know how faulty for this specific use case. And here's the kicker — research from twenty twenty-four shows that summaries scoring high on faithfulness metrics can still cause task degradation. Because the metrics measure semantic faithfulness, not task-specific criticality. A summary can be perfectly faithful to the semantic content and still drop the one detail that matters for the specific downstream task.
Give me that in concrete terms.
You tell the agent at the start of a session: "never touch the database schema files, and no emoji in commit messages." Three hours later, you compact. The summary captures the debugging work beautifully. It drops those two constraints. Why? Because from a semantic perspective, those constraints are not part of the active task — they're meta-instructions, guardrails. The summarizer deprioritizes them. The agent continues, touches a schema file, uses an emoji in a commit. Nothing errors. You might not even notice.
Those are exactly the kind of instructions I give at the start of a session.
Everyone does. And compaction is structurally incentivized to drop them, because they look like low-information preamble compared to the dense technical work that follows. The practical mitigation here is architectural, not algorithmic. Put durable rules outside the conversation entirely — system prompts, CLAUDE dot MD files, places compaction can't reach.
That's actually actionable. But it doesn't solve the verification problem. It just moves the vulnerable information to a safe location.
Right. And you can't move everything to a safe location. The specific debugging context, the chain of reasoning, the false paths you already eliminated — that stuff lives in the conversation, and compaction has to handle it.
So let me press on this. Daniel's question — "are we all just trusting that the summary is good because the agent keeps working afterwards?" — is that what we're doing?
Yes. And "the agent keeps working" is a deeply flawed signal, for three reasons. One, failures can be delayed — the agent makes a decision based on compacted context that looks fine now and causes a problem three turns later. Two, the agent may compensate for missing context by asking redundant questions — you've seen this, the agent suddenly asks about something you discussed an hour ago, and you think it's being thorough, but actually it forgot. Three, silent degradation — slightly worse code quality, slightly less relevant suggestions, none of it flagged as an error.
That third one is the scary one. You don't even get a signal that something went wrong.
Compounding across multiple compactions, it gets worse. Summaries of summaries. Each generation loses a little more fidelity, and there's no natural correction mechanism. The system that Claude Code uses — demanding verbatim user quotes in the summary — is actually a deliberate anti-drift measure. It anchors the summary to exact language from the user, preventing the slow semantic drift that happens when each summary paraphrases the previous summary.
So someone thought about this.
Someone definitely thought about this. The verbatim quote requirement is not an aesthetic choice. It's a fence against compounding error.
Let me ask the uncomfortable question, then. Is there any path to actually verifying compaction fidelity, or is the problem fundamentally unsolvable?
The uncomfortable answer is that verification, in the strict sense, would require comparing the compacted context against the original full context — which defeats the purpose of compaction. If you have to keep the original around to verify the summary, you haven't actually saved any context window.
So it's circular.
It's circular in the strict sense. But there's speculative work on approximations. The idea is you build a small verifier model that estimates information loss without needing the full original. One approach: compare perplexity distributions between the original and compacted contexts. If the compacted version has significantly different perplexity patterns on key entities, that's a red flag. Another approach: generate probe questions from the compacted context and see if the agent can answer them consistently with what the original context would have supported. Early research on information-theoretic approaches to summarization evaluation from twenty twenty-four and twenty twenty-five suggests this might be possible, but it's not production-ready.
So we can imagine a compaction verifier. We can't build one yet.
We can't build one that's reliable enough to trust. And the stakes are asymmetric — a false positive from the verifier means you're flagging a good compaction as suspicious, which is annoying. A false negative means you're trusting a bad compaction, which is the failure case we're worried about. The verifier has to be extremely good at not missing things, and we're not there.
What would "good enough" even mean here? Ninety-nine percent accuracy?
Honestly, I don't know that a percentage captures it. The issue is that the importance of any given token is context-dependent in a way that's hard to formalize. The log line that reveals the race condition — in a different debugging session, that exact same log line might be irrelevant. The compressor can't know what's important without understanding the task, and understanding the task is what the frontier model is doing when it writes the summary.
Which circles back to why we're using frontier models for this in the first place.
Yes. The compaction task is actually harder than it looks, because it requires understanding what the agent will need next — not just what was said. That's a reasoning task, not a compression task. And reasoning is what frontier models are good at.
So the dedicated compressor exists in the literature, but using it would mean optimizing for the wrong thing — token reduction rather than task preservation. And even if we solved that, we'd still have no way to verify the result.
That's the state of play. And I want to be clear — this is not a criticism of how compaction is currently implemented. Given the constraints of API boundaries, cache economics, inspectability, and the fundamental selection-versus-compression distinction, using a frontier model for summarization is probably the right engineering decision today. The uncomfortable part is admitting that we can't verify the output.
Which means every compact is an act of faith.
Every compact is an act of faith. And the faith is grounded in something real — the agent usually does keep working, the summaries are usually pretty good, the engineers who built this clearly thought hard about the failure modes. But "usually" and "pretty good" are not verification. They're heuristics.
Let's talk about what you can actually do, then. Practical steps for someone using these tools today.
First, move durable constraints out of the conversation. System prompts, project files, CLAUDE dot MD — anything that should survive compaction needs to live somewhere the compactor can't reach. If you tell the agent "never touch this file" in a chat message, assume that instruction dies at the next compact.
Second?
Treat compacted context as best effort. When the agent starts making strange mistakes — asking questions it should know the answer to, suggesting approaches you already rejected, missing constraints you know you gave it — the first thing to suspect is compaction loss. Be ready to re-provide context. Don't assume the agent remembers just because you discussed it three hours ago.
That's a mindset shift. Most people blame the model when it forgets things.
Sometimes it's the model. Sometimes it's the compaction. You can't tell which from the outside, so the practical response is the same either way — re-provide the context.
Third?
For critical work, consider running compaction in parallel with a diff check. Keep the original context available — not in the active conversation, but saved somewhere — and compare agent behavior on both versions for a few turns before committing to the compacted version. If the agent gives different answers or misses constraints on the compacted version, you've caught a fidelity problem before it compounds.
That's a manual process, but I can see doing it for high-stakes work.
It's manual and it's tedious, but it's the only verification method available right now that doesn't require building a research-grade evaluation pipeline. And it works because you're using the agent itself as the detector — if the agent behaves differently with the full context versus the compacted context, something got dropped.
What about the transparency angle? Should these tools be telling us more about what happened during compaction?
I think the field needs a compaction transparency standard. After every compact, the agent should expose: how many tokens were removed, what key entities or constraints were preserved, and ideally some kind of confidence score. Right now, compact is a black box — you press the button, the context shrinks, and you hope for the best. Even a basic summary of what was dropped would let users calibrate their trust.
"I dropped forty thousand tokens, preserved your three open TODO items, and I'm eighty percent confident I kept all your explicit constraints." That would change how people use these tools.
It would. And it's technically feasible — the compaction step already has to identify what to keep and what to drop. Exposing that decision as structured metadata is an engineering problem, not a research problem.
So we've got specialists that exist but aren't used, a fidelity problem with no clean solution, and some practical mitigations that amount to "don't trust the compaction." What's the open question that defines where this goes next?
Two open questions, actually. First, will we ever get a dedicated compaction model that's good enough to displace the frontier-model approach? The economics might shift if someone builds a small model trained specifically on agent transcripts — not prose, not QA, but the actual structured conversation format that coding agents produce. The training data exists, it's just proprietary and constantly shifting.
And the second question?
Can we build a verifier that doesn't require the original context? Something that can look at a compacted summary and estimate information loss from the summary alone — perplexity distributions, consistency checks, probe questions. The twenty twenty-four and twenty twenty-five research suggests it's possible in principle, but nobody has demonstrated a production-ready system. If someone cracks that, compaction stops being an act of faith and starts being an engineering decision you can actually evaluate.
And until then?
Until then, every compact is a bet. The bet is usually right. But you should know you're making it.
If you take one thing from this conversation, it's that compaction fidelity is an unsolved verification problem — and "the agent kept working" is not evidence that the summary was good. It's evidence that the failure wasn't catastrophic enough to notice.
And the practical corollary: move your durable constraints somewhere compaction can't reach them. System prompts, project files, not chat messages. That one change eliminates the most common and most invisible failure pattern.
Which leaves us with the open questions. Will a dedicated compaction model ever be good enough, or does the selection-versus-compression distinction mean frontier models will always win on quality? And can we build a verifier that doesn't need the original context — something that lets us actually evaluate compaction rather than just trusting it?
As context windows grow, this gets more important, not less. The bigger the original context, the more information a compaction can silently drop. Longer windows make the fidelity problem harder, because the signal-to-noise ratio gets worse and the compressor has more opportunities to drop the wrong thing.
Thanks to Hilbert Flumingtop for producing. This has been My Weird Prompts. If you've got a weird prompt about something that keeps you up at night — especially something technical that you suspect has an uncomfortable answer — send it to us. Show at my weird prompts dot com. We'll read it, research it, and maybe lose some sleep over it too.
We'll be back soon.
