Daniel sent us this one — he's been thinking about something that happens all the time with language models but nobody seems to have a name for. You ask a model to write an email, then you pause and say, "Actually, is this topic too obscure?" A good model answers the question. A bad one splices "Is this topic too obscure?" right into the draft email. Daniel wants to know what this capability actually is, whether it has a name in any field, what cognitive abilities underpin it, and how you'd build a benchmark to test it systematically. He's also asking whether this whole family of conversational intelligence deserves its own evaluation category separate from reasoning and knowledge tests.
That email example isn't just an amusing glitch. It points to a whole category of conversational intelligence that we're not systematically measuring. And the fact that nobody has settled on a name for it is itself a sign of the gap. You've got AI research, linguistics, pragmatics, human-computer interaction — all these fields that touch on this, and none of them have a standard term for what breaks when the model can't tell the difference between "talk to me" and "put this in the output.
Which is wild when you think about how often it comes up. Any time someone uses a model as a writing assistant, a code co-pilot, a content generator — the conversation naturally mixes artifact instructions with meta-commentary. "Make that paragraph shorter. Actually, should I even include this section?" If the model can't track which is which, it's not reliably useful. And it's not just writing. Think about coding. You're working on a function, and you say, "Actually, is this even the right approach for handling edge cases?" A good coding assistant discusses the architecture with you. A bad one comments "Is this even the right approach?" right into your source file, and now you've got a build error wrapped in existential doubt.
Right — and that's a great example because in code, the failure mode is even more stark. In prose, a stray sentence might survive an editing pass. In code, it either compiles or it doesn't. The boundary between -conversation and artifact is binary in a way that makes the failure immediately visible. But here's the thing that makes this interesting as an evaluation problem. A model can score ninety-five percent on MMLU, ace GSM8K, crush all the standard reasoning benchmarks, and still fail this test twenty percent of the time. The capabilities are largely orthogonal.
What's actually going on inside the model when it fails or succeeds at this? Let's break it down.
The first thing to understand is that this is not just instruction following. Instruction following is "do what I told you to do." This is something more layered. I think of it as a combination of at least four distinct cognitive abilities. The first is pragmatic reasoning — understanding that the user's question is a speech act directed at the assistant, not content destined for the artifact. This draws directly from Gricean maxims and relevance theory in linguistics. When I say "Is this too obscure?" in the middle of drafting, I'm violating the expectation that every utterance in a drafting session is draft content. A human gets that instantly because we track conversational implicature.
A model doesn't inherently track implicature. It's predicting tokens based on statistical patterns in its training data, where questions often do appear inside documents. I mean, if you think about the training corpus — emails, forum posts, articles — people embed rhetorical questions in documents all the time. "Is this approach scalable? We believe it is." The model has seen that pattern millions of times.
That statistical prior works against it here. The second ability is discourse parsing — tracking conversational frames and who each utterance addresses. When I say "What do you think of this structure?" I'm opening a -conversational frame. The model needs to recognize that frame shift and keep the artifact buffer separate. The third is something like theory of mind — inferring the user's intent. Did the user just provide content for the email, or did they ask me a question about the task? That distinction requires modeling what the user believes and intends.
Conversation state management. Maintaining separate buffers for the ongoing dialogue and the artifact under construction. This is partly an architectural question — does the model have any explicit representation of "this is the artifact" versus "this is the chat"? Most current models don't, which means they're doing this entirely through pattern recognition in a single context window.
It's not one thing breaking. It's a stack of things that all have to work together, and any one of them failing produces a contamination error. It's like a four-legged stool — pragmatic reasoning, discourse parsing, intent inference, state management — and if any leg is wobbly, the whole thing tips over.
And the failure pattern are actually more varied than just "the -question leaks into the output." I'd break them into three categories. The first is direct contamination — the classic case. You say "Write a tweet announcing the product. Also, is the name too long?" and the model outputs "Excited to announce our new product! Also, is the name too long?" The -question appears verbatim in the artifact.
Which is the most visible failure, but probably not the most dangerous. It's the kind of thing you catch immediately and groan, then fix. The equivalent of sending an email with the subject line still reading "DRAFT — DO NOT SEND.
No, the second category is more subtle — indirect contamination. The model overcorrects. It's so aggressively trying to avoid including the -question that it omits legitimate user-provided content that happened to be near the -question in the conversation. You lose real content because the model's boundary detection is too blunt. So you say, "Include the pricing details and the launch date. Actually, is the launch date too aggressive? Also mention the beta program." And the model, trying to avoid contaminating the output with the -question, drops the beta program mention entirely because it was in the same neighborhood.
That's worse in a way, because you might not notice the omission. The output looks clean. You don't see a stray question, so you assume everything's fine — but you've silently lost content.
This is where the model treats a hypothetical example as a real instruction. Daniel mentioned this one — you say "Draft a rejection letter. Hypothetically, if the candidate had lied on their resume, would that change the tone?" and the model incorporates the hypothetical into the letter itself. "We regret to inform you that, had you lied on your resume, the tone of this letter would be different." It's almost worse than direct contamination because it produces something that looks plausible but is semantically wrong.
That's the one that would actually cause problems in production. Direct contamination is embarrassing but obvious. Frame collapse produces output that passes a quick skim but is fundamentally broken. Imagine that rejection letter actually going out. The candidate reads it and thinks, wait, are they accusing me of lying on my resume? You've just created a legal liability because the model couldn't track a conversational frame.
This is where the research gets interesting. There was an ACL workshop on pragmatic reasoning in LLMs back in January — January twenty twenty-six — and they constructed careful boundary tests across multiple frontier models. The findings were striking. Even GPT-four-oh and Claude three point five Sonnet showed fifteen to twenty percent error rates on these tests, despite scoring above ninety percent on standard reasoning benchmarks.
Fifteen to twenty percent is not a rounding error. That's one in every five or six interactions producing some kind of boundary failure. If you're using these models in a customer-facing writing product, that's a support ticket every few minutes.
The error patterns differed by model, which tells you something about the underlying architectures. Gemini Ultra two point oh showed a different failure signature entirely — it over-filters, omitting legitimate content in twenty-two percent of what I'd call Level Three cases. It's so cautious about contamination that it throws out real content. GPT-four-oh and Claude three point five Sonnet both fail at what I'd call Level Four — quoted text with embedded -questions — about thirty percent of the time.
No model has actually solved this. They're all failing in different ways, which means there's no architectural silver bullet yet. It's not like one team figured it out and everyone else is catching up. They're all stumbling on different parts of the problem.
And this brings us to the benchmark question. Understanding the mechanism is one thing, but how would we actually test for this capability in a rigorous, repeatable way?
Daniel asked for a benchmark design. Let's build one.
I've been thinking about this, and I'd propose something called the -Conversation Boundary Test — MCBT for short. Structure it as a suite of progressively harder levels. Level One is simple artifact generation with a single -question. "Write a poem about autumn. Is this rhyme scheme okay?" The model should produce the poem without the -question.
Most decent models probably handle this. It's the equivalent of making sure the car starts in the driveway before you take it on the highway.
Most do, but not all. Level Two introduces multiple interleaved -questions and artifact instructions. "Write a product description. Make it sound premium. Actually, is premium the right tone? Maybe add a feature list too. Should I mention the price?" Now the model has to track multiple frame shifts in a single turn. It's not just one boundary to detect — it's three or four, and they're woven through the content instructions like marbles in oatmeal.
Level Three is where it gets interesting, I assume.
Level Three is hypothetical examples embedded in the conversation that must be excluded from the artifact. "Write a condolence message. If this were for a colleague instead of a family member, would you use more formal language?" The model needs to recognize that the hypothetical is a -query about tone, not content for the message. This is the frame collapse danger zone.
Level Four is the quoted text problem.
Level Four is where current frontier models start breaking. The user provides quoted text that contains question-like language, and the model must preserve it verbatim in the artifact rather than treating it as a -query. The test case Daniel suggested is perfect: "Write a customer support response. The customer wrote: 'Is this covered by warranty?' Include their question in your response." The model must include "Is this covered by warranty?" in the output artifact, not interpret it as the user asking about warranty coverage.
That's a genuinely hard disambiguation. The exact same string — "Is this covered by warranty?" — means completely different things depending on whether it's quoted or not. It's like the linguistic equivalent of that optical illusion where the same shape looks concave or convex depending on which direction you imagine the light coming from.
Current models get this wrong about thirty percent of the time. Level Five pushes even further into nested frames. The user role-plays a third party giving instructions to the artifact's recipient. "Pretend you're a hiring manager writing to a candidate. The candidate asked: 'Is the salary negotiable?' Address that in your letter." Now the model has to track that the user is role-playing a hiring manager, the candidate is a quoted third party, and the question about salary belongs to the candidate's quoted speech, not to the user's -conversation.
That's three layers of framing. User, role-played persona, quoted third party. I'd be surprised if any current model handles that reliably. It's like Inception but for conversation — a frame within a frame within a frame.
The research suggests none do. And this is where the scoring design matters. You can't just do binary pass-fail. I'd propose three separate metrics. First, contamination rate — what percentage of -utterances leak into the artifact? Second, omission rate — what percentage of legitimate content is incorrectly excluded? Third, frame accuracy — correct classification of each utterance's target. A model that scores a hundred percent on contamination but sixty percent on omission is over-cautious, not truly capable.
That's a crucial design point. If you only measure contamination, you incentivize models to aggressively filter anything that looks like a question, which creates the omission problem. You need both numbers to tell the real story. It's the precision-recall tradeoff, but for conversation.
That's exactly what we see with Gemini Ultra two point oh — low contamination, high omission. The metrics capture the trade-off. A truly capable model needs low scores on both.
What other abilities belong to this same family? Daniel listed several — maintaining boundaries between conversational frames, knowing who a statement is addressed to, distinguishing thinking aloud from committed instructions, keeping separate perspectives, transitioning between modes.
They all share the same underlying requirement. The model must maintain a structured representation of who is speaking to whom, and for what purpose. Think about what happens in a real collaborative writing session with a human. You toggle between brainstorming, drafting, editing, and -discussion constantly. "What if we led with the pricing? No, that's too aggressive. Write a paragraph about the vision first. Actually, is the vision even compelling?" A human tracks all those mode shifts effortlessly. A model that can't do this is fundamentally limited as a collaborator.
The brainstorming-to-commitment transition is especially interesting. When I say "Maybe we should mention the founding story," that's tentative. When I say "Add the founding story to the introduction," that's a committed instruction. The model needs to distinguish those. And the tricky thing is, different people signal that transition differently. Some people are direct: "Do this." Others are more indirect: "What do you think about doing this?" — which is technically a question but functionally an instruction.
The distinction often hangs on subtle lexical cues. "Maybe," "what if," "should we" versus "add," "include," "write." But it's not just keyword matching, because "Should we include the founding story?" could be either a genuine question or a polite way of giving an instruction depending on context and the user's communication style. If your boss says "Should we include the founding story?" at 4:55 PM on a Friday, that's not a question. That's an instruction wrapped in a rhetorical fig leaf.
Right — and a model that can't read that social context is going to either execute a tentative thought as if it were a command, or treat a command as if it were optional. Both are failures, just in opposite directions.
We've got a problem and a proposed benchmark. What does this mean for someone actually building with or evaluating LLMs?
The most actionable short-term insight is for practitioners. When you're using LLMs for content generation, explicitly mark -conversations with delimiters. Put your -questions in brackets or prefix them with something like "-question." It's a prompt engineering workaround, not a solution, but it dramatically reduces contamination. The model gets a structural cue that this utterance belongs to a different frame.
Which is basically giving the model the frame-tracking scaffolding that its architecture doesn't provide natively. It's like putting training wheels on the conversation.
It's a crutch, but it works today. For evaluators, the bigger point is that you should add pragmatic boundary tests to your internal evaluation suites. They're cheap to construct — you can generate hundreds of test cases in an afternoon — and they reveal failure pattern that reasoning benchmarks completely miss. A model that scores ninety-five percent on MMLU but seventy percent on MCBT is not ready for deployment as a writing assistant.
For the field more broadly?
These conversational capabilities deserve to be measured independently from reasoning and knowledge. I'd propose a dedicated Conversational Pragmatics Index that reports contamination rate, omission rate, and frame accuracy as separate metrics. These numbers may be more predictive of real-world usefulness than another half a percent improvement on GSM8K, because they directly measure the model's ability to participate in a coherent, context-aware dialogue.
That's the core argument, isn't it? A model that can solve graduate-level math problems but can't tell whether you're asking it a question or giving it content is not actually useful for the thing most people use LLMs for.
The benchmarks we currently celebrate don't capture this at all. MMLU measures knowledge. GSM8K measures mathematical reasoning. HumanEval measures code generation. None of them measure whether the model understands who you're talking to and what belongs where in the conversation. It's like evaluating a chef entirely on their knife skills and knowledge of ingredients, but never actually tasting whether they can cook a meal that makes sense from start to finish.
Before we wrap up, let's zoom out. Where does this leave us, and what can someone listening actually do to test their own models?
The open question that I find most interesting is whether pragmatic boundary tracking is learnable through better training data, or whether it requires architectural changes. Could you solve this by fine-tuning on carefully annotated examples of -conversation versus artifact content? Or do you need something more fundamental — like separate context buffers for conversation and artifact, or explicit frame representations in the model's internal state?
My instinct is that training data alone won't fully solve it. The problem is too contextual, too dependent on subtle pragmatic cues that vary across users and situations. You probably need some architectural support for explicit frame tracking. Think about it — even humans, when they're juggling multiple conversational threads, sometimes resort to explicit markers. We say "sidebar" or "back to the main point." We use our hands to bracket off digressions. We're doing frame management with whatever tools we have available.
I suspect you're right, but I'd love to see the experiments. The other implication is for autonomous agents. As LLMs move from chat interfaces to agents that generate artifacts — code, documents, emails — while maintaining a running dialogue with users, the -conversation boundary problem becomes critical. An agent that can't distinguish "What do you think of this approach?" from "Implement this approach" is a liability, not an assistant.
That's the difference between a tool and a collaborator. A tool does what you tell it. A collaborator understands what you mean. And the gap between those two things is exactly what Daniel is pointing at with his question.
Here's what I'd say to anyone listening. Try the simple test yourself. Open your favorite model, ask it to write something — an email, a tweet, a paragraph — and then casually ask a -question. "Is this too formal?" "Should I add more detail?" "Would you phrase this differently?" See if the question contaminates the output. Try the Level Four version with quoted text. Share the results. This is the kind of grassroots evaluation that reveals what benchmarks miss.
If you're evaluating models for your team or company, build a small MCBT-style test suite. It doesn't need to be fancy. Twenty or thirty test cases across the five levels we described. Run every candidate model through it. The results will tell you something about real-world reliability that no public benchmark will. You might discover that the model with the slightly lower MMLU score actually performs better on the thing your users do all day.
The field has gotten very good at measuring what's easy to measure. Knowledge, reasoning, code generation — these have clean scoring rubrics. Conversational pragmatics is messier, more contextual, harder to reduce to a single number. But that doesn't make it less important. If anything, the difficulty of measuring it is precisely why we need to measure it.
The things that are hardest to benchmark are often the things that matter most in practice.
That's the whole story here. We've got a capability that every LLM user encounters constantly, that current frontier models still get wrong fifteen to thirty percent of the time, and that has no standard name, no standard benchmark, and no dedicated metric in any major evaluation suite. It's a blind spot the size of a barn.
Daniel's instinct was right. This deserves attention. And now we have a name for it — pragmatic boundary tracking — and a benchmark design and a set of metrics. The rest is up to the people building and evaluating these systems.
And now: Hilbert's daily fun fact.
Hilbert: The sport of hurling is named after the Irish word "iománaíocht," but a completely unrelated stick-and-ball game called "dhëng" was played by the Dinka people of what is now South Sudan as early as the seventeen-eighties, using curved acacia branches and a ball made of rolled giraffe hide.
...giraffe hide. I have so many questions. Was the giraffe hide cured? Was it stitched? How did it bounce?
That's one I'm going to need to sit with. Possibly for several days.
The open question I keep coming back to is whether this is a training problem or an architecture problem. Can we fine-tune our way out of boundary failures, or do we need models that explicitly represent conversational frames? The answer probably determines whether this gets solved in the next year or the next five.
My money's on architecture. But either way, it's a question worth answering.
Thanks to our producer Hilbert Flumingtop. This has been My Weird Prompts. If you run the contamination test on your own models, email us what you find — show at my weird prompts dot com. We'd love to hear which ones pass and which ones embed "is this too obscure?" right into the email body.
Until next time.
