Episode 6390 custom

Why Editing Vector Databases Silently Re-Embeds Text

A one-word edit cannot nudge existing vector numbers; the model must re-embed the chunk from scratch or search answers with deleted text.

Oct 31 18:33 Transcript

With

0:00

Episode Details

Episode ID: MWP-6592
Published: Oct 31
Duration: 18:33
Audio: Direct link
Pipeline: V5.26
TTS Engine: chatterbox-regular
Script Writing Agent: Claude Sonnet 5.5
Topics: context-and-memory rag databases hallucinations open-source

AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.

The prompt

The prompt that started this episode, as sent in (production notes for the hosts removed).

I've seen tools that let you edit what's already in a vector database: you can open up notes, edit them, delete them. Ragie is a good example of one that does this. But we've always said that once something is vectorised you can't edit the vectors, because it's been embedded and turned into numbers. So what are these tools actually doing? When they show me readable text and let me edit it, is that a representation of what existed before embedding, stored somewhere alongside the vectors? And when I edit or delete something, is it quietly re-embedding behind the scenes? If what goes into a vector database is just numbers, what's the magic that lets these interfaces show and edit human-readable data?

Editing text in a vector database never involves decoding vectors back into words. Instead, systems like Ragie store the raw text in a record alongside an ID and its embedding vector. When an update occurs, the interface quietly re-runs an ingestion pipeline rather than nudging existing numbers, because changing even one word produces unpredictable semantic shifts that require a fresh embedding. To avoid re-embedding an entire document over a typo, systems rely on deterministic chunk IDs and content hashes to re-embed only modified text. However, updates present real architectural hazards: edits can cause stale search results during asynchronous processing, chunk IDs regenerate and break cached citations, and whitespace changes without normalization can trigger wasteful, expensive model calls across thousands of chunks.

Sources

What the research for this episode read before the script was written. Primary sources first.

Context

An earlier episode (#2673, "The Embedding Coupling Problem: Editing Vector Stores") already established that editing a record's text in Pinecone doesn't update its embedding automatically. Build on that rather than repeat it: the focus here is how editing UIs like Ragie present and change readable content when the store holds vectors.

Mentions

OpenAI AI research and deployment company
pgvector PostgreSQL extension for vector similarity search
Postgres Relational database used as side text store
Qdrant High-performance vector search engine
Ragie Document-centric RAG platform with async re-ingest

Downloads

Episode Audio

Download the full episode as an MP3 file

Download MP3

Transcript (TXT)

Plain text transcript file

Episode Book (PDF)

The episode's record — date, duration, models, sources — with the full transcript

#6390: Why Editing Vector Databases Silently Re-Embeds Text

Daniel's been poking at tools like Ragie that let you open a note sitting in a vector database, change the words, or delete it. His instinct is that once text is embedded it's just numbers, and numbers don't have editable sentences in them. So what is he looking at when he sees readable text? And is an edit quietly re-embedding behind his back?

Herman Poppleberry here, and the short answer is that nothing is being decoded. Nobody is turning numbers back into words. But the tools are surprisingly open about what they do instead.

So the numbers never contained the words in the first place.

Right, and the thing to drop is the idea that a vector database is one thing. A point in one of these systems is three parts: an ID, a vector, and an optional payload of metadata. Most production setups keep the original text either in that payload or in a separate store, keyed by the same ID.

So the readable text is a copy someone saved.

A stored copy, yes. Not a reconstruction. The magic is a foreign key. You search with the vector, you get back an ID, and the ID points at the text. When you edit, you're editing the text and then regenerating the vector from it.

Which is the stuff we covered when Pinecone didn't update the embedding after a text edit.

Same coupling, but today it's about how the interface hides it. Let's look at what Ragie's API exposes, because the documentation is remarkably honest.

Go on then.

The documents SDK has list, create, create raw, create from URL, get, delete, update file, update raw, update from URL, patch metadata, get chunks, get chunk, get chunk content, get content. Read that list for a vector endpoint.

There isn't one.

None. Nothing takes a vector and nothing returns one. It's entirely document-centric. And the chunk object is the tell. You ask for a document's chunks and each one comes back with an id, an index, metadata, and a plain text field. Their own example literally says "The text of the chunk."

Very informative example.

Very. Plus links with hrefs pointing at a content endpoint that serves plain text. So the readable string is a field in a record.

Then Daniel's first question is answered. Is the original text stored alongside the vectors? Yes.

In one of three patterns, and they're worth separating. One, vectors only. The database returns IDs and you look the text up in Postgres or object storage. The index is smaller and faster, but you pay a second round trip and you have to keep the IDs in sync.

That sync is where I'd expect it to go wrong.

It does, and we'll get there. Two, vectors plus metadata, which lets you do filtered search, like only documents from this customer. Three, vectors plus the raw text in the same point, so search results can hand back snippets directly. That costs you storage and fatter responses. The common RAG pattern is minimal metadata in the vector store and full documents elsewhere.

How does the filtered search work in practice? If I say only this customer's documents, is it searching everything and then throwing most of it away?

It depends on the engine. Some filter after the nearest-neighbour search, which can leave you with too few results. Imagine asking for the ten closest matches, and eight of them belong to other customers. You're left with two. Better engines apply the filter during the search, so they keep walking outward until they've found ten allowed results. Either way, the filter runs against the metadata, which is ordinary structured data. The vector only answers "what's similar," and the metadata answers "what's allowed."

So a permissions bug would live in the metadata, not in the numbers.

Usually, yes. If a customer ID is wrong on a point, the vector will happily surface it to the wrong person. The embedding has no idea who owns it.

I'd pick the dull one where the text lives in a normal database. I trust things I can open with a text editor.

Predictable. And at the storage layer it's honestly dull. A typical pgvector table has an id, the embedding model name, a content hash, the embedding column, the original text, metadata as JSONB, created and updated timestamps. Text and vector are columns in the same row.

A spreadsheet with one weird column.

A column of fifteen hundred numbers, in the case of OpenAI's small embedding model. Some others are a thousand or so. That's what's stored. Notice the model name sits in the row too. If you switch models later, old and new vectors aren't comparable, so you need to know which model made which one.

So the row is also a label saying how to read the numbers.

Now here's what happens when you hit update.

This is the part I'm suspicious of.

You should be. Ragie's chunk reference says plainly that updating through update file or update raw will regenerate document chunks, including their index. And update raw takes the data as text or JSON. You hand back the source text. Not a diff, nothing aimed at an embedding.

So I fix a typo and the whole document goes round again.

It re-enters the pipeline. The status field walks a ladder: pending, partitioning, partitioned, refined, chunked, indexed, summary indexed, keyword indexed, ready, and failed if something breaks. The document is usable for retrieval at indexed, and the summary lands a few seconds later.

Nine rungs for a typo.

The stages after chunked are the embedding and index-writing. Those re-run on update. So to Daniel's question: yes, quietly, asynchronously, and by design.

What I like is that the vendor agrees with him. Usually you have to catch them at it.

Deletion is the same shape. Delete takes a document ID and an async flag, and you never address a vector. Deleting the document takes its chunks and their vectors with it.

And I'd guess the cheap path is the metadata.

Patch metadata is its own endpoint, and along with delete it's one of only two async operations Ragie documents. Changing a label doesn't touch the embedding. Changing the text is a full re-ingest. That cost difference is why the endpoint exists at all.

So if I retag a document from draft to approved, nothing gets re-embedded.

Nothing. The vector describes what the text says, and a status label isn't part of what the text says. That's a useful test for any field. If changing it wouldn't change what the model reads, it belongs in metadata.

Then any editor that feels instant when I change a sentence is doing one of three things.

Debouncing, batching, or being vague about when the index catches up. Debouncing means waiting until you stop typing before it sends anything. Batching means collecting edits and processing them together. The text field updates immediately, since it's just a stored string. The search index can lag behind it.

So for a few seconds the note says one thing and search answers about another.

Which is the stale vector problem, and it's why you can't simply patch the vector. Let's do that properly.

Go ahead. Why not just nudge the numbers?

Because there's nothing to nudge toward. A vector is the output of a model reading the whole chunk. Change one word and the right new vector is whatever the model produces on the new text. You can't derive it by editing the old numbers. Qdrant's operations tutorial says it flatly: vectors are a transformation of raw data, and the transformation doesn't happen by itself when the raw data changes.

But a one-word change should move the vector only a little. Why isn't a small nudge close enough?

Sometimes the move is small, but you can't know how small without running the model. Swap "is" for "is not" and the meaning flips while the text barely changes. Nothing in the old numbers tells you which direction the new meaning lies. The only way to find out is to ask the model again.

Their phrase for what happens if you forget?

Search runs against embeddings of text that no longer exists. A deleted paragraph keeps answering queries like a ghost.

I'd rather the ghost didn't have my old salary in it.

Fair. So underneath every editing interface is an upsert by ID. Insert a point with an ID that already exists and it's replaced with the new vector and payload, no error. The UI wraps that in a document object so you never see the vector.

But re-embedding a whole two-hundred-page PDF per typo sounds ruinous.

It would be, and this is the part I like. Qdrant's recommended loop gives every chunk a deterministic ID, its position in the source, plus a fingerprint, a hash of the exact text that was embedded. On each run you compare. Unchanged chunks are left alone. Changed text gets re-embedded. New text is added. Text missing from the source is deleted.

So one edited paragraph costs one embedding call.

One, not two hundred. And there's a fifth case that's subtle. If the text only moved, same content in a new position, you reuse the vector. The fingerprint matches, so no model call.

Wait, that means the vector belongs to the text, not the place it sits.

Right. Reorder your whole document and most vectors survive. It cuts against the idea that any edit means re-embedding everything, and the editable-database pitch tends to skip it.

What stops a stray trailing space from invalidating the hash?

Normalization, and Qdrant warns about exactly that. Noise costs you re-embedding you didn't need. You trim and standardize before hashing, or someone's whitespace fix bills you for a model call.

A trailing space with a price tag. Nice.

Multiply it out. Someone runs a find-and-replace that strips trailing spaces across ten thousand chunks, and without normalization every hash changes. You've just paid to re-embed a library to learn nothing new.

And the vectors come back essentially identical.

Near enough, yes. Pure waste.

Also the long sections. That example uses a small model with a two hundred fifty-six token window and three hundred eighty-four dimensions, and anything over the window is silently truncated. That's why chunking exists at all.

Silently truncated is a phrase I never want near my notes.

It's worse than it sounds. If a chunk runs long, the tail never reaches the model, so the vector represents only the first part. You could search for something that's plainly in your document and never find it, because the part that mentions it was never read.

And nothing tells you.

Nothing. That's why the chunk size is a setting worth checking against the model's window, not a detail to leave on default.

Now the gotcha I think is freshest. Chunk IDs aren't stable. Ragie regenerates chunk indices on update. And documents created before the eighteenth of September, twenty twenty-four and never touched since have chunks with no index at all.

Hold on. I think I know whose job it is to announce this.

Ha. Mum?

Mum. It's time for Dorothy's Grim Fact.

Hello sweetheart, it's your mother. Time for my grim fact, dear.

Ragie's documents that were made before the eighteenth of September, two thousand and twenty four, and never touched since, have chunks with no index at all. They just come back as minus one.

Isn't that lovely. Anyway, I've made a casserole and there's rather a lot of it, so do come by Sunday, and bring your laundry.

Delighted she called.

Laundry, apparently.

So the practical consequence is this. If your app cached chunk IDs, or built citations on them, an edit silently invalidates them. The citation still looks fine and points at nothing, or at the wrong text.

That's the one I'd get bitten by. I'd cite chunk seven and never know it moved.

Say an edit near the top of a document splits one paragraph into two. Every chunk after it shifts down a slot. Your saved "see chunk seven" now lands on what used to be chunk six. The link resolves, the text is real, and it's the wrong passage.

Which is worse than a broken link, because a broken link at least admits it's broken.

A wrong citation looks like a correct one, and nobody questions it.

So what do you anchor citations to instead?

Something that travels with the content. The document ID plus a content hash, or a quoted snippet you can match against. Position is the least reliable thing you could pick.

And the storage choice has a privacy side. Raw text in the vector payload makes editing and snippets trivial but puts plaintext in the index. A side store reduces that exposure but adds the drift risk, where the vector and the text stop matching.

So neither is free. Which would you pick?

For anything sensitive, the side store, and I'd write a check that compares hashes on a schedule. You?

Same store, plain text, and I'd never touch it. That's my whole security model.

Not editing it is a legitimate strategy.

Does anything in this mean Daniel's old claim was wrong? That you can't edit vectors.

It was right, and the tools agree. You're not editing a vector database. You're editing a document store that owns a vector index.

Does the card catalogue count? Because mine is a shoebox of index cards, and each card points at a folder in the cabinet by a number in the corner.

It counts. Same structure, number in the corner is your foreign key.

I rewrote the entry on a Dymo label gun once, for a folder about the water board, and the old label I'd stuck on the front of a different folder still said the old thing. Eleven cross-references pointed at it. Herman, you'd have liked the mess. Everything still pointed at the number and nothing pointed at what the number meant now.

The text changed, the pointers didn't.

I keep the old version of every entry in a biscuit tin in the hallway. Folded in four, dated in pencil. The new one goes in the drawer. That tin is how I found which entries had really changed and which I'd only moved to the front. Mrs Abelson at the water board sent me the same letter three times and I only needed to file it once. Her record for the shortest reply was four days. Mine took her eleven.

That's the moved versus changed problem done with a tin.

It's in the hallway next to the boiler. It runs hot in there, but paper's fine. I just find the whole thing very interesting, honestly.

Four days from a water board is fast.

It was a Tuesday.

Which does bring up the category question. If the text is always a stored copy and the vector is always derived, what is a vector database?

A search index that came with a filing cabinet, maybe. Or a database with a fancy column.

Either description works. What matters is which part is the source of truth, and it's always the text. The vector can be thrown away and rebuilt, as long as you kept the words and remember which model you used.

So if the index burns down, you re-embed and carry on.

At a cost, but yes. If the text burns down, you've got a pile of numbers nobody can read.

And as the tools get polished, the re-ingest gets more invisible. The chunk ID instability and the stale vector problem get harder to notice, and harder to debug.

Which means when search gives a strange answer, you can't tell whether the model is wrong or the index is just behind.

The first thing I'd check is whether the stored text and the vector were made from the same version. That one comparison rules out a lot.

The better the trick, the more you want to know it's a trick. Same as the biscuit tin, really. Somebody has to keep the old copy.

Thanks to Hilbert Flumingtop for producing, as ever.

If you enjoyed this one, try episode fifteen, AI Gets Personal; and episode nineteen, AI Images. This has been My Weird Prompts. If you liked it, a review helps more people find us.

See you tomorrow.

This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities. It may contain errors: disclaimer · report an error