I was reading a post from a lead engineer at a major fintech firm yesterday who said their biggest regret over the last two years wasn't their choice of large language model or their cloud provider, but their embedding model. They are currently staring down a six-figure bill and three weeks of downtime just to re-index their primary knowledge base because they realized their initial choice had a ceiling they finally hit. It is a massive operational failure that most people do not see coming until it is too late.
Herman Poppleberry here, and that is a classic case of what we are starting to call vector debt. It is the silent killer of AI architecture because it is so easy to overlook at the beginning when you are just trying to get a prototype working. You pick whatever is the default in the library you are using, you embed ten thousand documents, and everything feels great. But that choice is essentially a permanent marriage to a specific geometric coordinate system. If you want to change that model later, you are not just changing a line of code; you are essentially changing the laws of physics inside your database.
It really is a trap, because unlike a standard database where you can just run a script to change a column from an integer to a string, you cannot just script your way out of a vector mismatch. Today's prompt from Daniel is about exactly this crisis of embedding lock-in and how to architect vector storage for longevity. He is asking about the impact of different models on retrieval accuracy and whether there is a safe setting for organizations that do not want to be stuck re-embedding their entire world every time a better model comes out.
It is a brilliant question because it touches on the fundamental physics of how these systems work. When we talk about embeddings, we are talking about taking a piece of text and turning it into a list of numbers that represents its meaning in a high-dimensional space. The model determines where the points go. If you use OpenAI's text-embedding-three-large, it creates a map of human language where similar concepts cluster together in one way. If you use Google's Gemini Embedding two, which just came out earlier this year, it creates a totally different map.
I like the map analogy. It is like trying to take GPS coordinates from a map of Mars and apply them to a map of Earth. Even if the locations represent the same thing in some abstract sense, the numbers are completely meaningless when you cross over. You cannot just add or subtract a constant to fix it. The underlying geometry is incompatible. This is why we say you cannot "convert" vectors. If you move from model A to model B, every single vector in your database becomes a random point in the new space.
That is the core of the lock-in. If you have one hundred million vectors in a database and you decide you want to switch from a proprietary model to an open-source one like BGE-M-three from the Beijing Academy of Artificial Intelligence—or "Bay-Eye" as they are often called—you have to pull every single one of those original documents out of storage, run them through the new model, and rebuild your entire index from scratch. For a large enterprise, that is not just a technical challenge; it is a massive operational and financial hurdle. We are talking about millions of tokens of inference and weeks of compute time.
And that leads us perfectly into the dimension regret problem. Daniel asked about the impact on accuracy and parameters. Most people think more dimensions always equals better retrieval, so they just crank everything to the max. They see that text-embedding-three-large supports three thousand seventy-two dimensions and they think, well, more numbers must mean more detail, right?
That is a common misconception that actually leads to a lot of wasted money and compute. While it is true that more dimensions can capture finer nuances, there is a point of diminishing returns that hits much earlier than people realize. In fact, if you look at the Massive Text Embedding Benchmark leaderboard, or MTEB, as of March twenty twenty-six, the top models are not necessarily the ones with the most dimensions. Cohere embed-v-four is leading at sixty-five point two, followed closely by OpenAI's text-embedding-three-large at sixty-four point six. But here is the kicker: you can get ninety-nine percent of that performance with half the dimensions if the model is architected correctly.
This is where the Matryoshka Representation Learning, or M-R-L, comes in. This is probably the biggest breakthrough for future-proofing that we have seen in the last couple of years. Herman, I know you have been diving deep into the research papers on this. Explain the nesting logic, because it feels like magic the first time you hear it.
It really is elegant. In a traditional embedding model, the information is spread out across all dimensions somewhat equally. If you have a one thousand dimension vector and you cut off the last five hundred, you lose the meaning entirely. But Matryoshka models, like the ones from OpenAI, Voyage AI, and the new Gemini Embedding two, are trained specifically to pack the most important semantic information into the first few dimensions.
Like a Russian nesting doll, hence the name.
Precisely. The model learns to make the first sixty-four dimensions a viable embedding on their own. Then the first one hundred twenty-eight are an even better one, and the first five hundred twelve are better still. What this means for a developer is that you can embed your data once at the highest possible resolution, say three thousand seventy-two dimensions, but you only store and search against the first seven hundred sixty-eight. If you realize later that you need more precision, you do not have to re-embed anything. You just start using more of the dimensions you already have stored.
That is the ultimate architectural insurance policy. It solves the dimension regret problem because you are not locked into a specific storage footprint forever. You can start lean and scale up, or start heavy and truncate down if your cloud bill is getting out of control. Google is actually recommending seven hundred sixty-eight dimensions as the production sweet spot for their newest Gemini models. They found it provides near-peak quality but at one-quarter the storage and compute cost of the full vector.
And the math on that is staggering. Research shows that truncating a Matryoshka vector to just eight point three percent of its original dimensions—say, taking a seven hundred sixty-eight dimension vector down to sixty-four—still preserves over ninety-eight percent of the full retrieval performance. When you consider that a lot of these vector databases charge based on the amount of memory or disk space you are using, that sixty to seventy-five percent reduction in vector size is a massive deal. It is the difference between needing a cluster of ten machines versus just two or three.
But let's pivot from the math to the infrastructure. Daniel asked about a safe setting for the vector database itself. We have seen this massive explosion of dedicated vector stores like Pinecone and Weaviate, but then we have the "just use Postgres" movement which we talked about in episode eleven twenty-three. Where do you stand on this for someone who wants longevity in twenty twenty-six?
If you are an organization that already has a standard relational database, specifically PostgreSQL, then P-G-vector is the boring, safe infrastructure choice that I would recommend nine times out of ten. The reason is simple: data gravity. If your metadata, your user records, and your actual content already live in Postgres, moving your vectors there eliminates the need for a complex synchronization pipeline between two different databases. We are seeing P-G-vector and the newer P-G-vector-scale extension handle workloads that used to require specialized hardware.
It avoids the "split brain" problem where your vector database thinks a document exists but your primary database says it was deleted five minutes ago. That is a huge headache to manage at scale.
Plus, you are using a tool that has been around for decades and has a massive ecosystem of support. If you are worried about lock-in, being on an open-source standard like Postgres is much safer than being on a proprietary cloud-only vector service. However, I will say that if you are doing something specialized, like multi-vector retrieval or if you need to store multiple different types of embeddings for the same document, something like Qdrant is looking very attractive. It is open-source, it is written in Rust so it is incredibly fast, and it supports what they call "named vectors."
That "named vectors" feature is key for the migration problem. It allows you to have an OpenAI embedding and a BGE embedding side-by-side in the same record. This is vital for what we call the "lazy migration" strategy. Herman, let's talk about that for a second, because it is a great practical takeaway for anyone listening who is already in the trap.
This is a strategy for when you have a billion vectors and you cannot afford to re-embed them all today. You do not have to stay on the old model forever. You implement a dual-embedding strategy. When a user submits a query, you embed it with the new, better model. You then search your "new" index. If you do not find a high-confidence match, you fall back to embedding the query with the old model and searching the "old" index.
It is like a rolling update for your semantic brain. Over time, as you re-process your data or as new data comes in, you populate the "new" vector field. Eventually, the new index becomes the primary and the old one gets retired. It is messy and it doubles your inference cost for a while, but it prevents that catastrophic downtime the fintech company faced.
It also highlights why choosing a model with high "multilingual" or "instruction-aware" capabilities is so important. One of the models Daniel might want to look at as a "safe" open-source pick is BGE-M-three. The "M-three" stands for Multi-lingual, Multi-functional, and Multi-granularity. It is produced by BAAI, and what makes it special is that it supports dense retrieval, sparse retrieval, and multi-vector retrieval all in one model.
Explain the difference between dense and sparse there, because that is a key part of the accuracy question Daniel raised.
Most people think of embeddings as just "dense" vectors—those long lists of numbers. They are great at finding the "vibe" or the general meaning. If you search for "cold weather gear," a dense vector will find "parkas" and "winter boots" even if those exact words are not there. But dense vectors are notoriously bad at finding specific serial numbers, technical jargon, or unique product codes.
Right, the "keyword" problem. If I search for "X-J-five-thousand," a dense vector might just give me other random electronics because they are "nearby" in semantic space, but they aren't the specific thing I asked for.
That is where sparse retrieval comes in. It is more like the traditional B-M-twenty-five or keyword search we have used for decades. BGE-M-three allows you to do both simultaneously. It generates a dense vector for the meaning and a sparse vector for the keywords. When you combine them, you get the best of both worlds. You get the semantic intelligence of AI plus the literal accuracy of a search engine. If an organization is not sure what their future data looks like, having a model that can handle both is a massive safety net. It effectively replaces the need for a separate keyword search pipeline.
It is basically a hedge against the limitations of pure vector search. I also want to mention Nomic Embed Text version two, which made waves in twenty twenty-five. It is the first embedding model to use a Mixture-of-Experts, or M-o-E, architecture. It is open-source and trained on an enormous dataset of over one point six billion pairs. Because it is an M-o-E model, it is incredibly efficient to run locally. If you are worried about the ongoing cost of calling an API every time you need to embed a query, self-hosting a model like Nomic or BGE is a way to decouple your growth from your vendor's pricing sheet.
The cost asymmetry is real. With a proprietary API like OpenAI or Voyage AI—which, by the way, was acquired by Anthropic late in twenty twenty-four—you are paying every single time you search and every single time you ingest. At small scales, it is pennies. But at enterprise scale, that becomes a significant line item. OpenAI does offer a fifty percent discount if you use their batch API for ingestion, and Voyage AI offers thirty-three percent off via their batch endpoint, which helps. But the marginal cost of a self-hosted model is basically just the electricity and the rack space.
Plus, there is the privacy and sovereignty angle. If you are in a regulated industry, sending all your internal documents to a third-party API just to get a list of numbers back is a hard sell for the legal department. Having a "safe setting" that involves an open-source model running on your own infrastructure simplifies that conversation significantly.
So, if we had to give Daniel a "Decision Matrix" or a set of safe defaults right now in March of twenty twenty-six, where do we land?
I will start with the proprietary side. If you want the absolute easiest path with the most flexibility, I would go with OpenAI text-embedding-three-large. But—and this is the key—set the dimensions to one thousand five hundred thirty-six. Do not use the full three thousand seventy-two unless you have a very specific reason. At fifteen thirty-six, you are getting incredible performance, it is M-R-L-enabled so you can truncate it later if you need to, and it is a standard that almost every tool supports. It still outperforms the older ada-zero-zero-two even when truncated down to two hundred fifty-six dimensions.
I agree with that for the proprietary route. It is the "nobody ever got fired for buying IBM" choice of the AI era. But if you want to avoid vendor lock-in entirely, my safe pick is BGE-M-three. It is incredibly robust, it handles over a hundred languages, and that hybrid dense-sparse capability is a lifesaver when you realize your users are searching for specific part numbers that the dense model is ignoring.
And for the database?
Start with Postgres and P-G-vector. Unless you are planning to hit a billion vectors in the first six months, Postgres is going to be more than enough. It is the "boring" choice that lets you sleep at night. You can always migrate to a specialized engine like Qdrant or Milvus later if you truly outgrow it, but starting there saves you so much architectural complexity.
What about the "Safe Setting" for parameters? We talked about dimensions, but what about things like the similarity metric? We see Cosine similarity, Dot Product, Euclidean distance. Does that matter as much for lock-in?
It matters immensely because if you index your data using Cosine similarity but your query code accidentally uses Euclidean distance, your results will be total garbage. Most modern models, including the ones from OpenAI and Cohere, are trained specifically for Cosine similarity. It is generally the safest default because it measures the angle between vectors rather than their magnitude. This makes it more robust to variations in text length.
That is a good point. If one document is a paragraph and another is a whole page, their "magnitude" in vector space will be very different, but the "angle" of their meaning should be similar. Cosine similarity handles that naturally.
There is one more thing I want to touch on regarding accuracy, which Daniel mentioned. We often look at the MTEB leaderboard to decide which model is "best." But those benchmarks are often based on general web data or Wikipedia. If you are working in a highly specialized domain—like legal discovery, medical research, or deep technical documentation—a model that is ranked number five on the leaderboard might actually outperform the number one model on your specific data.
We saw this in a developer documentation benchmark just last year. An open-source BGE model actually beat the top-tier proprietary models seventy-three percent of the time on specific coding questions while running eight times faster. The lesson there is: do not just trust the leaderboard. If you have the resources, run a small "golden set" test. Take a hundred of your most common queries, manually identify the right answers, and see which model actually finds them.
It is the only way to be sure. And that leads to a broader point about the future. People keep asking if we will eventually have a "universal" embedding model that makes all this migration stuff obsolete. My take is that we are actually going the other way. We are seeing more "instruction-aware" models, like the new Qwen-three-embedding from Alibaba.
These are interesting. You actually give the model a hint about what you are doing, right? Like, "Represent this document for the purpose of medical retrieval" versus "Represent this document for sentiment analysis."
The model actually shifts the vector's position based on the task. This makes the embeddings even more accurate, but it adds another layer of complexity to the lock-in. Now you are not just locked into a model; you are locked into the specific instructions you used during ingestion. It makes the "safe setting" harder to find because the "meaning" of the vector is now context-dependent.
It really reinforces the idea that you should optimize for flexibility rather than just raw performance on day one. If you choose an M-R-L-capable model and store your vectors in a flexible database like Postgres or Qdrant, you are giving yourself an "out" when the next big breakthrough happens.
And that is the most important mindset shift. In the old world of databases, we thought in terms of decades. In the AI world, we have to think in terms of months. Your embedding model is going to be obsolete in eighteen months. That is just the reality. The goal of your architecture shouldn't be to pick the "final" model; it should be to make the inevitable migration as painless as possible.
Build for the migration you know is coming. It is a bit pessimistic, but it is the only way to stay sane in this environment. I think that covers the core of Daniel's question. We have the "Safe Settings" of one thousand five hundred thirty-six dimensions, Matryoshka-enabled models like OpenAI's latest or BGE-M-three for open source, and P-G-vector for the infrastructure.
One final tip for the road: always keep your original text. It sounds obvious, but I have seen teams who thought they could save money by only storing the vectors and discarding the source text after processing. When they realized they needed to re-embed, they had nothing to re-embed from. They had to scrape their own website or re-extract text from thousands of P-D-Fs.
Oh man, that is the ultimate nightmare. "We have the coordinates but we lost the map and the destination." Do not do that. Keep your source data in a cheap cold storage bucket like S-three. It is the cheapest insurance policy you can buy.
Truly. Well, this has been a deep one. I love when Daniel sends prompts that let us get into the weeds like this. There is so much "AI hype" out there, but the real work is in these plumbing details.
It is all plumbing, Herman. Always has been. Before we wrap up, I want to give a shout out to our producer, Hilbert Flumingtop, for keeping the gears turning behind the scenes.
And a huge thanks to Modal for sponsoring the show. They provide the G-P-U credits that power our research and the generation of these episodes. If you are a developer looking for serverless infrastructure that actually scales with your AI workloads, check them out.
This has been My Weird Prompts. If you found this dive into embedding lock-in useful, we have a whole archive of similar deep dives at myweirdprompts dot com. You can search for keywords like "Postgres," "R-A-G," or "vector" to find related episodes. We actually did a whole episode on the evolution of AI memory back in episode eight forty-six that pairs really well with this one.
You can also find us on Spotify, Apple Podcasts, or wherever you get your audio fix. And if you have a second, leaving a review on your platform of choice really does help more people find the show.
We will be back next time with another prompt from Daniel. Until then, keep your vectors normalized and your dimensions nested.
Goodbye everyone.
See ya.
