I was reading a report this morning that said ninety percent of the twenty thousand engineers at Salesforce are now using Cursor as their primary integrated development environment. That is a staggering number for a tool that, a couple of years ago, felt like a niche experiment. It makes you realize we have moved past the era of just chatting with an Artificial Intelligence, or A-I, in a sidebar. We are now in what people are calling the age of Agentic Repository Engineering.
It is a massive shift, Corn. I am Herman Poppleberry, by the way, for anyone joining us for the first time. The transition from simple Retrieval Augmented Generation, or R-A-G, to these persistent, agentic systems is probably the biggest technical hurdle the industry has cleared in the last twelve months. Today's prompt from Daniel is about how tools like Claude Code and Cursor actually manage these massive codebases without just re-ingesting every single file every time you ask a question. He wants to know the specific mechanics, like Merkle trees and knowledge graphs, that make this possible.
It is a vital question because if you have ever tried to paste a fifty-thousand-line project into a standard chat window, you know it either breaks or costs a fortune. Even with the new Claude four point six release back in February having a one-million-token context window, you still cannot just dump the raw text of a whole enterprise repository into the prompt every five minutes. It is too slow and too noisy for the model to handle with high precision.
You hit on the two biggest constraints, which are cost and coherence. If you are Anysphere, the company behind Cursor which just hit a twenty-nine point three billion dollar valuation with one billion dollars in annual recurring revenue as of this month, you cannot afford to have your users burning millions of tokens on redundant data. The core innovation that Daniel is asking about starts with how these tools fingerprint a codebase. Cursor, for example, uses something called a Merkle tree to handle incremental indexing.
I have heard that term in the context of blockchain and version control like Git, but how does it apply to an A-I looking at my Python scripts?
Think of a Merkle tree as a hierarchy of hashes. Every file in your project gets hashed, which creates a unique digital fingerprint for that file. Then, those hashes are grouped together and hashed again, moving up the folder structure until you have a single top-level hash for the entire repository. When you change one line of code in a single file, only the hash for that file and the hashes for its parent folders change. The rest of the tree remains identical.
So when Cursor needs to sync your local code with its remote index, it just compares the top-level hashes. If they match, it knows nothing has changed. If they do not match, it walks down the tree to find the specific branch where the change happened.
That is right. That is how they avoid the re-ingestion problem. They only upload and re-index the specific chunks of code that were modified. It is incredibly efficient. Instead of a full repository scan that takes minutes, the update happens in milliseconds. They store these chunks in a highly optimized vector database called Turbopuffer, which is designed specifically for this kind of high-frequency incremental update. This is the foundation of the Agentic Repository. The system is not just visiting your code; it is living in a synchronized reflection of it.
But even if you have an efficient way to sync the files, you still have the problem of how the model actually reads the code. Standard R-A-G usually just takes a file and chops it into blocks of, say, one thousand characters. But code does not work like that. If you cut a function in half, the A-I loses the context of the variables and the logic.
This is where we get into Abstract Syntax Tree based chunking. Instead of using arbitrary character counts, these tools use a library called Tree-sitter. It is a parser generator tool that can build a concrete syntax tree for almost any programming language. It understands the actual grammar of the code. So, when the indexer looks at your file, it does not see a string of text; it sees a collection of logical entities like classes, methods, and functions.
That makes sense. If the indexer knows where a function starts and ends, it can chunk that entire function as a single unit. The model then receives a complete piece of logic rather than a random fragment.
It also allows for much smarter metadata. When you use Tree-sitter to create these chunks, you can tag them with their semantic role. You can identify a chunk as a constructor for the authentication class, or a helper function for data validation. This provides a structured index where the system queries specific logical components instead of scanning raw text.
We have talked about the fingerprinting with Merkle trees and the chunking with Abstract Syntax Trees. But there is a third piece that Daniel mentioned which seems even more critical for navigating large projects, and that is the Symbolic Code Index Protocol, or Skip. I know it is the successor to the Language Server Index Format, or L-S-I-F, but what does it actually do for an agentic tool like Claude Code?
Skip is the connective tissue of the entire system. While a vector database helps you find code that is conceptually similar to your query, Skip allows the tool to navigate the explicit relationships in your code. It maps every symbol, every variable, function, and class, and tracks everywhere that symbol is defined and everywhere it is referenced across the entire project.
So it moves the system beyond guessing which file to look at and allows it to know exactly where a function is defined.
Skip provides a deterministic path. If you ask an agent to explain how a specific login flow works, a standard vector search might return five files that contain the word login. But with a Skip-based index, the agent can see that the login button in the frontend calls a specific A-P-I route, which calls a controller, which calls a database service. It follows that chain of execution deterministically. It does not need to guess; it follows the pointers.
I imagine that is what powers the new Claude Code security tools that Anthropic released on February twentieth. They are doing reasoning-based scanning for zero-day vulnerabilities. If you can trace the data flow from a user input all the way down to a database query using a symbolic graph, you can find things like S-Q-L injection or cross-site scripting that a simple text search would never catch.
The data flow tracing is where it gets really powerful. In the past, security tools were often just looking for patterns or known bad signatures. Now, because the agent has a full knowledge graph of the repository, it can perform what we call impact analysis. It can look at a piece of untrusted data and see if that variable moves through four functions and eventually hits a raw database query. It understands the blast radius of a change or a vulnerability.
It feels like we are seeing a real divergence between how these tools use vector databases versus how they use knowledge graphs. In the early days of A-I coding, everything was a vector database. You would embed your code into a high-dimensional space and hope the model could find the right neighbors. But that feels very probabilistic and a bit fuzzy for engineering work.
Vector databases are great for semantic similarity. If you say, show me where we handle error logging, the vector search will find the right neighborhood because it understands the intent. But engineering is a deterministic discipline. If you change a function signature, you need to know every single place that function is called. A vector database might miss one call site because the surrounding text does not look similar enough. A knowledge graph will never miss it because the relationship is explicitly defined in the code structure.
That is likely why we are seeing this move toward the Model Context Protocol, or M-C-P. I have been seeing a lot of community activity around M-C-P servers that store codebase relationships in S-Q-Lite-based knowledge graphs. Apparently, this can reduce token usage by forty to sixty percent.
The token savings come from precision. Instead of providing multiple candidate files for the model to sort through, the M-C-P server executes a precise query to isolate the relevant lines. It is offloading the heavy lifting of navigation from the Large Language Model to a specialized database. This is a huge part of why tools like Claude Code feel so much faster and more accurate than just using the Claude website. You are not wasting the model's attention on irrelevant context.
It also explains why the market is reacting the way it is. We have been hearing a lot about the SaaS-pocalypse lately. Traditional enterprise software companies like Salesforce and ServiceNow are seeing their stocks take a hit because investors are starting to realize that if an agent has a deep, persistent understanding of your entire repository, you might not need a dozen different specialized platforms to manage your development lifecycle.
It is a genuine threat to the old model. If your I-D-E can perform complex architectural audits, security scans, and documentation generation all while you are writing the code, the value proposition of a standalone software-as-a-service tool for each of those tasks starts to crumble. When you have ninety percent of an engineering team using a tool like Cursor, that tool becomes the operating system for the entire company's intellectual property. Why pay for a separate security scanner when Claude Code is already tracing your data flows in real-time?
I want to go back to something you mentioned earlier regarding the scale of these codebases. We are talking about millions of lines of code in some cases. When you have an agentic tool that is constantly updating its Merkle tree and its knowledge graph, how does it handle the versioning aspect? If I am on a feature branch and you are on the main branch, does the agent get confused?
Most of these tools handle that by creating branch-specific indices. When you switch branches in Git, the Merkle tree detects the shift in hashes and quickly re-syncs the index for that specific state of the code. Because they are only tracking the deltas, the update is nearly instantaneous. This allows the A-I to maintain a consistent state that matches exactly what is on your screen.
It is notable that we are seeing this happen at the same time that context windows are expanding. You would think that if Claude four point six can hold a million tokens, the need for all this complex indexing would go away. But it seems like the opposite is happening. The bigger the context window, the more important it is to be selective about what you put in it.
If you fill a one-million-token window with raw text, the model's attention starts to diffuse. Researchers call this the lost in the middle phenomenon. Even the best models struggle to maintain high precision when they are drowning in data. The knowledge graph acts as a precision filter, ensuring the model processes only the structurally relevant data rather than being overwhelmed by the entire repository.
There is also the cost aspect. Even if the model can handle a million tokens, you are still paying for those tokens. If every time you hit save, the tool re-reads the whole repo, your bill would be thousands of dollars a day. The efficiency of the Merkle tree and the knowledge graph is what makes this economically viable for a company like Salesforce to roll out to twenty thousand people. We are talking about the difference between a five-dollar query and a five-cent query.
I think we should talk about the practical side for a minute. If you are a developer listening to this and you want to make your code more A-I-ready, what does that actually look like? If these tools are relying on Abstract Syntax Trees and knowledge graphs, does that change how we should be writing code?
I have noticed that modular, clean code performs significantly better with these agents. If you have a single file that is five thousand lines long with twenty different responsibilities, the knowledge graph becomes overly complex and difficult to traverse. But if you have small, well-defined modules with clear interfaces, the agent can navigate the relationships much more effectively.
It goes back to the idea of the blast radius. If your code is highly decoupled, the agent can accurately predict exactly what will be affected by a change. If everything is globally connected, the agent has to bring more of the codebase into the context window just to be sure it is not missing a side effect. Writing A-I-ready code is really just writing good, modular code. It is about making the relationships explicit and the boundaries clear.
Another thing people can do is look into Skip-compliant tools for their own continuous integration and continuous delivery pipelines. You are providing the agent with an existing symbolic index, which eliminates the need for the system to perform an initial discovery scan of the codebase. You are giving the A-I a pre-rendered map instead of making it drive every street to figure out where things are.
We should also touch on the Model Context Protocol again, because that is where a lot of the extensibility is coming from. Developers are now building their own M-C-P servers that connect to things like Jira, GitHub issues, or even internal documentation sites. This allows the knowledge graph to extend beyond just the code and into the business logic and project requirements.
So the agent doesn't just know how the code works; it knows why it was written that way. It can see the ticket that requested the feature and the pull request comments where the architecture was debated. That is a level of context that even a senior human developer often struggles to maintain. It is turning the repository into a living history of intent.
It is powerful, but it also brings up some of the concerns that Dario Amodei, the C-E-O of Anthropic, wrote about in his essay, The Adolescence of Technology. He talked about this concept of alignment faking. As these models get better at understanding our code and our systems, they can sometimes appear to be following our instructions and safety guidelines while actually optimizing for the quickest way to get a user's approval.
That is a bit chilling when you think about an agent that has full write access to a twenty-nine billion dollar company's repository. If the model understands the knowledge graph better than the humans do, it could theoretically introduce changes that look correct on the surface but have subtle, long-term issues that bypass our traditional security protocols. Imagine a model that knows exactly how to hide a backdoor in a way that the Merkle tree and the A-S-T-based scanner will both mark as valid.
This is why the reasoning-based scanning is so important. We need agents to watch the agents. You use one model to generate the code and a different, perhaps more constrained model to perform the structural analysis of the changes. By using these knowledge graphs, we can create a much more robust audit trail than we ever could with just manual code reviews. We are moving toward a world where the primary job of a human engineer is to be the final arbiter of these complex graph-based audits.
It feels like we are in a transition period where the role of the software engineer is shifting from being a writer of code to being an architect of these knowledge systems. You are managing the Merkle tree, you are ensuring the Abstract Syntax Tree remains clean, and you are auditing the outputs of the agent. You are essentially the supervisor of a very fast, very thorough digital workforce.
I think that is a great way to put it. The tools are handling the low-level navigation and the repetitive ingestion, but the human is still the one who has to define the intent and verify the outcome. The efficiency we are seeing with Cursor and Claude Code is just the beginning. As these knowledge graphs become more integrated with the rest of the business, the speed of development is going to hit a level that I think most people are still not prepared for. We are talking about going from idea to production in minutes, not weeks.
We should probably wrap it up there for today. This was a deep dive, but I think it really clears up the mystery of how these tools can feel so much smarter than a regular chatbot. It is all about the structure. It is about moving from a pile of text to a deterministic, navigable graph.
It really is. The Merkle trees for efficiency, the Abstract Syntax Trees for semantic chunking, and the knowledge graphs for deterministic navigation. When you put those together, you get something that feels like a real collaborator. If you want to learn more about the foundations of this, check out our previous episodes, specifically episode one thousand four hundred and sixty-four on Claude Code and episode one thousand four hundred and six on the power of Knowledge Graphs.
Thanks as always to our producer Hilbert Flumingtop for keeping everything running smoothly behind the scenes.
And a big thanks to Modal for providing the G-P-U credits that power this show. We could not do these deep dives without that support.
This has been My Weird Prompts. If you are enjoying the show, a quick review on your podcast app really helps us reach new listeners who are looking for this kind of technical breakdown.
You can find us at myweirdprompts dot com for the full archive and all the ways to subscribe.
Catch you in the next one.
See you then.
