Daniel's been thinking about AI workspaces — not the kind with vector databases and fancy retrieval pipelines, but the kind that's just folders and Markdown files. He uses a Git repo as a container, with folders for context, memory, prompts, project management. The agent reads and writes to those files, and over time the context compounds. Sessions get better. The critique he's raising is about what happens when you hand that pattern over to a platform. ChatGPT and Gemini have built their own versions of this, but the memory lives in their walled garden. You can't export it. You can't point a different agent at it. You're paying monthly to access your own project memory. And the weird gap he's spotted is this: there are plenty of agent-building platforms that let you hook in components, but you have to bring your own frontend. At that point, why use Markdown files at all when you're already building from scratch? So his question is really two things. First, why doesn't a simple Markdown-based agentic backend and frontend exist? And second, if someone wanted to build it themselves — with that two-tier memory hierarchy Claude uses, shared persistent memory across projects plus project-specific context — how would you do it?
The two-tier thing is the key to the whole question, actually. Let me start with the pattern itself, because what Daniel's describing is a workflow that a lot of serious AI users have converged on independently. You've got a repository — just a folder with Git — and inside it you have subfolders. Context might contain background documents, project specs, architecture decisions. Memory is where the agent writes what it learns session to session. Prompts stores prompt templates or conversation history. Project management might be a task list or a kanban board in Markdown. The agent reads from these before each response and writes back to them after. That's it. That's the whole thing.
And the appeal is that it's just text. You can open any file in any editor. You can diff it. You can roll it back with Git. There's no database to administer, no embedding pipeline to maintain. The agent's memory is a Markdown file you can read on your phone.
Right, and that's not a small thing. I've seen teams build elaborate RAG systems and then six months in, someone asks what's actually in the vector store and nobody knows. You can query it, you get results back, but you can't just... browse it. You can't grep it. With Markdown files, the memory is inspectable by design. That's the philosophical core of this pattern — the files are the source of truth, not some opaque embedding space.
Which is why Daniel's frustration with the walled gardens lands. You spend months building up context in ChatGPT, teaching it your preferences, your project structure, your conventions. And then you want to use Claude for a session and none of that comes with you. You're starting over.
The pre-IMAP era of email. That's exactly what this is. Before IMAP, your email lived on the server of whatever provider you used. If you switched providers, you lost everything. You couldn't take your folders with you. We're in that same moment with AI memory right now. Every platform has its own proprietary memory implementation, and none of them talk to each other.
So that's the pattern and the problem. But here's where the market gap gets interesting. Daniel's observation is that the agent-building platforms — LangChain, CrewAI, AutoGPT, the whole ecosystem — they all assume you want a vector database. They've got integrations for Pinecone and Weaviate and Chroma. But they don't ship with a frontend. You're expected to build your own UI. And once you're building a custom frontend anyway, the argument for keeping things simple with Markdown files starts to feel... quaint. Why not just use Pinecone?
Because Pinecone costs money and adds operational complexity and ties you to a specific vendor and requires you to manage embedding models and re-indexing pipelines and all of that for what? For a few thousand tokens of project memory? The BM25 algorithm — which is just a fancy bag-of-words search, no embeddings, no GPU — performs comparably to dense retrieval for moderate-sized document collections. You can run it on a laptop. For tens of thousands of tokens of Markdown files, BM25 or a tiny local embedding model like all-MiniLM-L6-v2 is more than enough. You don't need a vector database until you're dealing with millions of documents.
So the market has over-engineered the memory problem. Everyone reached for the most sophisticated solution before checking whether the simple one works.
And I think there's a reason for that, which is that the agent-building platforms are trying to be general-purpose. They want to support the enterprise customer who does have millions of documents. So they default to vector databases because that's the safe architectural choice for the high end. But it means the solo developer or the small team that just wants a folder of Markdown files and a chat interface... there's nothing for them. The platforms overshoot.
There's a project called Rowboat Labs that's trying to do something in this space. Local-first AI coworker, uses Markdown files for memory. But it's early. It doesn't fully solve the shared backend problem Daniel's asking about — the two-tier thing where you've got persistent memory that spans projects and project-specific context that lives in the repo.
And that two-tier architecture is good engineering. Claude implements it with a vectorized backend, but the logical separation is the same. You've got shared preferences — "I prefer this Python library," "I use this naming convention," "here's how I think about error handling" — that apply across every project. And then you've got project-specific context that lives inside the workspace. Claude reads both. The shared stuff persists. The project stuff is scoped.
Anthropic even has this CLAUDE.md convention now, where you put project-specific instructions in a Markdown file and Claude reads it automatically. It's the closest existing implementation to what Daniel's describing. But it's still inside the walled garden. You can't take your CLAUDE.md files and point a different model at them.
Well, you can — they're Markdown files. That's the thing. The format is portable. But the agent that knows to read them and write back to them, and the frontend that shows you what's in them while you chat... that's the missing piece.
So let's answer Daniel's first question. Why doesn't this exist? I think there are three reasons. One, the agent platforms overbuilt for the enterprise case. Two, the companies that did solve the UX problem — ChatGPT, Claude, Gemini — had every incentive to keep the memory proprietary because it's the moat. Your project memory is what makes switching costly.
And three, the people who want this are exactly the kind of people who could build it themselves, so they do. They hack together a script that reads Markdown files and pipes them into the API call. They use a CLI. It works for them. They don't productize it because they've already solved their own problem. The market gap isn't that the solution is hard — it's that the people who feel the pain most acutely are also the people most capable of building a bespoke workaround.
Which brings us to the second question. If you wanted to build this properly — not a hack, but a real system with a frontend and shared persistent memory and project-specific workspaces — how would you do it?
Let's start with the file structure, because that's the foundation. You'd have a shared memory directory somewhere — could be a local folder, could be on a file server, could be synced with something like Syncthing if you want it available across machines. Inside that, you'd have Markdown files for persistent preferences. "I use pytest for testing." "I prefer type hints everywhere." "Here are the libraries I reach for by default." That kind of thing. Then each project gets its own workspace folder with the standard scaffold Daniel described — context, memory, prompts, project management.
And the agent reads from both tiers. Shared memory first, then project-specific context. Writes updates back to the appropriate tier depending on whether the new information is project-scoped or globally relevant.
The read path is straightforward. Before each response, the agent — or the harness around the agent — pulls in the relevant context. You could do this with a simple keyword search over the Markdown files, or with a local embedding model if you want semantic search. The embedding model runs entirely on your machine, no API calls, no vector database. You embed the user's query, compare it against embeddings of all your Markdown files, pull the top few results, and include them in the context window.
And the write path? The agent generates new memories and the system has to decide where to put them.
This is the part that gets interesting. You don't want the agent just appending to a file endlessly — you end up with a giant log of session notes that nobody ever reads. You want consolidation. After each session, or periodically, the agent should review the memory files and consolidate. Merge related points. Remove contradictions. Update outdated information. This is basically what Claude and ChatGPT are doing under the hood with their vectorized memory, but you can do it with plain text and a decent prompt.
So the agent reads the existing memory, reads the session transcript, and produces an updated version of the memory file. That's a single LLM call. It's not real-time, but for most projects you don't need real-time memory consolidation.
And because the files are in Git, every consolidation is a commit. You've got a full history of how your project memory evolved. If the agent makes a bad consolidation, you roll it back. Try doing that with a vector database.
Now the frontend. This is where Daniel's frustration lives. The CLI works but it's workstation-bounded and, let's be honest, it's not a great experience for reviewing memory files side by side with a conversation.
What you'd want is something like a local web app. A FastAPI backend that serves the Markdown files and handles the chat API calls. A React frontend with a split view — chat on one side, file browser on the other. You can see the memory files update in real time as the agent writes to them. You can click into a file and edit it directly. The files are the source of truth, and the UI just reflects them.
And the whole thing runs locally. No cloud dependency. Your project memory never leaves your machine unless you choose to sync it.
That's the local-first principle. The database is the file system. The API is just a thin layer that reads and writes files. If the frontend breaks, you've still got your Markdown files. If you want to switch to a different frontend, the files don't care. They're just text.
There's a question about sharing, though. Daniel mentioned wanting a common backend that multiple agents or projects can access. If you've got concurrent writes to the same memory files, you need some kind of conflict resolution.
Git is the obvious answer for async sharing. Each write is a commit. If there's a conflict, you resolve it like any other merge conflict. It's not real-time collaborative editing, but for agent memory, you probably don't want real-time collaborative editing. You want each agent or each session to have a coherent view of the memory and to merge changes deliberately.
For something closer to real-time, you could use a CRDT-based approach. Conflict-free replicated data types. There are libraries that handle this for plain text. But honestly, for the use case Daniel's describing — a solo developer or a small team with a handful of active projects — Git is probably sufficient.
And the beautiful thing about Git is that it gives you an audit trail for free. You can see exactly what the agent changed and when. If the agent starts hallucinating memories — and they do, occasionally — you can spot it in the diff and revert.
Let's talk about the embedding model piece, because I think there's a misconception that semantic search requires a vector database. You mentioned all-MiniLM-L6-v2. What does that actually look like in practice?
It's a tiny model — about eighty megabytes. You download it once, it runs on CPU, no GPU required. You feed it a chunk of text, it spits out a vector of three hundred eighty-four numbers. You do that for all your Markdown files once — or whenever they change — and store the vectors in a simple file, maybe a SQLite database if you want to be fancy. When the user asks a question, you embed the question, compute cosine similarity against all the stored vectors, and pull the top K results. The whole thing runs in milliseconds for a few thousand documents.
BM25 is even simpler. No model at all. Just tokenize the query, count term frequencies across your documents, rank by relevance. It's been around since the nineties and it still works remarkably well for text retrieval.
The point is, you don't need Pinecone. You don't need a GPU. You don't need to pay anyone a monthly fee. The tools for building a perfectly good memory system out of Markdown files are all free and open source and run on a laptop.
The architecture Daniel's asking about is entirely buildable. A shared memory folder with persistent preferences. Project workspaces with context and memory subfolders. A thin backend that reads and writes Markdown files and handles retrieval with BM25 or a local embedding model. A simple web frontend with chat and a file browser. Git for versioning and sync. The whole thing is maybe a few thousand lines of code.
The reason it doesn't exist as a product is not that it's technically hard. It's that the incentives are wrong. The companies with the resources to build a polished frontend — OpenAI, Anthropic, Google — want your memory inside their platform because it reduces churn. And the open-source community has gravitated toward the more architecturally interesting problems — vector databases, agent orchestration, multi-agent systems — rather than the unglamorous work of building a good UI for a folder of text files.
There's also a weird dynamic where "just use Markdown files" sounds too simple to be taken seriously. If you pitch it to an engineer, they'll say "but what about scale?" and you have to explain that for ninety-five percent of use cases, scale isn't the bottleneck. The bottleneck is complexity.
I've said this before and I'll say it again: start with Markdown. Only add a vector store if cross-project retrieval becomes a bottleneck. Most people never reach that point. They install Pinecone on day one because a tutorial told them to, and now they're managing infrastructure for a problem they don't actually have.
The other thing Daniel's prompt gets at, which I think is underappreciated, is that the act of writing things down in human-readable form forces a kind of discipline. When your memory is a vector embedding, you can be sloppy. The retrieval will probably find something relevant anyway. When your memory is a Markdown file that you might open and read, you write differently. You organize. You consolidate. The format shapes the thinking.
That's the "second brain" idea, applied to agents. People build these elaborate personal knowledge management systems in Obsidian or Notion or whatever, and the value isn't just in having the information — it's in the process of structuring it. The same thing applies to agent memory. A well-maintained folder of Markdown files is a thinking tool, not just a retrieval store.
If someone listening wants to build this — and I suspect Daniel is asking because he's considering building it — where do you start?
I'd start with the file structure and a simple script. Don't build the frontend first. Build the thing that reads from shared memory and project memory, concatenates the relevant context, and sends it to the model. Get the read path working. Then add the write path — after each session, the agent produces an updated memory file. Get that working from the command line. Once the core loop works, then think about a frontend.
For the frontend, I'd look at something like Streamlit or Gradio for a quick prototype. You can have a chat interface and a file viewer up in an afternoon. It won't be polished, but it'll let you test the workflow end to end before you invest in a real UI.
If you want to go further, a FastAPI backend with a React frontend is the standard play. The backend serves files from disk and proxies chat requests to the LLM API. The frontend shows the conversation and the file tree side by side. You could build a minimum viable version in a weekend if you know the stack.
The sharing piece — the common backend across projects — is mostly a filesystem problem. If you're on a single machine, it's just a folder. If you want it available on multiple machines, Syncthing or a simple file server does the job. If you want multiple agents writing concurrently, Git with periodic commits and merges handles the async case. None of this requires a database.
If you really want real-time collaboration, you could put the Markdown files in something like ShareDB or use a CRDT library. But I'd argue that's over-engineering for the use case. Agents don't need real-time collaborative editing. They need a coherent view of the world and a way to merge changes. Git gives you that.
Let me throw a question back at you, actually. You've been thinking about agent memory architectures for a while. Is there a scenario where the Markdown file approach actually breaks down? Where you need a vector database?
Two scenarios. One, when you have so many documents that brute-force embedding comparison becomes too slow — we're talking hundreds of thousands of files. At that point, you need approximate nearest neighbor search, and that's what vector databases are optimized for. Two, when you need real-time, low-latency retrieval during every single agent step, not just at the start of a session. If your agent is making twenty tool calls and each one needs to pull relevant context from a massive memory store, you need something faster than reading files from disk.
But for the pattern Daniel's describing — a project workspace with a few dozen Markdown files, maybe a few hundred, and retrieval happens at session boundaries — neither of those applies.
Correct. The simpler architecture is the better one for that use case. The industry has convinced itself that vector databases are table stakes for agent memory, and they're just not. They're a specialized tool for a specific scale of problem, and most projects never reach that scale.
There's one more piece of this I want to pull on. Daniel mentioned that the CLI approach creates a workstation-bounded workflow, and his frustration with mobile terminal emulators is... relatable. But I think the deeper issue is that a CLI doesn't give you visibility into the memory files. You're talking to the agent, and it's reading and writing context, but you can't see what's in those files unless you open them separately.
That's the frontend gap. The ideal interface shows you the memory alongside the conversation. You can see that the agent just pulled in three context files. You can see that it's about to write an updated memory. You can intervene — edit a file, delete a bad memory, add a note. The conversation and the context are peers, not separate worlds.
That's what the walled gardens get right, actually. ChatGPT's interface is clean. You don't see the memory files, but the experience is seamless. The problem is that the memory files aren't yours. If someone built a frontend that was just as clean but backed by a local folder of Markdown files, that would be the thing Daniel's asking for.
Hilbert: Nineteen ninety-seven. I was a technical writer at a company called Meridian Systems. They made document management software — the kind of thing law firms used before SharePoint existed. We had this custom CMS with a proprietary database. All the documentation, all the knowledge base articles, everything was in this database. And then Meridian got acquired by a larger company that had its own CMS. They wanted to migrate everything. But the database format was proprietary. There was no export tool. So they paid me — and three other writers — to spend six months manually copying content out of the old system and into Markdown files. Six months. Copy, paste, format. Copy, paste, format. I still have those files. They're on a hard drive in my closet. Every piece of documentation Meridian ever wrote, now in plain text.
When Daniel talks about vendor lock-in for project memory, you've lived a version of this.
Hilbert: I've lived the worst-case version. And here's the thing everyone in this conversation is missing. You're all talking about agents and vector databases and frontends and whether BM25 is good enough. But the real genius of Daniel's pattern isn't the retrieval speed or the architecture. It's that it forces you to write things down in a way that a human can read. I've seen teams build these elaborate RAG pipelines and then realize they have no idea what's actually in their vector store. With Markdown files, you can open the folder and read the memory. That's not a limitation. That's the feature.
The inspectability. You can't grep a vector embedding.
Hilbert: You can't grep it, you can't diff it, you can't hand it to the new person on the team and say "here's what we know about this project." It's just numbers. Useful numbers, I'm sure. But numbers.
Do you still use those Markdown files from Meridian?
Hilbert: I do, actually. They're my personal knowledge base. Twenty-nine years of documentation habits, all in one folder. I've never hooked them up to an AI agent. I just read them. But I've been waiting for someone to build the thing you two are describing so I don't have to.
A frontend that treats your folder of Markdown files as the memory backend and gives you a chat interface on top.
Hilbert: That's the one. I'd pay for it. I'd pay for it once, not monthly. It's my files. They live on my machine. The software is just a window into them.
The gap in the market, sitting right there.
Hilbert: It's been sitting there for years. Someone will build it eventually. Probably someone who got burned by a proprietary database and has strong feelings about plain text.
That's actually a perfect place to land. The gap is real. The pieces are all there — local embedding models, BM25, Git, Markdown, FastAPI, React. The architecture is straightforward. The reason it doesn't exist as a product isn't technical — it's that the incentives point toward complexity and lock-in, not simplicity and portability.
The thing Hilbert's pointing at is the part that gets lost in architecture discussions. The value of being able to open a file and read what your agent knows about you. That's not a nice-to-have. That's the whole point of building a memory system in the first place.
The best architecture for memory might not be the most sophisticated one. Sometimes the right answer is a folder of text files and a good editor. Daniel's pattern is simple, portable, and human-readable. The market just hasn't caught up to it yet.
If you've built something like this, or if you're thinking about it, we'd love to hear from you. Send us a note at show at my weird prompts dot com. Or just write it down in a Markdown file. We'll find it eventually.
This has been My Weird Prompts. Thanks to our producer Hilbert Flumingtop. We'll be back soon.
