#What is Basic Memory?

This is a question I get a lot. Honestly, I don’t know how to really answer it. It’s a lot of things. It has a silly name. It has simple features, like reading and writing notes, so it is basic, but it really does do some powerful things. It also has “memory” in the name, so it obviously is supposed to help you remember things. It lets AI agents write notes to text files in Markdown. It indexes them for searching, then syncs them again when a user modifies them. That sounds basic, for sure.

It also does some other things, which, even though they are simple, can actually get pretty complicated. It parses and traverses all of the notes to create a semantic knowledge graph that can be queried and traversed recursively. What does that even mean? Why should you even care? The answer, maybe you don’t care, and certainly you shouldn’t need to really understand “how” it works under the covers. It’s a tool, and the most important thing about a tool is that it should “just work”. When something works well, it gets out of your way and enables you to do what you set out to do more easily. Really that’s what Basic Memory aims to be, a tool that works well enough to help organize your long term memory, with AI.

But why do I even need this tool? How is it useful?

This is the backstory, so feel free to skip to the end if you’d rather.

To answer this, I’ll start at the beginning (of me chatting with AI LLMs (Large Language Models)). When I first started chatting (using ChatGPT), It seemed like the conversation could go on forever. It was really amazing, I could just talk to this “thing”, and it would answer back. Sometimes garbage, but other times with really useful or helpful information. I started trying to talk to it about a lot of things, ideas I had, questions. Then, because I’m a software developer, programming. I was shocked, truly, to see that it could write a lot of code pretty well. Not everything, but then again, neither can I, and I’ve been a professional developer for a long while (decades). As I went deeper and deeper into “knowledge activities” with it, learning new code, using it to help me write React code, then eventually, just letting it write the code entirely, I began to hit some walls. Things would be going great, and the chat would hit a limit. No more messages in the conversation. Crap. So, I did what anyone might do, I copy pasted a whole bunch of chat info to a local text file, then started a new chat and pasted it back in. Back in business. But that got old really quickly. Soon, I had a huge mess of notes, and a bunch of chats with a lot of repeated information. What I wanted was one place to store all of it so I could manage it more easily.

At this point, I stumbled across Claude. It was even better at coding than ChatGPT, and it had this funky “personality” to it. Using it was actually fun. And, it had this cool feature, called Projects, where you could upload files and the chat could already see the info, without me having to copy/paste (Claude Projects allows the LLM to load info into its context window before a chat starts). That was a huge improvement, but, I still had to copy and paste as things got out of date, or maybe I’d forget something was in the Project info, but incorrect, or not relevant, and oh, where is that in the Project? It was hard to search. So, I decided to try and build my own version of Projects, and maybe figure out how to use the api to read it. I really didn’t know. But, I had already been using AI assistants to make all sorts of code projects I would have not been able to do otherwise (that’s another story). Now, I didn’t know how or what I was going to do, but I had a problem, and one I could see lots of other people were going to have when they used LLMs for chatting, how do I organize all the knowledge that is getting produced? And how do I keep it, instead of it being up there where I can’t work with it easily.

Then, late last year, Anthropic released the MCP (Model Context Protocol) spec. Like I said, I’m a programmer, so I’m fairly used to reading specification docs. But I had to read this one about three times before I understood (most of) it. And, there were all these sample apps they released that did really cool things! Tools to read and write local files (filesystem mcp), one that stored a semantic knowledge graph in local json files (memory-json mcp). They were all open source, I could just grab the code and run them. Right away, I could see this is what I needed. I’ve been a long time Obsidian fan, and have used Markdown for my daily notes for personal and work information. The idea came really naturally then, what if I could keep memories, like the memory-json, but in Markdown? Obsidian uses markdown to let people create a “second brain” of linked knowledge, what about a Zettlekasten for AI? I thought I was a genius. It would be so simple. I’ll bet I can do it in a week.

Well, a few months later, and I did have something that worked well enough to share with some friends. I was as amazed as anyone, maybe more, when it worked as I’d envisioned. Mind you, the features were limited and there were plenty of bugs. But, Basic Memory would let the AI write Markdown files, and I could see them immediately right inside Obsidian. Then, I could update the file and the AI could see my updates.

Aside: Building Basic Memory

Perhaps one of the more interesting parts about all of this is that I used AI extensively to build Basic Memory from the very beginning. It’s a truly native AI software project. Not a vibe coded project, a real software project, like every other one I’ve built professionally, with clean code, tests, architecture docs, etc. And, we (the AI and myself) used Basic Memory together keep track of what we were building as we built it. We dogfooded it from the beginning, a software term meaning “eating our own dogfood”, or using the thing you are trying to build as you build it. So, Claude and myself were, in fact the first users. There was no great plan, it changed as we went and we just kept building. At one point, I refactored the code (meaning changed the implementation), and removed half of it, after we discovered a better way to implement a feature.

The other interesting part of this anecdote is that Claude loved the idea. I know now, it’s fashionable to say that AIs are sycophantic and they love everything, but trust me, we had a lot of conversations where the AI shot my bad ideas down. What happened here is kind of special, at least it was for me. Because the AI (Claude) was enthusiastic about the idea of Basic Memory, and I told it we were very much collaborating, and I needed it’s best and most honest opinion, it gave them to me. It helped me connect the dots between different concepts that I wouldn’t have done alone otherwise. For instance, in any complex software project with a lot of data, there needs to be one thing that is the “source of truth”, otherwise, if for instance a file says one thing, but the version indexed for the search in the database says another, what do you do? We came up with a bunch of rules, the filesystem is always the source of truth, versioning should be simple, like git, etc. We wrote all of these down in Basic Memory, then we could reference them again. And these notes were very detailed. One of the instructions I gave was to “use prose” to explain, not lists. And what I got out was a lot of really detailed design notes with extensive depth about how the system was related and interconnected. And, the notes themselves were connected!

Then, I decided I had to just get something out, meaning release some version so other people could use it and I could hopefully get feedback. Of course, as a software developer, I know that releasing is actually the beginning of the pain and hard work that comes with a software project. Next, when people start actually using it, they find bugs and you have to fix them, and keep trying to build more features. Luckily, lots of people have found Basic Memory and started using it. I hear from new people all the time, and that is really wonderful. People file bug issues on github and make enhancement requests asking for more features.

So what did we end up building?

Now, why is Basic Memory useful, and what are these magical features?

Like I said, the idea is to keep things simple. But also, to make a set of really simple tools that work together and contribute to create something that is very powerful.

At its core, Basic Memory does three simple things that work together to create something powerful:

1. Stores knowledge in plain Markdown files - Everything lives locally on your computer, not in some cloud database.
2. Creates connections automatically - Every connection becomes searchable and traversable
3. Enables two-way collaboration - True two-way collaboration in plain text between AI and humans

Basic Memory has a few common tools: read_note, write_note, edit_note, search_note, etc. These let the AI write information in Markdown to files on your local computer filesystem.

The user can see these notes and read them, modify them, move them, combine, or delete them. All of this is synced back into the system so the AI can see your changes. All of the notes get indexed into a local search index, so you can query them easily. Also, the notes have some special formatting that the code parses and indexes into a graph (a set of connected objects, like a tree) that Basic Memory can query recursively (one result, leads to more results, and so on).

The Markdown system has a few special, again simple, conventions to enable this:

• Each note is an “Entity”, or a thing, that can contain a bunch of text.
• Entities can contain “Observations”, or facts about that thing. These are simple markdown lists, like - [category] Some fact.
• Entities can be related to each other via links. Basic Memory uses the [[wikilink]] convention to relate notes (Entities). These can also contain extra information to describe the type of relationship and context.

The tools the AI uses all have special instructions, so it knows all about these things and can write notes with the special formatting so all the things are connected. Now, using these simple things, a user can ask an AI assistant in a chat “What were we discussing about <Some Project> last month?”, and the AI can use the tools to search the data in Basic Memory, then find other things related to results, and know how they are related semantically (how a thing is understood), because the Markdown documents contain the information. All of this is just plain text (Markdown), and is easily read by humans as well as AIs. It turns out, also, that this is very convenient for a couple of reasons.

First, the “context window” or size of information within a conversation with an AI is limited. They are very large, and continue to grow but still, you can’t fit everything in there all at once. So, it’s helpful to be able to find relevant information easily and load it and it’s related information into the context window (new chat) easily. This is how Basic Memory enables “persistent, interconnected knowledge”.

Second, it enables a “common interface” between AIs and humans that they both understand really well, plain text!

And, because Basic Memory files are easily editable and sync automatically back into the system, there is a real two way information exchange enabled for long term, deep information between humans and AIs. This is what I mean when I say that your knowledge “grows with you”. You aren’t limited to what the AI decides to save for important “memories” in its own data store. And you aren’t confined by how much data you can store, it’s just files.

Now, then, what we have is a system, made of relatively simple parts, that is actually quite sophisticated. Basic Memory creates a set of tools and a process where you have a lot of potential to build, collect and manage rich knowledge together with an AI. It doesn’t force you into some sort of thing that is difficult to use. And, it doesn’t try to hide information or format it in a way that is hard to understand. There is no black box. You can see everything, and you can control everything.

There is a lot more work left to do, and I am really looking forward to adding even more features to Basic Memory. I really hope that, with this latest release people will continue to find Basic Memory useful, and it gets out of their way and is just helpful. All feedback is appreciated, or maybe considering telling a friend.

Thanks for reading,

-Paul