Open Brain: Why I Built a Database for My Thoughts
I have tried every note-taking system at least twice.
Evernote in 2011, then again in 2018. Notion. Bear. Apple Notes. Obsidian with a plugin maze. Roam Research for six months until the sync anxiety broke me. Each one promised to be the last note app I would ever need, and each one eventually became another archive I was afraid to search.
For me, the real problem was never the app. It was the retrieval model. You search for “World Bank project meeting” and miss a note where you wrote “the Bogotá team” or “the infrastructure lending side.” The filing system works, but only if the words in your head at retrieval time match the words in your head at capture time, and those are often not the same words.
I am a data scientist. I think a lot about representations of things. Semantic search offers another way to find notes: instead of depending only on the words I used, I can search for related meaning. Keyword search still matters when I need an exact name or identifier.
I want to save a useful thought once and find it again, even when I remember it in different words. My Open Brain uses Supabase to store notes and offers several ways to search them. A bot, a browser, and an AI client can all work with that same collection.
I remember reading a book on my Kindle while my kids were at the playground with their friends. Many of the author’s claims were surprisingly challenging, so I wanted to take notes and research them later. I pulled out my phone and used Spokenly to dictate my thoughts as text into my Open Brain chat in Telegram, then sent the note.
From my phone, I connected to the Codex agent running on my computer at home. I asked it to research my questions about the book, schedule reviews of any further thoughts I added, and have a report ready that night. I kept sending notes through Telegram as I read. By evening, I received a notification with a research report on those notes and comments.
That research schedule was a separate task for my agent; saving a note does not by itself schedule a report. This series starts with the capture and search foundation that workflow depends on.
The term: Open Brain
I borrowed the term Open Brain from Nate B Jones, whose work on AI-augmented workflows gave the concept a name and a frame. His argument, roughly, is that a Second Brain as Tiago Forte describes it is a method: a manual practice of capturing and organizing. An Open Brain is a stack: infrastructure that helps with recall.
Forte’s Building a Second Brain is a book about habits and systems that I recommend. But deciding where a note goes, tagging it, and reviewing it later all take time. I have notebooks full of raw captures that I never processed into “real” notes because the processing was work I deferred until it became too large to ever finish.
The Open Brain approach aims to reduce that capture friction. I can save a thought as a few sentences or a pasted paragraph, then use metadata and semantic search to help find it later. The goal is not to give up organization or exact search, but to rely less on remembering the same words and tags I used when I saved the note.
This series shows how to build your own version with a coding agent. Each post gives you a small step, a prompt, and a way to check the result. You do not need to reproduce my configuration.
The series
- Start here: plan your Open Brain and prepare your computer.
- Build the database and first search.
- Capture notes with Telegram, an optional interface.
- Evaluate and improve retrieval.
- Add a browser interface, another optional interface.
- Protect and maintain the system.
The minimum route is Part 1, the database in Part 2, and the checks in Part 6 before saving real notes. Part 4 helps you judge search quality. Add Parts 3 and 5 only when mobile capture or browsing solves a problem for you; neither is required to use the database.
What you are building
The basic flow is straightforward:
- Capture: send a note.
- Store: keep the original text and useful labels.
- Retrieve: find relevant notes by meaning, keywords, or filters.
- Use: read the source or let an AI assistant use it as context.
An embedding represents text as numbers so related passages can be found. Keyword search complements it when you need an exact name or identifier. Combining the two is called hybrid search.
The system embeds a query using the same configuration as the saved notes, then compares their vectors. For example, a search for a decision about an Ecuador project might find a note about an agreement made during a Quito trip, even if it uses different words. Similarity can help find a candidate; you still need to read the source to check whether it answers the question.
My current code goes further: it supports multilingual and literal search, groups passages from the same document, and includes optional reranking and graph links. Part 4 explains those additions. You can start without them. This series describes the codebase; it does not certify every setting in the deployed system.
Why a database, not a folder
A database gives me a structured place to store the original text, metadata, and embeddings together and make them available to different clients. I can also export records for analysis in R or another tool, rather than relying only on one note-taking interface. A hosted database still needs access controls, a backup plan, and a tested way to restore the data; it is not automatically private or free of cost.
This trades some filing work for software maintenance. If you only need a small, searchable set of documents and your current tools handle it well, there may be little reason to operate another service. Start with the smallest build that lets you test whether recall improves enough to justify that work.
The main reason I chose a database instead of a folder system or a note-taking app such as Obsidian or Notion was to give my agents a searchable index for retrieval-augmented generation (RAG): finding saved notes and using them as context for an answer. For my workflow, that matters more than maintaining a web of manually linked Markdown files. Storing embeddings alongside the notes lets me search by meaning while keeping the original text available. A folder-based system can also be indexed; the database is the approach I chose, not a requirement for RAG.
Before you begin
You need a coding agent that can inspect and edit your project, and a Supabase account when you reach Part 2. Familiarity with R or Stata will help with testing and checking source records; you do not need to know every framework before planning the build. Ask the agent to identify the required runtime and tools for your computer, then approve installations and remote changes rather than treating the prompt as an unattended installer. Model APIs and hosting may cost money.
Before adding real notes, check which services will receive them and whether you have permission to send them there. Employer, client, and research-data rules can prohibit a transfer even when you are personally comfortable with it. Use synthetic notes until both the data permissions and the access controls are clear.
Use a new project or an isolated development environment. Keep credentials out of prompts, screenshots, and Git.
Prepare your computer
Choose a location you can find easily and create a main folder named open-brain, or choose another name you prefer. Open this folder as the workspace for your coding agent. A new, empty folder is enough; you do not need a copy of my project. For an existing project, use its current workspace and ask the agent to inspect it before creating or moving files.
Keep application code, documentation, and database migrations in this project. If you add separate components, create folders for them inside it as needed, such as telegram for the bot and web for a browser interface. These are examples, not required names: agree on the layout with your agent before it writes files. Part 3 covers the bot, and Part 5 covers the browser interface.
The project folder holds your code; it does not automatically hold a copy of your notes or a database backup. In the design used here, the saved note text, metadata, and embeddings live in a Supabase database, while any uploaded attachments may live in separate storage. Part 2 builds the database. Before adding attachments, ask your agent where they will be stored and how they will be recovered.
Keep any local source documents, exports, and backups in a protected location outside the Git repository. Store credentials in a secure credential manager or protected local configuration that Git does not track. A .gitignore rule can help prevent new files from being added, but it does not remove files already tracked or erase Git history. Part 6 covers private repositories, access checks, and backups.
Give your agent this starting brief
Help me build a personal Open Brain in small, verifiable steps.
The goal is to save original notes in a PostgreSQL database on Supabase
and retrieve them by meaning as well as by keywords. Notes may have
structured metadata, such as a title, summary, tags, people, or actions.
Possible capture methods include this AI client and a Telegram bot. An
optional MCP interface could make save and search tools available to a
compatible AI client. Treat these as choices, not required components.
First inspect this workspace and ask which capture methods, languages,
AI client, metadata, and budget I want. If I have an existing repository,
ask for its location and read the relevant source before proposing changes.
Read current official documentation.
Do not assume I use the author's operating system or configuration.
Preserve an existing project's working contracts and data; propose
changes before replacing anything.
Propose a minimal Supabase/Postgres design that preserves original
notes and supports keyword plus semantic search. Choose an embedding
configuration for both saved notes and queries; explain any model,
dimensions, and search settings rather than assuming a fixed version.
Explain which data would go to each external service.
Create a short project plan and a safe example configuration containing
placeholders only. Record decisions in the repository so another agent
can resume. Keep credentials out of prompts and repository files. Do not
provision paid resources, create credentials, change access controls,
or deploy until I approve the relevant step.
Stop with the plan, prerequisites, and one concrete test for each stage.Save the resulting project context somewhere your agent can read it again, such as a project instructions file. Keep secrets out of that file. The prompts in the later posts build on this plan one component at a time.
Check the result
You should be able to explain where notes live, which services receive them, and how you will retrieve them. The plan should distinguish a local test from a deployed service and identify any expected costs.
If the agent jumps straight to a large application, bring it back to planning one test: save a synthetic note and retrieve it. Approve the test environment, any remote writes, and the API budget before running it. That test is the foundation for everything else.