Open Brain: Finding the Right Notes
Before you start: have Part 2’s working search and saved test IDs. You do not need a Telegram bot or browser interface to begin.
Done when: you have a repeatable set of queries with expected results and evidence for keeping or rejecting one improvement.
Finding a related passage is easier than finding the right passage. A personal knowledge system needs to handle paraphrases, exact references, different languages, and limits such as “only this project.”
In Part 2’s fictional collection, “Who will share the script, and when?” should lead to “Ana will send the code on Friday.” The note naming the script is related, but does not say who will send it or when. If the note about Ana is missing, adding a more elaborate answer generator will not fix the missing source.
While I am working on a project, an idea often comes to mind about something else: another task, a feature for a software package, or a question about epistemology and AI that I want to research later. The easiest way for me to capture it is to grab my phone and send a note to my Open Brain through Telegram.
The note does not have to be long or polished. I try to capture the thought as it occurred to me, even if it does not make much sense at first. Later, I search for what the idea was about instead of trying to remember the exact wording. Those searches have been quick in my own use, but that does not guarantee the same result for every query. The checks below help identify where retrieval needs work.
What each addition does
- Hybrid search combines meaning-based and word-based matches.
- Document grouping keeps several passages from one document from filling the whole results list.
- Reranking sorts an initial set of search results again, using a closer comparison with the query.
- Graph links can bring in explicitly connected material.
Graph search is optional. It does not make every query better, and a reranker cannot rescue a relevant note that never entered the candidate set.
Keep a small scorecard
Start the evaluation with Part 2’s three test notes. For each query, record the expected note ID, whether it appeared in the first five results, whether its project filter held, and how long the search took. Write down the expected result before testing.
For the question about who will share the script, use the ID returned when you saved the note about Ana. Record the displayed IDs from the unchanged search, then repeat the same query after one proposed change. This gives you a comparison you can inspect instead of a general assurance that search is “better.” It is a test procedure, not a claim about my measured results.
Save synthetic queries and their expected IDs in your project so you can repeat them. Keep real notes and sensitive exports out of the repository. If you do not have an evaluation script yet, ask your agent to create a small one. Keep a few separate queries for checking whether an improvement helps beyond the examples used to tune it.
In my implementation
The MCP search reviewed for this series in October 2026 combines meaning-based matches with English and unaccented text matching, optional Spanish search, exact terms, and typo tolerance. It groups document passages and adds nearby context. Reranking and graph links are optional. Extracting graph links, searching them, and using them in reranking have separate switches. Only the MCP path has a guard against weak matches based on meaning alone. These are code capabilities, not verified runtime settings. Test the clients and settings you use rather than assuming they all follow this path.
Prompt: measure before expanding
Evaluate our current Open Brain search before adding features.
Create or extend a small retrieval evaluation. Start with Part 2's three
synthetic notes and test paraphrases, exact references, and project filters
through the clients we use. Do not build new clients for this comparison.
Once that baseline is repeatable, add cases relevant to my collection:
English/Spanish, typos, date filters, long documents, and questions with
no useful match. Explain which cases have not been tested yet.
Write down expected source IDs before running comparisons. Measure
whether relevant notes are found and appear near the top, plus latency
and API cost. Obtain my approval for the API budget and any new provider
before running measurements. Distinguish candidate retrieval from
final displayed document results.
Check that every optional stage preserves authorization and all requested
filters, including graph-added results. Show failures with query,
expected IDs, and actual IDs. Keep the current configuration unchanged.Check the result
The report should include concrete cases, not just an overall score. For example, analysis.do should retrieve the note with that filename. The Survey analysis filter should exclude the travel note even when optional graph retrieval is enabled. A long document should not crowd out all other sources.
For “What hotel did we book?” when no note contains that information, the system should make the lack of evidence clear. If it generates an answer, require traceable source passages.
Prompt: test one improvement
Choose one improvement justified by the failed search cases. Compare it
with the unchanged baseline on the same test set.
If the cases do not justify a change, report that and keep the baseline.
If testing a reranker, keep the candidate set fixed and compare no
reranker, the current option, and one documented alternative. Ask before
sending data to a new provider or running paid calls.
Report quality, latency, cost, and regressions. Recommend adoption only
if the tradeoff is worthwhile on the separate check queries too. Reject
authorization or filter regressions. Keep the baseline when gains are
unclear, and compare extra cost and latency with limits I approve.
Do not migrate embeddings, enable graph search, or deploy changes as
part of this comparison.Keep changes to facts in mind too. “The review moved to Friday” should not erase the evidence that Tuesday was the earlier plan. If outdated answers become a recurring problem, evaluate explicit dates and superseding records before adding a larger memory framework.
If a relevant note never appears, investigate retrieval first. If it appears too low, investigate ordering. If the right source appears but an AI answer misstates it, investigate answer generation separately. These are different failures.
Rerun representative searches after changes as part of the maintenance checklist.
Further reading: Supabase hybrid search.
Next: protect and maintain the working system, or add the optional browser interface if browsing would help you use it.
Previous: mobile capture (optional). Return to the database and first search, or review the full series.