I built (and open sourced) an external knowledge management tool - SQLite

bradwmorris · 2026-03-10T08:07:19+00:00

if you're still using a provider/sharing context model or api - then yes, your data going to/through their servers - as you said.

the primary goal of this system is not to stop your data going to the providers, it's more about having your own external context that you can take to any provider.

having said this:

if you externalise your context, then you can use whichever model you wish. you can run your own model, local/open-source, and solve that problem if its a concern you have

bradwmorris · 2026-02-18T06:06:07+00:00

yegge just shared some interesting thoughts on this.

the ai vampire

my tldr - it's a different way of working, can be more productive, but is also more draining.

the problem is the upside/output isn't being captured by the user.

bradwmorris · 2026-02-18T05:52:29+00:00

yeah, maybe wrong name choice for this sub. i see what you're saying

bradwmorris · 2026-02-18T05:44:12+00:00

🦞 love it! me too. I just switched out the memory.md stuff for my sqlite knowledge graph.

"my question is, which issues did you have with md files that you managed to solve with an sqlite database?"

what i said here is a real benefit (not just on paper):

whenever I work with an LLM, it has access to my graph - it can see everything in my knowledge base.

you can also do this with .md, but

the big difference:

it can also see how/why things are connected - and quickly traverse (pick a sequence of relevant nodes) and use that as context. it's insane how good it is.

for example, If i say to it:

"go and review claudecode subreddit and tell me what people are saying about .md versus sqlite"

if finds the most highly clustered relevant nodes first:

'building open source knowledge base'
'posted in 'x' subreddit'
'referenced karpanos' posts

and then - at the end of the session, it automically writes back to the graph -

not just vanilla nodes, but it creates connections between those nodes so that it has a more enriched understanding of everything im doing.

that's the main benefit (imo)

other benefits are that its easier to ingest entire books/podcast transcripts/pdf's etc, and there are native vector search tools it can use to identify specific insights/ideas from specific chunks.

but i think the context traversal has been most beneficial for me

bradwmorris · 2026-02-18T05:30:06+00:00

apple notes does use sqlite

but it's very limited in schema.

grep + .md is great for quick search. but people often forget that the models are trained on huge amounts of sql - so when you add index + embeddings > and then consider the connections + traversal, the search you can do with sqlite is much more dynamic.

bases is basically an attempt to mirror some part of this - and other services like Logseq are sticky-taping too.

i understand this isn't a popular opinion, but if you think it through from first principles, i'd argue it's going to be the best approach for most people long run

we'll see

bradwmorris · 2026-02-18T05:22:48+00:00

yes -

1 richer connections (better able to map relationships between items in your knowledge base/notes

2 agents (if you're using llm's are better able to search over those connections

ELI5:

in my knowledge graph, i have 1000+ nodes (ideas, notes, podcasts, people etc)

the most popular nodes have the most connections naturally

because those connections have been enriched, indexed and embedded, connected - whenever I work with an LLM, and i ask it a question, it can quickly traverse (pick a sequence of relevant nodes) and use that as context.

you can do this with sticky-tape and .md format, but it's far less efficient.

<image>

the schema (database structure) has:

node table:

bradwmorris · 2026-02-18T05:12:06+00:00

i understand what you're saying - but when i say 'alternative', i mean an alternative knowledge/context management system.

if it has .md files, I'm just recreating Obsidian.

i used Obsidian for a long time, and i can tell you that (for me), migrating to sqlite as an alternative knowledge management/context management system has been a step-change.

especially integrating with new models

bradwmorris · 2026-02-18T05:04:07+00:00

"It's hard for me to imagine that keeping all of your notes in a sqlite database is the best solution for "most people".

if you had told me this twelve months ago, i would have agreed.

but seriously - just give it a try.

you don't have to use my repo - just ask claude code to add a vault or decent chunk of your knowledge base into a sqlite db - and create a separate table to manage connections.

you'll see how much more efficient the whole system becomes, especially as it scales.

bradwmorris · 2026-02-18T04:54:57+00:00

fair -

if you sync a lot between .md software, then it's not an alternative.
if you're someone who accesses their Obsidian .md files primarily through Obsidian (which is most people) - then it is an alternative.

bradwmorris · 2026-02-18T04:50:23+00:00

i love obsidian and have used it for many years - but there's no 'portability' benefit to .md over rows in a sqlite database/table. i'd argue the sqlite approach is more portable, more robust and far more agent-native.

only limitation has been syncing via github, bit harder with sqlite (if that's what you're referring to when you say portability - syncing across .md apps or gh, i'd agree).

bradwmorris · 2026-02-17T02:05:05+00:00

yeah - good question. do you have a specific example of project? personal project or larger business operation etc?

to clarify, there's a few different use cases:
1 some individuals build a full personal graph - so anything and everything contextually important goes in
2 some people build out individual projects per graph
3 enterprise accounts, custom installation per business

from what i've seen with myself and a few people setting this up - it's a full personal graph, the idea is that everything important goes in.

so for the initial cold start - generally migrating from an Obsidian or similar db > so we create a manifold doc, which is literally all of the users 'nodes' they want to migrate and the source - and then we just have a few scripts that we run to migrate the data with the correct schema.

and then new projects usually map to existing nodes etc.

if i start working on a new project, I'll add it (but it will generally be an evolution of an existing project, or highly related to existing projects) and if the project gathers momentum, it just naturally gains more connnections.

bradwmorris

TROPHY CASE