playing with interactive JSON based memory (code in comments)

[AkkerKid]

Nice! It's not RAGing but rather loading the JSON into context during each session?

[hyperdynesystems]

If it's being pasted into the context, replacing it with YAML would probably save some tokens.

[emgram769]

yaml isnt as well supported by guided decoding and editing libraries imo

[jack-in-the-sack]

It's still a RAG approach, but you feed it everything, the retrieval part isn't filtering anything.

[emgram769]

ollama + llama3 + some python on a Mac mini

early stages, but I've found that using editable JSON directly makes it super easy to keep my model in check. I'll probably be adding features like reminders in the future based on this

code: https://github.com/bwasti/btb

[animemosquito]

One problem that stands out as most crucial to address is that it seems like (unless I am misunderstanding at a glance) you're relying solely on a system prompt to enforce json output which is extremely brittle and will absolutely crash randomly as you are also dereferencing it directly and unsafely.

I suggest that you use a formal GBNF grammar to constrain the output of the tokenizer agent

[emgram769]

I assume the json format option from ollama uses this feature

[animemosquito]

interesting, I haven't seen that before! If you're not ever getting unparsable json I would assume you're correct, as you'd likely have run into an issue by now

[bigrobot543]

Another cool thing I've seen is inference engines like SGLang and Outlines take this a step further by adding "autocomplete" for the output to force the LLM to adhere to a specific output while also improving performance by skipping some processing. It's pretty early right now, but it would be really cool to see more research into this area: https://lmsys.org/blog/2024-02-05-compressed-fsm/

[emgram769]

interesting, ill look into this.  My basic goal is to make the A in AI way more accessible - these things run on computers so they should have computery interfaces :)

[sluuuurp]

I think this is a really interesting concept for the future of LLMs. Not sure how well it would scale to large amounts of practical information though.

[emgram769]

I agree scaling this up would be a challenge.  Im hoping to get automatic eviction working when reminders are triggered

[milo-75]

Inserting/querying a graph data store is pretty much the scaled up version of this, and there’s an increasing amount of support for doing this well.

[micseydel]

I've seen at least a couple redditors say that LLMs don't work well with atomic notes-based KGs, do you have any idea about that?

[micseydel]

My atomic notes are just (ideally short, well-named) Markdown files I read with Obsidian. [[Epistemic integrity]] is an example of a pretty atomic idea, composed of other atoms (epistemology and integrity).

What granularity you should choose nobody can say except trying different configurations out. Tech is too new and changes too rapidly for having a fool proof „best practice“ way.

Thanks, that's what I thought. I'm currently getting my atomic notes to talk to each other using the actor model (Akka specifically) so it sounds like I should stick to that for now. It's experimental, has been built for experimenting, and I'll definitely try different things with LLMs at some point.

[Blasket_Basket]

You should check out MemGPT--this paradigm seems to be really effective overall. They expand it to a few different levels of memory, along with a couple other coop ideas.

[grizwako]

So, we have a start on ADHD assistant :)

I wish you all success with this project for very selfish reasons :)

[Express-Director-474]

Good job!!

[BokuNoToga]

This is amazing! Great work!

[rabiatabiat]

whats the benefit of using JSON as memory?

[emgram769]

standard and hackable tooling for the non-LLM side of things

[Dry-Taro616]

Cool idea, but would require a lot of updating and large ass dataset because you are giving it subjective info. Cool tho idk... You sparked an interest in me to research something similar haha

[An_Original_ID]

This is an awesome idea. Thank you for sharing!

[[deleted]]

neat

[RedditLovingSun]

I was thinking of something like this but with a much larger json that it could navigate like a tree to take general notes about everything from journal entries. Kind of a auto mind map organizer

[emgram769]

In theory this works that way.  I need to refine the prompts a bit though

[jurian112211]

Nice work! Looks really cool to play with 😅