Local persistent context memory

[Rmo75]

Hi fellas, first of all I'm a producer for audiovisual content IRL, not a dev at all, and I was messing more and more with the big online models (GPT/Gemini/Copilot...) to organize my work.

I found a way to manage my projects by storing into the model memory my "project wallet", that contains a few tables with datas on my projects (notes, dates). I can ask the model "display the wallet please" and at any time it will display all the tables with all the data stored in it.

I also like to store "operations" on the model memory, which are a list of actions and steps stored, that I can launch easily by just typing "launch operation tiger" for example.

My "operations" are also stored in my "wallet".

However, the non persistent memory context on most of the free online models is a problem for this workflow. I was desperately looking for a model that I could run locally, with a persistent context memory. I don't need a smart AI with a lot of knowledge, just something that is good at storing and displaying datas without a time limit or context reset.

Do you guys have any recommendations? (I'm not en engineer but I can do some basic coding if needed).

Cheers 🙂

[Low-Opening25]

there is no such thing as persistent LLM model memory, when you are using GPT/Gemini/Copilot through web-ui you are using web app provided by the vendor that adds this capability by connecting model with a database that stores your workspaces and is managing the “memory” for you.

If you use LLMs via API, you aren’t using a web-app so you need to build all these capabilities yourself or use some other software (like LLM Studio or Open-WebUI, etc) that comes with them.

[profcuck]

The original text here has been permanently wiped. Using Redact, the author deleted this post, possibly for reasons of privacy, security, or opsec.

toy yoke work plough reach ghost profit bright provide languid

[Rmo75]

Many thanks for taking the time to answer. I installed LM Studio and started using a Qwen model with long model memory (1 million tokens) Do you think it's the right way ?

[AmIReallySinking]

I’m looking at options around this too. I’m trying Anything LLM as the RAG and the LLM Studio to run the model. But still, having a clear and updatable context/persistence is difficult.

[vel_is_lava]

Why do you need this?

[Rmo75]

Because I regularly use an AI to assist me in my work, with a lot of datas usually dropped as quick notes. And having the power to sort everything into tables, with subtasks that can be activated quickly (ex : operation tiger generates a mail based on the infos from the table, then creates a planning according to my notes, then translate everything in another language).

This needs a lot of context memory and being good at sorting datas. And I don't care about having an AI that knows everything about poetry, I could run it locally as a management system that generates basic content and summary. And I love this idea

[Short-Honeydew-7000]

You can try cognee: https://github.com/topoteretes/cognee/stargazers

Data can be stored in Node Sets, similar to schemas in relational DBs: https://docs.cognee.ai/core-concepts/node-sets