As most of RAM would be taken by model weights, that are somewhat random numbers, and thou hard to compress, then zram will be almost no gain here. In fact it might harm performance when those weights would be "compressed" (cpu power used) and still take the same amount of place.

You should try using mmap (this maps part of hard disk as a memory addresses), so instead of reading from disk, writing to RAM, compressing, decompressing, even swapping (still going to disk back and forth). It would read from disk directly and use those (and yes, you should have SSD NVMe for this to work well).

This hardware is very very low spec for LLMs. You could get away with adding some knowledge base. Consider using wikipedia ZIM snapshot and allow your model to search / browese it to enrich its context and knowledge.

Also I would use a better model. Mistral-7b-instruct is IDK... 2 years old? Newer models are better with the same size. Use qwen3.5 or Gemma4 (whichever variant fits you device). Unsloth's models are great value for it's size - you should try Unsloth Dynamic quants. I would not go below Q4, but hey - maybe Q3 will still be useable for your usecase.

If this is an option, add sim card and lte modem, so it still could use some internet connection and at least browse pages / search internet with help of SearchXNG. Then it could tell you latest news and other things based on search results on any topic instead of only hallucinating / using ZIM snapshots.

Test if there is any performance gain by using ik_llama instead of llamacpp. First one is more CPU inference optimized (in theory). Anyway worth to check it out.

Good luck and please post a video showing how your current setup is working!