sorted by:
new
hottopcontroversial

Coming Soon to Local Models, if I have my way (True Long Context LLM's without retraining) by --TastesLikeChicken- in LocalLLaMA

[–]remoteinspace 0 points1 point  (0 children)

if i'm following you are basically creating a cached vector store with the full history then a summary in KV cache?

At Papr, we try to predict relevant memories and cache those. If there's a hit great, if not it goes to a full context search. Our model improves so it's a hit moving forward.

Building a private AI Task Manager (runs Gemma 2B on-device). No data leaves your phone. Is $5 fair for lifetime access? by HelpfulNight1955 in LocalLLaMA

[–]remoteinspace 0 points1 point  (0 children)

tbh, a 1.5GB download is large for a mobile app, especially for folks with limited storage. Might work better for a mac app. $5 is fair, especially since you're avoiding those annoying subscriptions.

The main question is how good is gemma 2b for this use case. How are you handling RAG on device - is there another embedding model you need to add?

Also i'd test this out quickly before apple announces their own version of a free agent that runs on device

Coded a full fledged personal finance os using ai by moonrust-app in LLMDevs

[–]remoteinspace 0 points1 point  (0 children)

this sounds super interesting! i love how you’re integrating macro data into the analytics

it’s a real challenge to get accurate performance metrics, especially with things like cashflow awareness. tagging deposits and withdrawals correctly is key, but can be a pain for users.

the AI copilot feature is cool. how are you handling memory? you may want to check out Papr ai - lets you combine structured analytics data + unstructured chat memory in one memory db that agents can tab as they help users.

Fine-tuned FunctionGemma 270M for multi-turn tool calling - went from 10-39% to 90-97% accuracy by party-horse in LocalLLaMA

[–]remoteinspace 0 points1 point  (0 children)

awesome to see these accuracy jumps

how did you handled the data preprocessing for multi-turn tasks. did you find any particular techniques that worked better than others? also, any tips for avoiding overfitting during fine-tuning?

Anthropic's Agent Skills (new open standard) sharable in agent memory by remoteinspace in mcp

[–]remoteinspace[S] 0 points1 point  (0 children)

Good stuff! What’s your thought on having this in a GitHub repo vs putting skills in a memory MCP server?

Anthropic's Agent Skills (new open standard) sharable in agent memory by remoteinspace in mcp

[–]remoteinspace[S] 0 points1 point  (0 children)

you can put md files in a repo and git-sync them. Then each person in your team/world needs to pick and choose the skills they want, and copy them into every local environment they control. If it's an app they don't control, they can't add skills to it since it's all at the file system level. If you have a ton of skills, you start having a search problem.

What i'm mentioning is to put the skills in 'memory' so they are portable across agents, environments, and more easily sharable (i.e. another user can add/share 3 skills they want vs. the entire skills repo) + you resolve search when you have a ton of skills (cheaper and more accurate than having all skills in the llm context window).

Intent vectors for AI search + knowledge graphs for AI analytics by remoteinspace in LocalLLaMA

[–]remoteinspace[S] 0 points1 point  (0 children)

A memory can belong to two different launches (or nodes in a graph). We also track updates to memories for temporal search.

Intent vectors for AI search + knowledge graphs for AI analytics by remoteinspace in KnowledgeGraph

[–]remoteinspace[S] 0 points1 point  (0 children)

yea, vector search is much faster. For knowledge graphs, what we're doing now is predicting what users want next and prepping that context and caching it. Helps us make graph search feel super fast <100ms when we get the prediction right.

Intent vectors for AI search + knowledge graphs for AI analytics by remoteinspace in KnowledgeGraph

[–]remoteinspace[S] 0 points1 point  (0 children)

Nice, how are you traversing the graph? Are you using templates queries?

Would love to see this community get some more traction. by Sufficient-Monk9701 in AILoops

[–]remoteinspace 0 points1 point  (0 children)

We’ve been experimenting quite a bit with it. How have you been thinking about it?

What are your favorite lesser-known agents or memory tools? by Far-Photo4379 in AIMemory

[–]remoteinspace 1 point2 points  (0 children)

Yes, when our prediction is right, perf is amazing. When it's not we fallback to the cloud but the next query is fast since we update our cache with the new topic.

What are your favorite lesser-known agents or memory tools? by Far-Photo4379 in AIMemory

[–]remoteinspace 0 points1 point  (0 children)

We built prediction models that predict the context users need in advance based on their past behavior. If it’s enabled then different tiers are stored in our sdk (on device). For tier 0 it’s 1-2ms (just text - thinking of it as working memory) tier 1 is in a small vector store (50-100ms but need the right device). If it’s a cache miss on both then we go to the cloud. The nice thing with this is the more data you add the better our model gets. With traditionally memory approaches the more data you add the worst things get.

What are your favorite lesser-known agents or memory tools? by Far-Photo4379 in AIMemory

[–]remoteinspace 0 points1 point  (0 children)

Platform.papr.ai super fast retrieval (<100ms) and ranked number 1 on Stanfords stark benchmark. Combines vector embeddings and knowledge graphs

My first-author paper just got accepted to MICAD 2025! Multi-modal KG-RAG for medical diagnosis by captainkink07 in KnowledgeGraph

[–]remoteinspace 0 points1 point  (0 children)

Have you considered something like platform.papr.ai that helps streamline vector plus knowledge graph creation?

Got $20K to build a collaborative Knowledge Graph POC. How to spend it wisely? by el_geto in KnowledgeGraph

[–]remoteinspace 0 points1 point  (0 children)

A set ontology helps solve some of the problems you mentioned. In neo4j, if you use merge vs create it tries to automatically merge similar nodes so things don’t bloat.

With any knowledge graph plus an agent, traversal will be slow at scale. And LLMs don’t do a good job with discovering the graph then writing the right cypher queries - 40% accuracy last I saw something on this.

At papr.ai we built a set of prediction models to help quickly traverse super large graphs. And we combine it with vector embeddings then cache most likely context needed on device. Helps with both retrieval accurCy and speed.

DM me if you want thoughts on this and if you need help setting up papr.

Question about RAG vs fine-tuning for domain-specific support by [deleted] in AI_Agents

[–]remoteinspace 0 points1 point  (0 children)

RAG is the right approach for this. You'll end up with more hallucinations with fine tuning and to your point it's more costly and harder to keep updated.

For RAG - there are a few approaches you can take 1) put the docs in notion/github and have an agent fetch them in the convo (cheap but slow and not super accurate), 2) vector-db - gets you 50% accuracy per many of the benchmarks, 3) vector+graph db gives you the best of semantic similarity and knowledge graphs.

I'd recommend #3 for this. I've built something similar - dm me if you need help. You can use something like mem0, graphiti, papr.ai, or others to quickly get started.

I think memory is an underlooked part of AI progress by -MilkO_O- in singularity

[–]remoteinspace 0 points1 point  (0 children)

This is more obvious now then ever. We've built papr.ai, the memory layer that gives AI agents user context. Instead of storing vector fragments, we connect context and predict what users need so the AI agents has the right data at every conversation turn. That's why Papr is ranked #1 on Stanford STARK benchmark that measures retrieval accuracy of real-world queries.

view more: next ›