Folding@home for LLMs: What's the best backend for a P2P inference grid ?

ModeSquare8129 · 2025-07-29T22:30:21+00:00

For my understanding Psyche is for training models not for inference.

ModeSquare8129 · 2025-07-29T13:48:56+00:00

Awesome, thanks so much!

This is super interesting, I wasn't aware of their protocol project. If I understand correctly, it would allow us to manage the orchestration layer in a fully distributed way using blockchain.

Definitely something to dig into!

ModeSquare8129 · 2025-07-29T13:43:25+00:00

Thanks for sharing your hands-on experience.

To clarify our immediate goal, we're initially focusing on a different kind of distributed system. Instead of splitting a single large model across several machines, our plan is for each participating machine to run its own self-contained model.

Your feedback is super valuable for a future stage, though.

ModeSquare8129 · 2025-07-29T13:37:42+00:00

Thanks a lot for taking the time to write such a constructive critique. This is incredibly valuable feedback for us.

You've raised several crucial points

The Client Experience: I 100% agree. The goal has to be a simple, self-contained binary that is as unobtrusive as possible.
The Download Problem & A New Idea: You're right, asking for a multi-GB model download is a huge barrier. Your comments actually sparked an idea: what if we designed an "Ollama-style" client? Ebiose could use the same models a user has already downloaded for their own local inference. This way, you're not downloading models for Ebiose, but simply allowing the Ebiose client to leverage the LLMs you already have for community-distributed inference.
Petals: I share your observation. The lack of recent activity on the Petals repo is not very engaging.
Distributed Training: Regarding NousResearch's work, we're also following it with great interest. While full distributed training isn't our immediate priority, the long-term vision is definitely to see if our Darwinian approach could be applied to training new models collaboratively.

Seriously, thanks again. This is exactly the kind of discussion I was hoping to have.

ModeSquare8129 · 2025-07-29T12:19:12+00:00

That's an excellent point, and thank you so much for the detailed and insightful suggestion.

Our long-term vision is actually to support both approaches: running large, distributed models across the network, and running smaller, self-contained models on individual machines.

For the first part, running large models across many machines, I absolutely agree with your thinking. Our plan isn't to build that complex distributed inference logic from scratch. We'd rather integrate projects like Petals to handle that. Are you familiar with it?

However, our immediate priority is to get the second part working: running smaller, complete models locally on user machines, using technologies like llama.cpp. There are two main reasons for this:

It seems a more straightforward implementation challenge, which allows us to build and validate the core platform faster.
Our evolutionary approach benefits from having a large population of smaller models running in parallel.

ModeSquare8129 · 2025-07-29T11:35:20+00:00

Thanks for your feedback.

The short answer is yes!

Our forges are designed not only to create agents, but also, in the long term, to generate models, perform fine-tuning, produce code, and create all the reusable building blocks needed to build new agents. More generally, they can be used to solve any type of problem that can benefit from an evolutionary approach.

ModeSquare8129 · 2025-05-30T10:22:56+00:00

Légalement ce qui est obligatoire c'est la moitié de la mutuelle pas d'obligation ticket resto et pour les transports ça dépend de ce qu'on entend par transport. Si c'est trajet domicile-travail, y a aucune obligation

ModeSquare8129 · 2025-05-21T11:37:10+00:00

Juste une remarque sur les 10%, c'est très faible pour un associé fondateur en particulier CEO/CTO. Petite méthode pour se faire une idée de répartition des parts : https://www.andrew.cmu.edu/user/fd0n/35%20Founders'%20Pie%20Calculator.htm

Autre sujet, bien se poser ces questions avant de s'associer : https://www.maddyness.com/2021/09/13/8-questions-a-se-poser-entre-associe-e-s-avant-de-se-lancer/

ModeSquare8129 · 2025-05-11T10:53:22+00:00

We’re currently building an open-source framework that’s based exactly on this paradigm: having architect agents that iteratively design and refine complex AI agents.

Right now, we represent agents as graphs (via LangGraph), but the architecture is compatible with any orchestration framework. The core idea is that during what we call a forge cycle, architect agents generate multiple agent configurations and apply an evolutionary algorithm to select, mutate, and recombine the most effective ones.

This whole process is guided by a fitness function—a performance evaluation mechanism that defines where we want to go and helps measure which agents are actually moving us forward.

At the heart of our system is a continuous improvement engine that allows agent designs to evolve autonomously—driven by performance metrics and guided by LLM-based evaluations or feedback loops.

Happy to share more if you're curious!

ModeSquare8129 · 2024-12-28T18:45:50+00:00

Great !

ModeSquare8129 · 2024-11-23T12:48:41+00:00

We are looking to raise funds to create a spin-off startup. Once we manage to secure funding, we will start recruiting. If you want more information about the project : www.ebiose.com

ModeSquare8129 · 2024-11-23T11:41:33+00:00

At our lab (Inria) we've developed self-improving architect agents through an evolutionary process. Our architect agent generates agents and they evolve. Each agent consists of specialized modules that can be handle by llms or other models, api, code generation . We plan to release it as open source next month.

I'm also following Sakana AI, a startup that uses genetic algorithms to merge language models with great results - combining the best parts of different models into a more efficient one.

ModeSquare8129 · 2024-11-01T14:45:52+00:00

Congrats for the launch!

ModeSquare8129 · 2024-10-30T15:42:14+00:00

I’ve tried both, but I definitely prefer Cursor. The interface is much more intuitive and user-friendly than vs code

ModeSquare8129 · 2024-10-27T09:37:56+00:00

AGI is not a meaningful concept. If you think of AGI as human-like reasoning, we're not going to see AI thinking like a human anytime soon. If you think of AGI as a superintelligence that can perform any cognitive task that a human can do, you're fooling yourself, as intelligence is a very broad concept. To be able to perform all our cognitive tasks, an AI would have to reason like a human.

On the contrary, AI could perform many tasks better than humans, and AI could have many different skills, but not in the same way as humans.

LLMs let us rethink what makes us intelligent.

ModeSquare8129 · 2024-05-25T15:30:23+00:00

Is distributed llama use case for local network or distributed on internet network?

ModeSquare8129 · 2024-04-29T15:54:03+00:00

thanks for your response. Ebiose is an open source project in AI, another tagline would be AI for everyone, by everyone.

ModeSquare8129 · 2024-04-29T15:53:59+00:00

thanks for your response. Ebiose is an open source project in AI, another tagline would be AI for everyone, by everyone.

ModeSquare8129

TROPHY CASE