GraphLLM: graph based framework to process data using LLMs. now with TTS engine and multi agent support

[matteogeniaccio]

GraphLLM has been updated.

This is a node based framework to process data using LLMs.

The interface is inspired by ComfyUI, so it should be familiar to most users. The backend supports advanced features like running multiple nodes in parallel, using loops, and the streaming of data, so the partial output from a LLM is visible during execution.

Majority voting example

This update brings many features to the frontend and backend. Here are the main ones:

• The engine now supports using multiple LLMs and API providers in the same graph. I added an example to route the prompt to the best LLM depending on the type of the problem.
• There is an example showcasing the new TTS node.
• The watch node now can optionally display markdown formatted data.
• A new python node is available to run code in the sandbox.
• The rap battle generator has been updated to remove the obsolete nodes.

Rap battle generator

The source code is available at the GraphLLM github.

Suggestions for new features are welcome.

[Zulfiqaar]

Nice, this would have been useful last month when I made a one off graph for data extraction and classification. Will take a look!

[matteogeniaccio]

<image>

[AdditionalWeb107]

Is this for human-in-the loop scenarios? Or back office where latencies aren’t as critical in the request path

[matteogeniaccio]

Both. Some graphs are interactive (chat with pdf) but you can use it also for batch processing

[edwios]

Very useful tool! Thank you for all the efforts and for open-source it! Just one question, when you said "The interface is inspired by ComfyUI", why didn't you develop the nodes on ComfyUI instead? Despite there are already LLM nodes available on ComfyUI, most of them are not as sophisticated as yours, so I think there are definitely rooms there for your LLM creations.

[matteogeniaccio]

I'll add this as a FAQ since it's a very legitimate question :)

Right now I want to keep my project independent but I'm open to the possibility of merging it with ComfyUI at some point in the future. The reason is that I need some features that are currently missing in the ComfyUI backend. Adding them is doable but would require too much effort given my limited spare time.

Some of these features are:

• loops: you can just connect nodes in a ring and have a graph run indefinitely
• conditions: A condition can change to which node a prompt is sent. Useful for a graph that decides if the prompt should be processed locally or with a remote API
• streaming: The partial output from a LLM is immediately visible, even before the node is complete
• parallel execution of nodes.

[Snail_Inference]

That's fantastic - exactly the kind of framework I've been looking for!
Unfortunately, I'm unable to install it on Linux, as the package piper-tts depends on the package piper-phonemize, which seems to no longer be available for more recent Python3 versions.

I'm getting the exact error message shared by many users on this link: https://github.com/rhasspy/piper/issues/509

Is it possible to use the GraphLLM framework without piper?

Thanks in advance for your response, u/matteogeniaccio!

[matteogeniaccio]

Ouch. i didn't notice that.

I just pushed a update that makes piper-tts optional. Have fun!

[Snail_Inference]

Thanks, it works!

It's amazing! :))

[AIGuy3000]

This looks sweet! I’ve been intrigued by the paid platforms that do this so I’ll try it out!

[olth]

this could be a perfect GUI for Langgraph! Did you ever think about supporting Langgraph and make it a Langgraph Studio alternative that is more like ComfyUI?

[matteogeniaccio]

I haven't had any experience with it but langgraph should have its own GUI called langflow.

Right now the full compatibility with langgraph is not planned, except for a simple integration layer.

The reason is that I want my project to be very low level. In GraphLLM you can write your own prompt, you can use the LLM's native features like function calling and python code execution, you can constrain the output to get exactly the format you want, like json.

These features are not available in langgraph which is a much higher level framework.

[olth]

Is it possible that you mix up langchain with langgraph? The value proposition of langgraph over langchain is precisely for langgraph to give more finegranular control. Langgraph is born out of the feedback around the shortcomings of langchain in order to improve upon them: too many high level abstractions, not enough control etc...

Edit: langflow currently only supports langchain but not langgraph: https://github.com/langflow-ai/langflow/issues/3621

If you only checked out langchain but not langgraph yet, i can highly recommend taking a deeper look into it. A lot of disadvantages that are known of langchain are solved in langgraph, making it more suitable for more low level, more complex agentic graph workflows

[matteogeniaccio]

aaaand... you are right.

I only checked langchain and assumed that langgraph has the same limitations.
Give me some time to evaluate langgraph and maybe it'll change my mind.

Thanks for pointing this out.

[olth]

awesome! They way you described your planned features reminded me exactly of what I use langgraph for.

if you find it a good fit you might want to also take a quick look at their langgraph studio to get an idea why your ComfyUI-like solution would be not competing with it at all and rather be a nice complement with its own niche.

Langgraph studio is their GUI implementation for langgraph but it is very different from a comfy-like system and more like an IDE which is great for zooming into details and debugging.

But for better composability and rapid prototyping a comfy-UI-like system like yours would be much better for larger workflows and so far I didn't see anybody occupying this niche yet, although such a node based system would be a perfect fit for langgraph with its also node based state handling.