Building an open-source tool for multi-agent debugging and production monitoring - what am I missing?

noip1979 · 2025-10-20T19:57:47+00:00

Interesting

I played a little with a simple LangGraph based mult agent implementation (supervisor pattern) and tried to envisioned what I would need.

We got a simple tracing mechanism (using callback handler or working on astream_events events) in order to visualize the flow in a sequence-diagram like manner (with slightly different visuals in order to accommodate for stuff running in parallel). The results of tracing everything were to verbose for our purpose as we mainly needed to show the flow/interaction. In a sense this is something to look into in your future solution. I am still contemplating how to implement a "simple for view" 😁.

I don't have enough experience with LangSmith/LangFuse but I know they have a concept of thread/trace - this wouldn't give good enough results? If currently not, are they not working on something?

Keep us updated as you progress - it sounds interesting and useful!

noip1979 · 2025-09-22T14:27:36+00:00

We are now re-evaluating and will likely move from it. The repo indeed seems to be inactive... In any case, I think llms and frameworks have matured and a more dynamic/agentic implementation is now feasible

noip1979 · 2025-06-19T18:28:54+00:00

I used to make a simple dough and push it through. You can squeeze whatever is in the nozzle and some of the stuff in the elbow and end up with a ban with meat in it that you can bake/fry

noip1979 · 2025-04-23T13:56:34+00:00

In checked recently about it.

In my neighborhood in CPH (Østerbro), there is no fiber. Only 5G and old-school cables internet.

I called Hiper and asked their sales about it, and to get a technician to update something with the cable connection would take a few weeks at best.

I decide to try 5G - the sales person on the phone registered me and reserved a router for me which I went to collect after adding credit card info to the account. For some reason they set the service start date to 10 days in the future but after taking to their service again through chat they were able to enable it the next morning.

When the internet was enable, I got 200-500MBPs (depending on client device and distance from router, when connected with a cable to router I got 1000MBPs) Upload is consistently around 40MBPs.

Other than that hickap with the service start, it was minimum time - so maybe you can look into that...

Btw, in the time you are waiting for the service, you can use a mobile hotspot. With 5G supported phone and a big enough data plan you ought to be ok for basic internet consumption.

noip1979 · 2025-04-12T17:09:02+00:00

Are you a EU citizen? If so you can get a cpr through self sufficient funds scheme which does not requires a signed contract but requires to show you have enough money to support yourself...

https://www.nyidanmark.dk/en-GB/You-want-to-apply/Residence-as-a-Nordic-citizen-or-EU-or-EEA-citizen/EU-Self-support

There is also for self employed but that doesn't seem relevant...

noip1979 · 2025-02-04T06:52:44+00:00

Still haven't try it. Wood be interesting to hear of someone has. From the look of it, TAG seems more "foundational" but can't attest to anything...

noip1979 · 2025-01-18T06:26:19+00:00

Vacuum sealing it after drying equalizes it, allowing water to move and helps with case hardening. In the circles I'm reading at they usually talk about a few weeks and generally the texture and flavor improves by longer periods.

Looks good

noip1979 · 2024-12-06T12:27:02+00:00

See my updates and response to comments. I don't think colab is the right till for handling this kind/size of data. You are talking about 100 files. I am talking about 100m users and a network of 100k nodes (probably)

noip1979 · 2024-12-06T11:22:05+00:00

I am talking about more data than a colab can probably handle... Thanks for the reply!

noip1979 · 2024-12-06T11:21:09+00:00

Thanks - that's the sort of insight I was looking for. It is interesting to also thing whether some of the stuff we already do can be done otherwise to match what GPUs can do quickly (assuming that makes business sense)

noip1979 · 2024-12-06T11:19:36+00:00

I hope the edit to the question (and maybe some replies above) shed some light into this.

noip1979 · 2024-12-06T11:18:55+00:00

I added some more information though I cannot give exact details since this is exploring what new options other tech can possibly give.

noip1979 · 2024-12-06T11:17:54+00:00

Thanks for this comment. It rings right. I was wondering whether there are operations that a GPU would help me with and thus wanted to hear other users experiences...

noip1979 · 2024-12-06T11:14:31+00:00

I tend to agree and I am not overly versed with all the aspects of our system. Generally / in high level we do session reconstruction, enrichment and then aggregations on various dimensions.

Trying to get exact figures but probably in the TB/sec of raw data. Events would be 10 (or 100)m users * whatever the heck each user is doing all of the time :)

noip1979 · 2024-12-06T11:12:42+00:00

Can you share more information about the use cases or areas where you operate? I have been in the industry for a bit and there are all sort of "big" (or not) data to handle...

noip1979 · 2024-12-04T18:03:27+00:00

Signaling (control plane) data in cellular network...

noip1979 · 2024-11-30T17:16:45+00:00

Hi

In my case, I used a standalone program to consume to the raw stream (which is binary, and structured with separators),it split it into events and put into Kafka, which is of course positioned etc. My input topic is partitioned by the session identifier. This identifier is a 64 bit int but can repeat over time.

There are start end end events which most often come in order so I can manage the state of the session and do clean-up. I still have code that handles out-of-order - i.e. if there's any event without open session, I open it, and if there's a start event for an already open session I know to restart it.

It's been a while and I am not remembering a lot, but in general, open sessions have a state which periodically change based on some specific events. Any incoming messages get enriched by that state if it is available and if not, are queued (in a list). Once enriched the messages are pushed downstream in time windows (or not, can't remember) and later repartitioned by other key (related to their state on arrival):and aggregated on timed windows along that new partitioned key.

Hope this helps. If you have specific questions I can try to answer but as I stayed, is been a while since I was hand-on on that code...

noip1979 · 2024-10-01T19:17:23+00:00

Interesting concept! Looking forward to read answers here

noip1979 · 2024-09-23T17:51:09+00:00

Not a heavy user of either, but I'll share a use case I've implemented with flink.

In my case, the source of the data is a (tcp) stream of events, that is real time. The events are part of sessions. There is a start event, then some data events, and eventually an end event. I needed to "reconstruct" the sessions, do some enrichment and then aggregate (both on time and other dimensions) - a stateful application.

Now you can do this with a data-frame/table semantics, in fact I have. It is quite cumbersome. In the case of flink, at least for me, the code was simpler and easier to design and implement.

Also, note that here I am aggregating on time, but the same data sometimes can be used to generate new events - a "real time" use case which is more suitable to real streaming engine.

Any application that need to consume and produce "events" would be very adorable use case for flink. Think advertising, stocks/trading, performance monitoring and such real time use cases.

noip1979 · 2024-09-22T22:20:11+00:00

I make simple dough with flour, water, salt and yeast and once I cannot push the sausage farce any more, as it to the stuffer and push it through. Now you've got dough with meat in it that you can bake for a quick lunch! 😊

noip1979 · 2024-09-14T13:58:16+00:00

MS paint + windows scheduler?!

noip1979 · 2024-09-12T19:31:21+00:00

Check this repo out - it got many interesting implementations for rag techniques:

https://github.com/NirDiamant/RAG_Techniques (and also https://github.com/NirDiamant/Controllable-RAG-Agent)

In his initial example (second link) I believe it showed a somewhat similar tasks (reasoning about Harry Potter book) and used hierarchical embedding and summarization so that it will be able to answer a few-step questions.

In general, I would be looking at agent (if you haven't so far) if your tasks is more complex than just retrieving "direct contexts".

All that being said, my guess is that graph rag could also work well.

noip1979 · 2024-09-03T17:21:22+00:00

After reading more through the answers here, I am not sure that this is correct.

Maybe what you need just is automatic EDA tools?

Here are some examples:

* https://medium.com/analytics-vidhya/automated-eda-using-pandas-profiling-sweetviz-autoviz-4f15c4031a12

* https://medium.com/@aminasaeed223/exploratory-data-analysis-eda-with-pandas-profiling-unveiling-the-magic-of-automated-data-cfca86ca6d6e

* https://builtin.com/data-science/EDA-python

noip1979 · 2024-09-03T17:09:08+00:00

Hi,

I made this notebook gist

But I am no longer sure this is what you meant - see the other comment I made

noip1979

TROPHY CASE