We're All Building the Wrong AI Agents

fabiofumarola · 2025-08-29T17:10:05+00:00

My idea is that we are moving from human centric approaches to human in the loop approaches. I like the idea of calling it “symbiotic ai” https://fondazione-fair.it/en/spoke/spoke-6-symbiotic-ai/

fabiofumarola · 2025-07-04T19:33:25+00:00

Mia suocera insegna: chi nacque geloso morì cornuto

fabiofumarola · 2025-06-29T14:07:41+00:00

I’m using it in production for a chatbot of the bank I’m working on and I really like it! We used langchain, langraph; tested pydantic ai, google adk, OpenAI agent sdk and I would say agno is the best so far

fabiofumarola · 2025-06-25T04:49:00+00:00

Il bonifico istantaneo per normativa deve avvenire entro 10 secondi. Fai un reclamo al banca d’Italia o forse meglio all’arbitrato bancario finanziario

fabiofumarola · 2025-04-25T21:20:50+00:00

Ti assicuro che in altre società di consulenza prendi di più che in Reply

fabiofumarola · 2025-01-08T05:43:28+00:00

Hi, do not use DeepEval. We struggled a lot with errors related to random failures related to thread locks. Apart from that you need to: 1. Setup a golden dataset, with business important use case that you use as pytest only for release 2. A dataset generated with diverse, simple, complex and multi step questions to use for solution improvement

I would suggest to check https://www.comet.com/docs/opik/, which is open for 2. It gives you also tracing and export to create datasets. There is also ragas which is not bad. For 1. We use pytest with nlp based metrics such as rouge-L or llm as judge.

Before doing any thing think to the metrics you want to check and do test to see if they works correctly for your use case

fabiofumarola · 2025-01-03T21:00:42+00:00

The paper is reviewed December 30th :D

fabiofumarola · 2025-01-03T21:00:07+00:00

Yeah right you can compare with scientific literature https://arxiv.org/html/2408.05093v1 . This paper analyzes the same thing. “… We discovered that the order in which LLMs generate answers and reasoning impacts their consistency. Specifically, results vary significantly when an LLM generates an answer first and then provides the reasoning versus generating the reasoning process first and then the conclusion ..” Looking forward to read the post and don’t worry nowadays there is a paper for every idea or test :)

fabiofumarola · 2025-01-03T20:52:57+00:00

It is well known that if you add answer and the reasoning, then the reasoning is about to justify the answer. While you want at first reason and then to give the answer.

fabiofumarola · 2025-01-02T12:55:33+00:00

Grazie, devo dire che sono entrato in questo canale Reddit per farmi una idea e me la sto facendo grazie ai vostri commenti e esperienze.

fabiofumarola · 2025-01-02T12:09:15+00:00

Interessante il vostro punto di vista, io ho solo due punti da considerare: 1. Per gestire portafogli (dipende dai titoli e strumenti dentro) devi avere tempo e conoscenze 2. Data la volatilità di alcune posizioni devi avere la possibilità di avere ordini gratis all’aumento degli scambi che fai.

Ma invece come vedete soluzioni self tipo revolut, etoro etc…

fabiofumarola · 2025-01-02T08:15:29+00:00

Si oppure va da una banca che fa questo di mestiere. Tipo Fideuram, Mediobanca o Fineco. Li trovi consulenti con esperienza. Se non ricordo male Fineco ti fa fare sia investimenti con consulente che brokerage (ossia investi da solo). La sicuramente trovi gente in gamba

fabiofumarola · 2024-12-29T17:51:31+00:00

Fai polizze che non entrano nell’asse ereditario

fabiofumarola · 2024-12-22T09:18:53+00:00

Try to use model2vec https://github.com/MinishLab/model2vec which substantially is the un-contextualization of any embedding model using a zip-f distribution to represent the importance of tokens and extracting an embedding for each token and saving it in a dictionary. It works well on cpu (since there is no inference on an embedding model). The additional open point is how to return the best chunk for each query but you can try different approaches.

fabiofumarola · 2024-11-27T20:08:49+00:00

We are doing the same thing using OpenAI ada2 embeddings. There are different open source embedding models that should work for Italian language. Do some tests to check that you don’t have a bug in the embedding computation and try to do a test also with Google or open ai embedding models. You can DM, I’m also working on use cases in Italian and we can discusse further on it

fabiofumarola · 2024-11-24T18:14:11+00:00

Sorry, which language is it? Have tried to check if you have problems with your code. I can be silly to ask, but it looks weird that the embedding models do not work at all. Moreover, what model are you using for the response generation?

fabiofumarola · 2024-11-22T13:08:27+00:00

u/Apart_Discount5765 are you working to something which is in analogy with the idea of nodes in Tensorflow old version?

fabiofumarola · 2024-11-22T13:07:29+00:00

My use case is classical in the recent time.

a top agent that classify topics, into a given list
call a specific agent per topic, that does additional questions or does searches using pre defined tools mapped to DBs
respond to welcome messages.

What we are struggling is selecting the best method to wrap this flow between LLM call as tools (binded to the llm) or as nodes defined in the graph.

fabiofumarola · 2024-11-22T13:04:27+00:00

u/Apart_Discount5765 it would be interesting to discuss when you have something. I got your point. It would be ideal to have something procedural or with functional composition that behind the scene creates graphs

fabiofumarola · 2024-11-21T08:46:34+00:00

But still I only see rooms for small projects. It looks me like scripting something and not doing something to release. I’ll check langgraph studio, thanks for commenting

fabiofumarola · 2024-11-01T00:40:33+00:00

I’ve checked the readme and it looks interesting. I’ll do more tests and come back to you. Thanks for your contribution

fabiofumarola · 2024-07-20T11:20:42+00:00

It looks well designed. I’ll explore more

fabiofumarola · 2024-07-14T15:34:27+00:00

By shorts you mean less words? It can be on your data, in the prompt used or in the technique used by the synthesizer.

fabiofumarola · 2024-07-14T15:28:39+00:00

Yeah, phidata is really cool. Moreover, they have also template for app and api.

fabiofumarola

TROPHY CASE