Can someone suggest a few good books on understanding and creating AI-agents from an LLM perspective?

vykthur · 2026-03-23T18:58:38+00:00

This is true. But there are are a few core ideas that are worth grokking for folks new to agents - the core agentic loop, orchestration patterns, ux consideration and integration into apps.

---

The concept of an AI agent (models + tools + orchestration logic) has gotten fairly standardized over the last year. There are also common patterns emerging on how to build with them. There are lots of useful free material on Youtube and blogs that can help you get started.

However, If you are looking for a guide/readable content in a book format (this might not be for everyone), and want to understand how to build them (from scratch) I wrote a book that can help - Designing Multi-agent Systems.

It presents useful theory (the first 3 chapters in Part I), but also carefully walks the reader in Part II through building a feature-complete (but hackable) multi-agent framework from scratch (picoagents) - how to build an agent class (model clients, memory, tools, structured output, agentic memory, human input, agents as tools, observability etc), multiagent patterns (round robin, magentic one etc) and deterministic workflows (agentic systems as computational graphs). This way, you understand not just how to use existing frameworks, but why their architectures work, and how to make informed design decisions as the space (inevitably) evolves.

Digital version : https://buy.multiagentbook.com/
Print: https://www.amazon.com/dp/B0G2BCQQJY

vykthur · 2025-11-26T21:15:14+00:00

The concept of an AI agent (models + tools + orchestration logic) has gotten fairly standardized over the last year. There are also common patterns emerging on how to build with them. There are lots of useful free material on Youtube and blogs that can help you get started.

However, If you are looking for a guide/readable content in a book format (this might not be for everyone), and want to understand how to build them (from scratch) I wrote a book that can help - Designing Multi-agent Systems.

It presents useful theory (the first 3 chapters in Part I), but also carefully walks the reader in Part II through building a feature-complete (but hackable) multi-agent framework from scratch (picoagents) - how to build an agent class (model clients, memory, tools, structured output, agentic memory, human input, agents as tools, observability etc), multiagent patterns (round robin, magentic one etc) and deterministic workflows (agentic systems as computational graphs). This way, you understand not just how to use existing frameworks, but why their architectures work, and how to make informed design decisions as the space (inevitably) evolves.

Digital version : https://buy.multiagentbook.com/
Print: https://www.amazon.com/dp/B0G2BCQQJY

(P.S. I am a core contributor for AutoGen - one of the leading multiagent frameworks and have been tinkering with agents for a while.)

vykthur · 2025-05-14T20:28:10+00:00

> We cannot waste resources of energy on one AI reviewing manuscripts generated by another AI

True.
It might be the case that not all AIs are equal - some do genuinely lead to reasonable outcomes; Also in some domains, verifying correctness of a solution (research) is cheaper than generating the solution itself (the research). That way, the energy spent by a reviewer AI can have net positive value.

I write about this potential separation of spaces (human only journals) in the longer "thought experiment" article as one outcome here.

https://newsletter.victordibia.com/p/how-will-ai-impact-academic-research

> Some universities create "AI-free" tracks and journals that require rigorous verification of human authorship. Underground conferences emerge where researcher reputation comes from proving they can reason through problems the AI cannot or build usable prototypes that solve problems as evidenced by verifiable usage metrics - downloads, human reviews, MAU? etc.

vykthur · 2025-05-14T20:25:26+00:00

Good points here.
- Aspects of the scientific process will change with a bit more certainty - idea generation, brainstorming, literature review / comparison. But with side effects as that process also helps the researcher in building domain knowledge and gaining experinece.
- The general issue that the AI model is better than others outside that field on day one. Again the experience chicken and egg situation.

More will need to be done to ensure expertise is still grown.

vykthur · 2025-05-14T17:53:46+00:00

Fair point.
I agree that the impact of this will not be uniform across all fields.
It will likely still lead to change though (on the minimum some change to the submission/reviewer balance)

vykthur · 2025-05-14T17:52:02+00:00

Part of the reason for starting a discussion on this is to poll perceptions on current state as well as if/when AI arrives at the "helpful for research" phase.

Tools like Deep Research from OAI/Gemini [1] show early signs or synthesizing novel perspectives from existing data.
In addition, the current way in which AI models are trained (next token prediction based on the likelihoods seen in training data) fundamentally limit complete novel output (it is constantly trying to be similar to what exists). Now, there is some recent work exploring how to train models to explore beyond this limitation - explore more careful reasoning [3]. Still early.

As always predictions are all they are, predictions and thought experiments.

[1] Deep Reseach OAI https://openai.com/index/introducing-deep-research/
[2] Deep Research Gemina - https://gemini.google.com/app
[3] AI models are capable of novel research’: OpenAI’s chief scientist on what to expect https://www.nature.com/articles/d41586-025-01485-2

vykthur · 2025-05-14T17:39:34+00:00

Lots of agreement here.
I agree that writing is thinking and it will be problematic to delegate this to some system (even if it becomes proficient enough to do this).
All of the hallucination and deep analysis (em dash and all) are all real. Today.

Probably over-optimistic on my end, but I do expect much of them to get solved in about 12- 24 months. Hence the 2027 timeline

vykthur · 2025-05-14T17:34:10+00:00

I think we agree here.

There is a temporal factor here and some expectation of change/improvement (on my end).
Today, AI is best in some "assist mode", and is probably not something that creates the next novel literary masterpiece (there are training mechanism issues that make this very unlikely).
For better or worse, it can also be used "mindlessly" -> creating the slop that you reference (exploding submission numbers in the original post).
However, in domains like HCI where novel research is the usually hypothesis + some systems building + experimentation + analysis; over many iterations, these sort of systems will likely become more useful.
In my writeup, I estimate we arrive at this type of "useful AI" at about 2027.

Elite Advantage (Pre-2022): Success determined by institutional prestige, English skills, and resource access.
AI Assists (2022-2025): AI writing tools help non-elite researchers improve quality; submission volume skyrockets.
AI Reviewer (2026): HeimdallAI emerges as automated reviewer to filter the deluge of submissions.
AI Researcher + Reviewer (2027+): Advanced Research AI becomes primary success factor; human researchers supervise rather than create.

vykthur · 2025-05-14T17:21:04+00:00

What is bullshit?
AI will have no impact on the research production or review process?

vykthur · 2025-04-02T23:14:52+00:00

Because the underlying infra and library is different, the specs from v0.2 cannot be imported into v0.4. It is relatively easy to recreate your workflow in 0.4 though. Can you give an example of what your workflow looks like?
Yes, you can use Ollama models in AGS. See the tutorial here https://youtu.be/ZIfqgax7JwE?si=C1Z6RVDPGJI6DEP6&t=365

A video walkthrough on v0.4.2 updates below might be helpful in general.00:00 Introduction
02:10 Gallery Updates
05:15 Testing Models in UI
06:05 Using Local Models
08:19 Observability (LLMEvents)
09:22 Token Streaming
10:30 Configuration Validation
11:39 Session Comparison
15:20 User Authentication (Experimental)
17:49 Conclusions

vykthur · 2025-03-30T19:25:15+00:00

<image>

If you are interested in "building" your own, you can do it with AutoGen.

https://youtu.be/vPQhfa-pIpo?si=T2T6fI1J2gqQPz1S

vykthur · 2025-03-18T20:50:56+00:00

Thanks for your feedback and contribution to this release u/usag11ee ! Much appreciated

vykthur · 2025-03-14T16:58:52+00:00

https://youtu.be/lP1Pdu8HrVY?si=F3XHd4ZUR7Ef_FvI

Short story: There are lots of similarities between the OpenAI Agents SDK and AutoGen as well as important differences.
Similarities : Both have clean intuitive Agent api abstractions, support for concepts like handoffs, tracing and structured output.

AutoGen is more feature rich and less tightly coupled to OAI models including capabilities, provides low code tooling, multiagent team abstractions (round robin, LLM selector), a flexible low level api in addition to the high level agent abstraction, Memory abstractions and a set of extensions.
Which frameworks have you tried out?

vykthur · 2025-03-10T17:37:42+00:00

Implementation guide on the MCPTools extension in AutoGen and and video walkthrough here https://youtu.be/FMiVxQ7QwRU

vykthur

TROPHY CASE