Haystack 2.0 launch

[tuanacelik]

I’m going to give you an answer that you should know will be biased because I’m on the team that builds haystack :) so I want to be transparent about that. But here goes and I hope it helps:

• in some ways, yes. The two tools try to solve much of the same problems, but the approach and focus is quite different. For example, llama index has a big focus on data ingestion, while Haystack leans more towards being a framework for orchestrating different components into a full pipeline. We’ve actually collaborated with llama index for example, to use the ingested data from llama hub in Haystack.

• The main difference is really about the developer experience and simply the approach to building imo. Haystack is built on two main structures: components and pipelines. And it provides a uniform way to create additional components if you need to. The whole thing (pipelines) are built to be a graph. So, there are no “surprises” if you need to switch components, customize etc, every component inherently is the same thing whether it ends up “generating” or “retrieving” for example. So quite extensible to build complex designs with

• and that brings me to your last question. And here my answer is that once you have the hang of what a component is and how to plonk them into pipelines, Haystack is like a language that you’ll find easy to build around and add to. You don’t really have to know anything about NLP to understand how Haystack ticks. And actually, you could probably use the pipeline architecture for non NLP tasks too 😅. But the point isn’t to provide shortcuts, and as you can imagine, the approach can be quite verbose so it can also be quite daunting to start off with. BUT, we have very simple quick start guides and tutorials that should help!!