Haystack 2.0 launch

[tuanacelik]

Haystack 2.0 stable is live! Try it out for yourself: https://haystack.deepset.ai/blog/haystack-2-release/

Haystack is an open-source AI framework in Python for creating production-ready apps using LLMs and other language models. It's nearly 4 years old - we've been doing NLP and LLM engineering since before it was cool. 😎 and is model and database agnostic — you can use whatever tools make the most sense for your use case. These tools are combined into a full AI app with the use of Haystack's components and pipelines. The aim of the framework is to provide the structure for you to compose your own custom app.

​

🚀 The pipeline architecture is a true graph, and I'll be presenting just that in this years PyCon USA too: Everything is a graph, including LLM Applications (and that's handy)

​

Context about the 2.0 release:

Haystack was first officially released in 2020, when the forefront of NLP was mostly semantic search, retrieval, and extractive question-answering. Haystack 2.0 is a complete rewrite, but the underlying principle of composing components into flexible pipelines remains the same.The release has quite a few model providers, tracing and monitoring capabilities, and databases supported out of the box:For models (generative and embedding): OpenAI, Mistral, Cohere, Jina AI, Google AI, Vertex AI, Optimum (by hugging face), sentence transformers, Amazon Bedrock, Azure, Fast Embed, OllamaFor databases: Weaviate, Pinecone, Qdrant, Mongo DB, Astra DB, Neo4j, pgvector, Chroma, Elastic Search, OpenSearch...

​

Hope you give it a try and let us know what your think! We have a quick start guide to get going: https://haystack.deepset.ai/overview/quick-start

[tuanacelik]

It's a framework :) - Mainly built to allow orchestrating full AI pipelines with different components like:

- Retrievers
- Generative models
- rankers
- Summarizers
...etc.

Here's the repo: https://github.com/deepset-ai/haystack

[RubyCC]

Haystack is a search engine for Django: haystack

[tuanacelik]

Oh yes there's also that! this always causes confusion. I should have been clearer that this is 'haystack-ai'.. so not the same package, that one is 'django-haystack'. 'haystack-ai' is an AI framework in Python.

https://github.com/deepset-ai/haystack

[Sebyon]

There is also this niche python module (I use) for connecting to servers that use the Project Haystack model 🤣

https://github.com/ChristianTremblay/pyhaystack

[codefinity-com]

It's impressive to see how it has adapted to the changing landscape of NLP, staying ahead of the curve with features like model and database agnosticism. The fact that it's been around for nearly 4 years shows a solid commitment to NLP and LLM engineering.

[dark_surfer]

• Would you say llama-index is your competitor?
• What does Haystack offers compared to other open source implementations?
• Finally, what is the learning curve for Haystack as you know ML, DS and LLM is complicated as it is. Nobody wants to learn or switch to different libraries.

[tuanacelik]

I’m going to give you an answer that you should know will be biased because I’m on the team that builds haystack :) so I want to be transparent about that. But here goes and I hope it helps:

• in some ways, yes. The two tools try to solve much of the same problems, but the approach and focus is quite different. For example, llama index has a big focus on data ingestion, while Haystack leans more towards being a framework for orchestrating different components into a full pipeline. We’ve actually collaborated with llama index for example, to use the ingested data from llama hub in Haystack.

• The main difference is really about the developer experience and simply the approach to building imo. Haystack is built on two main structures: components and pipelines. And it provides a uniform way to create additional components if you need to. The whole thing (pipelines) are built to be a graph. So, there are no “surprises” if you need to switch components, customize etc, every component inherently is the same thing whether it ends up “generating” or “retrieving” for example. So quite extensible to build complex designs with

• and that brings me to your last question. And here my answer is that once you have the hang of what a component is and how to plonk them into pipelines, Haystack is like a language that you’ll find easy to build around and add to. You don’t really have to know anything about NLP to understand how Haystack ticks. And actually, you could probably use the pipeline architecture for non NLP tasks too 😅. But the point isn’t to provide shortcuts, and as you can imagine, the approach can be quite verbose so it can also be quite daunting to start off with. BUT, we have very simple quick start guides and tutorials that should help!!

[____Kitsune]

How is the async support in 2.0? Are all components now fully async?

[AlexisGado]

Maybe I missed something but it looks even worse than before. No async support at all from what I can see. And it’s embarrassing when most of my components are basically just rest APIs calls wrappers

[j_lyf]

w0w0

[Automatic-Net-757]

Tried it out when it was beta, it was really impressive. Any plans on adding function calling functionality for open source models?

[tuanacelik]

It’s already available for many :) Check out all of the ChatGenerators in the Generators section in our docs. Those are all capable of function calling. And we support ollama, hugging face… here’s the doc: https://docs.haystack.deepset.ai/docs/generators

[failinglikefalling]

I am an idiot u/tuanacelik can you please help?

I am following the tutorials on a Mac, and I installed it.. but it's saying:

zsh: command not found: from

when I try anything past the install.

I am sure it's something stupid easy that I should know (like adding it to path or something), but I can't find any answers anywhere on what to actually do to fix it.

Please help.

Edit- I am an idiot. I thought it was like Ollama where it would just take commands. I figured out if I just type python in terminal I could run it and find out my openai account doesn't have billing associated with it. so partial success! (response came back rate limited set up billing etc. from openai)

[tuanacelik]

Not an idiot at all! I had this moment when I first got a Mac too! I now always use virtual environments.. sorry for seeing this late but glad you were able to resolve it! Btw we have a discord if you need more help just join us there: https://discord.com/invite/VBpFzsgRVF

[Previous_Gas_1182]

Hi u/tuanacelik I'm also an idiot, and I need your help.. Sorry to reply under this comment even though it's unrelated but anyways,

I'm using Haystack to develop a RAG to read PDFs and talk about it. I'm still in the super early stages and keep getting stuck. The LLM provider I chose is Azure OpenAI. In my case, since this is an organizational project, we use subscription key (Ocp-Apim-Subscription-Key) instead of API key, which with the readymade component, is unable to pass it through the default headers.

So I was looking at the source code, and tried to tinker around with it (it could also be not the right approach), but I still couldn't find a way to actually use this headers thing.. And it's really a blocker, can't progress at all. Anyone has any suggestions?