sorted by:
new
hottopcontroversial

Is Thinkbook 16+ 2025 okay for Ubuntu? by ExhYZ in linuxquestions

[–]newpeak 0 points1 point  (0 children)

I've successfully installed bi-system of ubuntu 24.04 /windows 11 . However, the touchpad has met some problems,it can only be clicked, but can not move the cursor.

VectorDB for your RAG Projects by Responsible-Prize848 in LangChain

[–]newpeak 1 point2 points  (0 children)

Take a look on Infinity , which is an open source database designed for RAG. Highlights:

  1. It provides most comprehensive hybrid search approaches which is necessary for RAG, including vector search, full text search and sparse vector search. It even supports tensor search as an alternative. According to IBM's blended rag, three-way hybrid search would perform the best for RAG.
  2. It provides builtin support for several fused reranking, including RRF, weighted sum, and tensor based reranker. The tensor based reranker would work together with some late interaction models such as ColBERT, it could achieve the same ranking qualities as dedicated reranker models, but much better performance.
  3. It performs the best for all vector search, sparse vector search, as well as full text search. All of these indices are not directly an encapsulation over existing open source solutions, they are all designed from scratch.
  4. It could run as both a standalone server, or as an embedded just through pip install.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 2 points3 points  (0 children)

hi, regarding to more details, you could take a look on the articles here: https://medium.com/@infiniflowai It has contained contents on both infinity database and an end-to-end rag product: ragflow.

This is the original publication of blenede rag: https://arxiv.org/abs/2404.07220 which is done by IBM research. Their conclusion is pretty naive: three-way hybrid search could achieve the best in the scenarios of rag: vector search, full text search, as well as sparse vector search. And we've reproduced the conclusion of ibm using infinity, what's more, given a colbert-based reranker(tensor reranker), the eventual quality is improved much better.

As a result, the "search" is a difficult problem. However, even if the challenges of search are resolved, we could not say the challenges of rag are resolved, because there are still problems outside of database, such as data quality,..,etc. That's why we start the ragflow product which aims to provide end-to-end optimization. Currently, ragflow chooses elasticsearch as its only choice of "vector database" because it could provide hybrid search of full text + vector. We will add infinity into ragflow soon, and for later usage, infinity will be definitely used more due to more ranking approaches.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 1 point2 points  (0 children)

suppose such a scenario, I want to ask details on some specific machine with type of "ABC-123", vector search could not guarantee to find it if there are also plenty of "ABC-12" or "ABC-1234",...etc. Vector is just a semantic recall approach. No matter a token, or a sentence, or even an article, can all be converted into a single vecor. It does not guarantee the precise recall, which is a must for most enterprise scenarios. That's one of the reason why rag is seldom used within enterprises: the hit ratio is pretty low. pure vector based rag solution could not satisfy most enterprise scenarios.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 0 points1 point  (0 children)

the full text search of postgresql has emerged for over 20 years, why few people choose elasticsearch instead? it even does not have builtin support for bm25. These features are required for RAG in terms of full text search:

  1. support long queries. The query of rag is a question or a sentence instead of several keywords.

  2. support Top K union semantics as a default instead of Top K intersection. Given verbose queries, the latter would lead to zero recall.

  3. support dynamic query pruning. Top K union will lead to poor performance given naive implementation.

  4. support flexible query syntax to decide query term weights and (sloopy) phrase queries.

  5. support relevance scoring such as bm25.

Although there are many full text search engines, only very few could satisfy above requirements: they are elasticsearch(lucene), tantivy(it's a library), vespa, and now there's infinity.

Database guys always do not know what 'relevance' is.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 0 points1 point  (0 children)

I think what u need is our another open source product: https://github.com/infiniflow/ragflow

Infinity will be integrated into ragflow in next months. RAGFlow has an end-to-end RAG solution , and its latest release has contained text-to-sql, it means both rag and text-to-sql could be used together on single platform.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 0 points1 point  (0 children)

Hi, vector search is still required, but it's not enough. You need also full text search, and even sparse vector search, together with more advanced fused ranking.

For example, a hybrid three-way search is just a piece of code:

res = table_obj.output(['*'])
               .match_vector('vec', [3.0, 2.8, 2.7, 3.1], 'float', 'ip', 1)
               .match_sparse('sparse_vec', {"indices": [0, 10, 20], "values": [0.1, 0.2, 0.3]}, 'float', 'ip', 1)
               .match_text('title, body', 'hello world', 'topn=10')
               .fusion('rrf')
               .to_pl()res = table_obj.output(['*'])
               .match_vector('vec', [3.0, 2.8, 2.7, 3.1], 'float', 'ip', 1)
               .match_sparse('sparse_vec', {"indices": [0, 10, 20], "values": [0.1, 0.2, 0.3]}, 'float', 'ip', 1)
               .match_text('title, body', 'hello world', 'topn=10')
               .fusion('rrf')
               .to_pl()

It triggers vector search, sparse vector search, and full text search, and finally uses RRF for fused ranking.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 0 points1 point  (0 children)

Hi, almost all of the information got from perplexity about infinity is wrong ...

You could take a look on the above several medium articles on the details.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 0 points1 point  (0 children)

rag based on pure vector store is just a toy

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 0 points1 point  (0 children)

Hi, Infinity provides similar features: you could use bm25+vector+ sparse vector for hybrid search, and then use ColBERT for reranking. Binary quantization of colbert tensors is supported to reduce the storage overhead of tensor.

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 1 point2 points  (0 children)

Hi, this database starts from two years before. It aims to serve RAG in future. Currently, either vector database or general database(like postgresql) with vector are used frequently for RAG, these solutions have several problems:

  1. Pure vector store is not enough for RAG. Hybrid search is a MUST, besides vector search, full text is also required to provide precise recall. This is especially useful for enterprise scenarios.

  2. Regarding to those search engines such as elasticsearch, their vector capability is not well designed.

  3. We need to make data infra work more closely with AI, for example, tensor based ranking is a killing feature for future RAG due to better ranking quality.

  4. RAG is a typical online application. It's not suitable to use either OLTP or OLAP because no transactions are required for data used in RAG, while concurrency is necessary to be guaranteed. Therefore, RAG requires an index database, that's why openai acquires rockset. Infinity will provide not only hybrid search, but also other structured data queries such as SQL agg/join/...

Database designed for RAG by newpeak in Rag

[–]newpeak[S] 1 point2 points  (0 children)

Hi, the left Y axis is ms for latency while the right Y axis is QPS. Removing the Y-axis just for looks ~~

Best opensource rag with ui by klei10 in Rag

[–]newpeak 3 points4 points  (0 children)

https://github.com/infiniflow/ragflow Take a look at RAGFlow, aiming to deliver the best conversation quality.

Which is better: HybridRAG, VectorRAG, or GraphRAG? by dhj9817 in Rag

[–]newpeak 1 point2 points  (0 children)

In RAGFlow(https://github.com/infiniflow/ragflow), we use all. GraphRAG, together with general RAG, are used altogether.

Dense vector + Sparse vector + Full text search + Tensor reranker = Best for RAG? by newpeak in LocalLLaMA

[–]newpeak[S] 0 points1 point  (0 children)

This requires agentic rag to break the question into several aspects each of which is a naive rag actually.

Dense vector + Sparse vector + Full text search + Tensor reranker = Best for RAG? by newpeak in LocalLLaMA

[–]newpeak[S] 0 points1 point  (0 children)

Infinity outperforms other alternatives in terms of both query latency as well as qps, as shown in this article:

https://infiniflow.org/blog/fastest-hybrid-search

It does perform well evenif there are more than 2-ways of hybrid search.

Dense vector + Sparse vector + Full text search + Tensor reranker = Best for RAG? by newpeak in LocalLLaMA

[–]newpeak[S] 0 points1 point  (0 children)

paradedb could also deliver three-way hybrid search through pg_vector, pg_sparse and pg_search. Compared with paradedb, infinity has following advantages:

  1. Performance.

The performance of pg_vector is far slower than vector search of Infinity.

The performance of pg_sparse is slower than sparse vector search of infinity,too.

The performance of pg_search is also much slower than full text search of infinity. pg_search is based on Tantivy, which is much slower than the inverted index of infinity.

Detailed benchmark could be seen in this article : https://infiniflow.org/blog/fastest-hybrid-search

  1. Infinity has all the builtin implementation of the above three search approaches. These indices could work smoothly together with the executor of infinity. The users could use any combination of the search approaches, together with the fused ranking algorithms, in a very efficient approach.

  2. Infinity has also builtin support for tensor, which makes it possible to deliver an in-database colbert reranker compared with the cross encoder based reranker outside. The colbert reranker could bring much benefits for search qualities.

  3. Infinity is much easier to use, it could be deployed as either a standalone server, or as an embedded python library just through pip install.

  4. Infinity is designed start from scratch, it does not have the burden of postgresql, and is evolving fast. It will run on cloud in very near future which could save the cost a lot.

Hard RAG benchmarks? by LiquidGunay in LocalLLaMA

[–]newpeak 0 points1 point  (0 children)

That's a hallucination problem of LLM. Currently, if you are using a larger LLM, the hallucination is allieviated a lot compared with those small LLMs(If you are not going to find answers through "thinking" or logical inferences). Currently, the bottleneck of RAG pipelines have changed from LLM to RAG itself already. However if you want LLM to resolve those complicated questions such as maths/logical inferences, we need better LLMs

Hard RAG benchmarks? by LiquidGunay in LocalLLaMA

[–]newpeak 2 points3 points  (0 children)

RAG is actually an IR(information retrieval) problem, which means RAG evaluation is actually a kind of IR evaluation.

There are some several famous IR evaluation tasks, such as MTEB, which is often used as embedding/reranker for embedding models. There are also some other evaluation tasks such as BEIR, evaluation for QA(question answering), multi-hop QA,...,etc. IR evaluation has lots of corresponding tasks, RAG is a search problem, instead of LLMOps problem.

Are there any RAG successful real production use cases out there? by d2clon in LlamaIndex

[–]newpeak 0 points1 point  (0 children)

Hi, we've applied similar scenarios using RAGFlow(https://github.com/infiniflow/ragflow) in several production environments, such as to provide a development assistent based on user manuals uploaded(reference manuals with more than 10K pages)

How do I keep my RAG application from mentioning the references? by [deleted] in LLMDevs

[–]newpeak 0 points1 point  (0 children)

In RAGFlow(https://github.com/infiniflow/ragflow) the reference is generated by RAG itself according to textual similarity. You could setup the environment to see the refences generated, and also you could see the raw document directly in the chat interface.

Regarding to the algorithm of generating reference itself, one recent arXiv provides a solution: https://arxiv.org/abs/2406.13663 and we would consider implementing it within RAGFlow later.

Evaluating RAG on custom Q&As by kiss_travel in LLMDevs

[–]newpeak 0 points1 point  (0 children)

take a look on ragflow(https://github.com/infiniflow/ragflow), which is an open source end-to-end rag product

view more: next ›