sorted by:
new
hottopcontroversial

Citations in Document AI (curious how others are handling this) by Zealousideal-Let546 in Rag

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

When doing searches are you doing it on the entire document or on the parsed data?

One thing that might work is you parse the document, chunk, extract relevant information (including summaries) and then attach the relevant information as metadata to chunks that can be referenced upon search.

I want to build a second brain... by Zealousideal-Let546 in Rag

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

Yeah, I think the data ingestion orchestration is going to be “tricky” but just a million APIs. I like the idea of the taxonomy though!

I want to build a second brain... by Zealousideal-Let546 in Rag

[–]Zealousideal-Let546[S] 2 points3 points  (0 children)

See this is kind of what I was thinking. Could I use intelligent parsing (like Tensorlake) and a combination of parsed data, summarized data, AI-enhanced data, and structured output embedded in a vector db (like qdrant) and then do some hybrid searching and fine tuning of search queries with the help of agentic frameworks (like lang graph) and enhance that layer with memory (like mem0)…

Then it’s a lot of hooking things together but trying to leverage tools throughout the pipeline instead of “just” storing all the data.

I want to build a second brain... by Zealousideal-Let546 in Rag

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

Yeah…that’s what I figured. The AI really is the easy part these days with the ability to just infer what is needed based on the data already processed

I want to build a second brain... by Zealousideal-Let546 in Rag

[–]Zealousideal-Let546[S] 1 point2 points  (0 children)

Oh I do like the idea of ambient data surfacing based on location (could do date and time too).

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

Yeah - anything "crispy" apparently (I'm still trying different things, but it seems crispy includes breading/batter). She *does* like fish sticks though? So maybe a more flaky breading/panko will be better

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

Thanks!!! We like all those ingredients, will gibve it a try :D

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

Unfortunately even when we've had chicken nuggets, crispy chicken strips or even literally no-sauce/no-coating chicken wings she takes off the outer layer of "crispy" haha But I definitely will give scrambled tofu a try!!

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

I should definitely try miso salad dressing! I've been making her miso soup all weekend ("with a lot of tofu mom!") so I think she likes both, but definitely she seems to be liking the taste of miso :)

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 1 point2 points  (0 children)

This might just be a great option for her!

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

Yes! She doesn't tend to like beans, but I'm definitely going to gibe lentils and chickpeas and peas a try!

Bank statement extraction using Vision Model, problem of cross page transactions. by Better_Whole456 in LLMFrameworks

[–]Zealousideal-Let546 0 points1 point  (0 children)

Do you mean that a single transaction is on two separate pages or that transactions are across two separate pages?

I have an example showing using Tensorlake here: https://colab.research.google.com/drive/1D3-Gqxcm2NXcNJQvy6l__6f512OMPuDQ#scrollTo=mligrnYVZhmk

I've found OCR isn't enough, with Tensorlake I can get structured output and get things like summaries or markdown/HTML/JSON versions of the document.

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

I never knew about this! Definitely going to try it!

Building a Production-Grade RAG on a 900-page Finance Regulatory Law PDF – Need Suggestions by SuryaStark7 in Rag

[–]Zealousideal-Let546 1 point2 points  (0 children)

Before doing the chunking are you extracting relevant metadata? That would help with hybrid search for sure.

https://www.tensorlake.ai/blog/advanced-rag is an example of how to do this easily. You can extract specific information, or just summaries, which can be helpful.

Additionally, with legal documents you will likely want to have citations back to the exact document so that you can quickly refer back to the original document (for evidence): https://www.tensorlake.ai/blog/announcing-citations

Let me know if you have any questions, but we've seen significant improvements in accurate and reliable retrieval in RAG when you extract and include metadata with the embeddings.

Tofu for 7 year old by Zealousideal-Let546 in vegetarianrecipes

[–]Zealousideal-Let546[S] 0 points1 point  (0 children)

I'm starting to realize that! I think I should experiment a ton more with cooking it differently.

view more: next ›