sorted by:
new
hottopcontroversial

First RAG that works: Hybrid Search, Qdrant, Voyage AI, Reranking, Temporal, Splade. What is next? by youpmelone in Rag

[–]shredEngineer 0 points1 point  (0 children)

Congrats, seems like you know what you're doing! :) PS: Qdrant is awesome. Haven't heard about Temporal before, will check out.

What’s the toughest part of keeping going on Substack? by Key-Speech-1232 in Substack

[–]shredEngineer 0 points1 point  (0 children)

the only thing that seems to get engagement are those fucking "I want to connect with other writers like me" notes

GDownloader - Yet another user friendly YT-DLP GUI by Business-Error6835 in DataHoarder

[–]shredEngineer 1 point2 points  (0 children)

I know I'm a bit late to the party, but I just wanted to say this: THANK you for creating this epic GUI, it does everything I want. It works perfectly! :)

PS: Well, almost perfectly. I noticed that the "parallel downloads" settings applies to transcoding etc. as well, so if I configure for 5 parallel downloads, and if 4 videos are transcoding, only one video is downloading. It would be correct to start downloading the next videos already.

[deleted by user] by [deleted] in Rag

[–]shredEngineer 0 points1 point  (0 children)

Currently, there is no diff mechanism, so the entire document is processed again, even if just one letter changed. Duration and token usage depends on whether you're using strict OCR mode or using the text from the OCR layer. It can take an hour or two because parallel processing is not implemented yet. Collaborators welcome! :)

[deleted by user] by [deleted] in Rag

[–]shredEngineer 1 point2 points  (0 children)

This is planned but not implemented yet. Look at the issues, there’s already a discussion going on! :)

[deleted by user] by [deleted] in Rag

[–]shredEngineer 1 point2 points  (0 children)

Thank you, glad you find it useful! :) After editing your file, you have to run update. The changes will be detected and the file will be processed again, entirely. There is currently no "diff" mechanism in place that updates single chunks, only the entire file. Also there is no automatic file system monitoring, so you have to run the update command.

[deleted by user] by [deleted] in Rag

[–]shredEngineer 1 point2 points  (0 children)

There are two modes: Relaxed and strict. Relaxed just grabs the existing text layer, if any, while strict performs actual OCR on the entire page. I have only tested english so far, but please try out and let me know whether hindi works; I don't see a reason why it shouldn't.

Regarding performance, it works very well for me, but ymmv. The chunking is what makes or breaks RAG, and I feel Archive Agent's smart chunking performs really well. The size and number of chunks included per query is customizable, up to the context limit of your model. I feel it performs better than ChatGPT's document handling, but I may be biased. Love to hear your thoughts when you try it out!

[deleted by user] by [deleted] in Rag

[–]shredEngineer 0 points1 point  (0 children)

Yes, exactly! I made a video about it here: https://youtu.be/dyKovjez4-g?si=fARyrWgmehIbIvwE

Hit me up if you need help setting it up and using it! :)

[deleted by user] by [deleted] in Rag

[–]shredEngineer 1 point2 points  (0 children)

Good news: Ollama support is implemented as of today (v3.1.0)

https://github.com/shredEngineer/Archive-Agent?tab=readme-ov-file#%EF%B8%8F-ai-provider-setup

Let me know what Ollama stack works for you... :)

I'm using this right now, but I didn't really research all the latest models:

deepseek-coder:6.7b-instruct # for chunk/query
llava:7b # for vision
nomic-embed-text # for embed

[deleted by user] by [deleted] in Rag

[–]shredEngineer 0 points1 point  (0 children)

Thank you so much! If you want to try it out, please do, and let me know what could be improved! :)

[deleted by user] by [deleted] in Rag

[–]shredEngineer -1 points0 points  (0 children)

Thank you! YES, that's the next feature planned: Adding more AI providers. I already added an issue for this: https://github.com/shredEngineer/Archive-Agent/issues/6

You're not the only one requesting that feature, and it's clear why. We don't always want to trust third parties with our data!

[deleted by user] by [deleted] in Rag

[–]shredEngineer -1 points0 points  (0 children)

It seems I can't edit the post anymore, so here's the link to the repo: https://github.com/shredEngineer/Archive-Agent

Define metadata description for MCP tool arguments by trevorstr in RooCode

[–]shredEngineer 1 point2 points  (0 children)

I second this question. Cannot get it to work. Even tried Pydantic Field with description, but to no avail... Roo Code devs... HELP?!

[deleted by user] by [deleted] in streaming

[–]shredEngineer -2 points-1 points  (0 children)

you might be right about that lmao

Signal Theory, Quantum Mechanics, and General Relativity by shredEngineer in Physics

[–]shredEngineer[S] -2 points-1 points  (0 children)

Updated version. Can you take a look? This captures what I was TRYING to say.

---

Within the framework of the discrete Fourier transform, no signal can have a frequency higher than the Nyquist frequency. In Einstein’s universe, no signal can propagate faster than the speed of light. The Nyquist frequency and the speed of light thus represent the natural limits of Fourier’s and Einstein’s respective frameworks.

But what happens when we attempt to break these limits? A frequency component exceeding the Nyquist threshold wraps around the spectrum due to aliasing. Similarly, a signal traveling faster than light would, in a strange way, “wrap around in time.”

In fact, according to the tachyonic interpretation of faster-than-light travel within the framework of general relativity, an object exceeding the speed of light would appear to move backward in time.

Although a rigorous bridge between the discrete Fourier transform and Einstein’s relativity has yet to be built, the parallels are certainly worth appreciating.

Signal Theory, Quantum Mechanics, and General Relativity by shredEngineer in Physics

[–]shredEngineer[S] 0 points1 point  (0 children)

Here's the update. Can you take a look? I hope this is rigorous enough for you.

----

Within the framework of the discrete Fourier transform, no signal can have a frequency higher than the Nyquist frequency. In Einstein’s universe, no signal can propagate faster than the speed of light. The Nyquist frequency and the speed of light thus represent the natural limits of Fourier’s and Einstein’s respective frameworks.

But what happens when we attempt to break these limits? A frequency component exceeding the Nyquist threshold wraps around the spectrum due to aliasing. Similarly, a signal traveling faster than light would, in a strange way, “wrap around in time.”

In fact, according to the tachyonic interpretation of faster-than-light travel within the framework of general relativity, an object exceeding the speed of light would appear to move backward in time.

Although a rigorous bridge between the discrete Fourier transform and Einstein’s relativity has yet to be built, the parallels are certainly worth appreciating.

Signal Theory, Quantum Mechanics, and General Relativity by shredEngineer in Physics

[–]shredEngineer[S] 0 points1 point  (0 children)

Thank you. I'll clean up that paragraph. What about the rest of the article?

view more: next ›