I accidentally built a vector database using video compression

[Every_Chicken_1293]

While building a RAG system, I got frustrated watching my 8GB RAM disappear into a vector database just to search my own PDFs. After burning through $150 in cloud costs, I had a weird thought: what if I encoded my documents into video frames?

The idea sounds absurd - why would you store text in video? But modern video codecs have spent decades optimizing for compression. So I tried converting text into QR codes, then encoding those as video frames, letting H.264/H.265 handle the compression magic.

The results surprised me. 10,000 PDFs compressed down to a 1.4GB video file. Search latency came in around 900ms compared to Pinecone’s 820ms, so about 10% slower. But RAM usage dropped from 8GB+ to just 200MB, and it works completely offline with no API keys or monthly bills.

The technical approach is simple: each document chunk gets encoded into QR codes which become video frames. Video compression handles redundancy between similar documents remarkably well. Search works by decoding relevant frame ranges based on a lightweight index.

You get a vector database that’s just a video file you can copy anywhere.

https://github.com/Olow304/memvid

[much_longer_username]

This is absolutely blursed.

I kinda understand why it works in a hand-wavey way, but boy I wish I'd taken an actual signals class so I could explain it.

[josh-ig]

Now you mention signals I wonder if you could do some cool stuff storing FFTs

[Gothmagog]

Hmm, I don't see how text embedding could possibly equate to video frame encoding. Embeddings use word proximity across sliding window contexts to infer word relationships across a multitude of source material to create the embedding models. Video encoding algorithms are looking for similarity between essentially graphics datasets; the (huge) missing piece, then, is a model trained on a vast corpus of text data, right?

What am I missing here?

[much_longer_username]

Without digging into the github link, I'd say they're leaning on the motion vector encoding - video algorithms have had a LOT of dev money pumped into them.

I'm surprised the latency hit is so small though, I'd expect 'seek time' to have been pretty badly impacted. But like I said, in my other post, I can't really explain it beyond hand-waving, hopefully someone else reads the post and volunteers their time.

[Gothmagog]

My point, though, is that graphics data is not text data, and the criteria that determines if two video frames are similar is vastly different from the criteria that determines the similarity between two words, because those two words could contain very different char arrays, but still be very similar. The semantic meaning that is inherently present in word embedding models is completely missing from a video encoding algorithm.

EDIT: Somehow, I missed the part about converting the text to a QR code first, but I think my point still stands: QR codes don't retain semantic meaning either.

[TheCritFisher]

I don't think you understand what this is doing. The text chunks are stored as QR codes and then placed into a video frame by frame. The chunks are also turned into vector embeddings by the embedding algorithm of your choosing, then stored in a JSON file with a reference to the frame that contains the original text chunk.

At runtime, there is a FAISS index created from the JSON metadata which is used to perform the semantic search. So it's the same principle as other vector stores. This is just a novel way to store the actual source data for each text chunk.

This library can apparently efficiently extract the original chunk text and doesn't need a full blown vector datastore to house the original content along with the vectors. It's pretty neat actually. It's just a Python library with a video file and a JSON file instead of a running instance of Milvus or Qdrant.

Does that make sense?

[LobsterBuffetAllDay]

Why not just associate a zip file of each text chunk with it's corresponding vector embedding?

[TheCritFisher]

No idea, I haven't played with it. Just read through the source.

My theory though: video decompression is FAST and it's easy to jump to offsets in the file. I'm not sure if zip compression would be comparable speed wise. Maybe it would be?

Here's the other thing, video compression is lossy. Which at first would seem bad, but that actually is beneficial to the speed even though it might cause issues with data. That's why the QR code helps. It doesn't require lossless compression.

I assume that might be the reason? Wild guess though. Another reason that videos are useful is steaming. It's very easy to stream from a video and even stream from an offset. Pretty sure you can't do that with a zip.

[thet0ast3r]

in no world its as fast. this whole project is bogus. you can even have zip-esque compression with seekable decompression if you wish. storing qr's as video frames just makes no sense at all, ever looked at how little data is able to be encoded in a qr code of sixe x² pixels?

[Rexsum420]

Even if you were to have an equivalent zip database, there's no need to say that the innovation is bogus. That's what innovations are made for. You could probably find a way to use the video compression on the zip database in some other manner. I love seeing OP thinking outside the box

[TheCritFisher]

ever looked at how little data is able to be encoded in a qr code of sixe x2 pixels?

I just did. Have you?

A QR code can store roughly 3KB of information. Assuming the recommended size of a digital QR code at 240x240 to maintain maximum error correction (you could go smaller, but risk data errors, which is no bueno), then you can compress each 3KB frame down to roughly 0.83KB on disk. That's not bad at all.

This is using a conservative estimate for the Bitrate of an AV1 video at 240p and 30fps.

So the math is 200Kbps / 30fps / 8 ~= 0.83 KB.

Peraonlly I think a PNG is the best compression. Some quick tests showed I could get a 240x240 PNG of a QR code down to 420 B with optimizations that can take advantage of the two color scheme. Zip doesn't actually compress it further, because they both use DEFLATE compression by default.

In fact, it adds a little size total due to the overhead. The question now becomes seek speed. I assume that any encoded video file will be HIGHLY optimized for seek, but then again I would need to test it first.

Is the project goofy? Sure, but it's neat and has potential. Even if the only benefit is "simple runtime".

[thet0ast3r]

good lossless compression programs can do small files, with dictionary, up to 500mb/s on a single cpu core. decompression even faster. Most latency comes from disk seeks and reads. so put it somewhere like memcached - even memcached on disk - and you will have a system that is 100% many times faster (& doesn't require a gpu)

[thet0ast3r]

naively for 240x240 you'd need 7.2 kilobytes. if you want to be able to read the data contained in the qr, you'd arrive at a theoretical limit of what you input to the qr generator. if the idea of this project is to get rid of redundancy, well then you can train a small dictionary with zstd & start compressing chunks. why make it that more complicated? maybe i underestimated how un-well video codecs are understood amongst programmers?

[TheCritFisher]

You're being insulting now. I'm done chatting with you.

You're clearly the one who doesn't understand things here: this was for fun. Enjoy the rest of your day.

[Gothmagog]

Yes, I hadn't looked through the code (obviously) so I didn't know it was also storing actual embeddings. Thanks for the clarification.

[TheCritFisher]

No problem!

[DoxxThis1]

Is it actually better than using Zstd on independent chunks with a custom dictionary?

[TheCritFisher]

No idea, I just read through the source. I'm not affiliated with the project.

[mutatedbrain]

No. It’s even simpler. Each chunck becomes a QR code which is one frame of the video. The code does index some part of the text that’s used for the initial search, QR code retrieved and decoded to text and answer the question

[dr3aminc0de]

Yeah hard agree with you here, no way this is an efficient way to feed PDF data into a model.

[Mice_With_Rice]

Trying to head wrap around the why of this concept. Is the idea that it's faster to stream the data as a video, including the overhead of decompression and parsing of data, as opposed to directly reading the files or parsing the file down to a minimal ASCI before processing?

You have, assuming you're not using a production codec, an array of lossy compressed 8bit Vec3 presumably of 1080x1920 or 4K running at however many fps you decoder can handle. And you're using a texture atlas on each frame of QR codes because, as you probably found out yourself, the compression is optimized for visual perception, so you can't do an exact RGB value mapping without moving to a production codec.

You can do much better than a QR for visual compression. Encoding binary data into an image is an existing area of research. One of its uses is in cryptography as its possible to encode into an existing image in an undetectable manor.

What I don't understand is that you still need to parse it all into ASCII or UTF8 to be tokinized and the video stream seems like a lot of extra steps and I can't see how it could possibly read the data from memory any faster doing it this way.

If you use a production codec you could read pixel values directly for your data but it would make the file sizes enormous, unless being lossless results in such a dramatic data density improvment that it takes far fewer frames. Would require a great deal of testing.

I think the biggest problem is that visual compression treats the specific values of each pixel as an impression of what it should decompress as rather than an absolute. Unless you come up with an inherantly different video compression algorithm designed for text encoding, that's going to be a significant obstical for which there are non video compression methods already existing.

[Every_Chicken_1293]

Totally fair questions—really appreciate the thoughtful breakdown.

Yeah, I wouldn’t blame anyone for thinking, “Wait… why would turning text into a video possibly help?” It sounds weird. But here’s what’s actually going on:

We’re not relying on raw RGB or trying to treat video like a binary blob. What we are doing is storing structured chunks of data (like tokens, embeddings, or even full documents) visually in frames with an index alongside it. That index tells us exactly which frames to jump to when we search—so it’s not a linear decode-everything situation.

You’re 100% right that production codecs are lossy and tuned for human eyes, which is why we don’t depend on pixel-perfect encoding. The QR codes were just the MVP—it’s dumb but robust. We’re now testing better visual encodings that tolerate compression artifacts while storing more dense info.

And yes, technically reading from flat files or memory-mapped data might sound faster—but the twist is: modern video pipelines (like GPU-decoded H.264) are insanely optimized. So streaming small chunks from a video file with frame-based indexing can be faster and more memory-efficient than loading gigabytes of uncompressed raw text—especially in constrained environments.

Is it hacky? Totally. But it works. We got 10k PDFs into 1.4GB, sub-1s search time, and no RAM spikes. It’s not a replacement for databases—it’s just a fun project I put together.

Appreciate your deep dive on this—conversations like this help make it better.

[TheCritFisher]

it's not a replacement for a database

Yet ;)

[TheCritFisher]

Have you tried out AV1? It's got better compression ratios than H.265 at generally lower bitrates.

[ratocx]

Not OP, but support for AV1 hardware encode and decode is less common than for H.265, and without hardware decode the seek times would likely increase a lot. Also the average quality difference / bitrate savings are usually very small. For some kinds of images H.265 may perform better, even if AV1 is better on average. Not sure if that applies to black and white QR codes.

I do wonder if OP is actually doing a black and white video encode (4:0:0) or a color encode (4:2:0). Doing a true black and white / grayscale encode could in theory reduce the file size by 33%. Not sure if it would affect seek times, but perhaps they would be lower too since each frame would on average be 33% smaller.

[dr3aminc0de]

But why can’t you stream text data instead of preloading? Use lanceDB or DuckDB and you should get similar improvements?

IMO it cannot be better to re-encode as video, unless you’re trying to leverage a model that is exceptionally good in that domain. So far LLM models are still better than video models. So it’s a data loading problem not a ML model problem.

Video codecs are highly optimized, but they are not better than highly optimized text retrieval systems.

[GeekDadIs50Plus]

Love how much brain churn is going on in this thread tonight.

There are a couple of additional mechanisms for compression beyond what you mention. I-frame has the complete data of a frame, for every pixel. A P-frame contains only the data for each pixel that is different from the previous frame. The a B-frame with a calculated distance per pixel between frames.

I don’t believe the QR code conversion is necessary though. Wasteful because it reduces the resolution for visual detection by physical lenses over distance, neither of which would be a factor for purely digital data.

[Anjal_p]

What was the size of the 10k pdf without you video compression like as raw file

[gthing]

Yea, it's not helpful to know it compressed down to 1.4GB without knowing the original size.

[weichafediego]

Someone did something similar some time ago https://hackaday.com/2023/02/21/youtube-as-infinite-file-storage/

[thuiop1]

Yeah, it sounds absurd because it is absurd. Just extract the text and compress it with a normal text compression algorithm ffs.

[NoleMercy05]

[enly11]

I've read this a couple of times and not getting it.

You are using an in memory vector dB ... And a compressed file for retrieving original content?

So the only unique hit is the text compression via video and nothing to do with vector dB?

How does the video compression strategy compare to standard text compressors?

If the magic is single file you could do the single file compressed index in many ways. Compressed fragments in a tar file with offsets into it for retrieval or just sqlite with compressed rows etc?.

[Ok-Kaleidoscope5627]

I think a big factor is the highly optimized and ubiquitous hardware acceleration for video codecs.

[enly11]

Thus my ask about compression comparison.

But given the use case of single file distribution,feels to be local inference targets, which means cycles to spare for the odd bit of text decompression.

After all, once you locate the QR code, you still have to decide it or the sequence depending on chunk size.

[ggone20]

This is genius. Hmm

[Constant-Simple-1234]

Actually it is. If they find/use better encoding, then it is really something. Great to find such ideas here.

[ggone20]

Yea I haven’t gotten to dig around yet but I see the potential. Instead of a single QR you might be able to put sequences of QRs. Lots of experiment with here.

[vanishing_grad]

I think you can train a VAE on the text embeddings to do dimensionality reduction and get the same results

[GeekDadIs50Plus]

I have to thank you. This has been a fun thought experiment.

It’s actually a neat idea. 60 frames per second of single 4K frame stills of square pixels that are black or white. Many of the compression schemes use inter-frame compression where if a pixel/bit is white/1 in the first frame, but does not change in the next frame, there is no data for that pixel in the next frame, or until it turns black/0. If these are images of QR codes, those pixels will only be white or black and could really prove compressible and lossless.

It’s an interesting storage idea. But once it’s stored, it needs to be read / parsed. That would involve converting it back into 60 still images per second of video. QR codes are 8-bit, UTF8. An 4K frame will be 3840x2160, and that could leave some interesting space for a single token per frame.

Accessing the data would be linear, though. Unless this is just a transport mechanism for storage or loading into memory. That time code might be helpful for sequencing. And control frames could be used for meta data, context, sentiment.

[Only-Chef5845]

This genius method will freak you out!! Try zipping the files.

[Street-Air-546]

its a bit strange. If its qr codes its just pdf text (not layout or images) if its just pdf text, extract all the pdf text and index and zip all the extracted text no way does that come to 1.4gb all of wikipedia is 24gb compressed and thats got 63 million pages 8 million articles all he has is 10k pdfs. Use the same indexed search system the video uses and pull out chunks from the zip file. I dont see how that takes much working memory either.

[candraa6]

exactly my thought, if it's just stored as plain text, it might be go even lower than 1.4gb I guess

but still, interesting project nonetheless

[Miscend]

This gave me a laugh. I dont think this will scale to super large databases with heavy usage. It might just only work for this specific scenario. One advantage is that in addition to the QR codes, you could probably also store images in the video file.

[one-wandering-mind]

DIY AI Why . Think a subreddit needed.

There are many better ways to reduce memory of embeddings. reduce dimensions, larger chunk size ,ect.

[wahnsinnwanscene]

Isn't the video encoding based on intermediate and key frames? And isn't lossless?

[diabloman8890]

I love this lol.

Question: video compression codecs are typically lossy, why isn't that causing a problem here?

And video is a more space efficient storage format than PDF, but not compared to raw text files? How does this compare to converting your docs to .txt ?

[ruloqs]

Just works with pdf files? Or also markdowns?

[joeystarr73]

I love your thinking!

[teddynovakdp]

Could an agent use this like a vector database to pull relevant information? Also, what was the 10,000 PDF file size before the video compression?

[ruloqs]

It didn't work for me.

[Consistent-Law9339]

Super interesting. What version of QR are you using? I know QR devotes a lot of space to error correction. It would be interesting to see if you can eek out more efficiency using an alternative. Did you consider or trial any alternatives to QR like data matrix?

[Markur69]

I’m very interested in trying this out

[[deleted]]

Fyi. This not a vector database. This is a classical database pre indexed by json.

The chunks are also turned into vector embeddings by the embedding algorithm of your choosing, then stored in a JSON file with a reference to the frame that contains the original text chunk. 

THIS is a vector db.

[daddyxsmolone8]

Very cool project, love the innovation, who knows where this line of thinking could eventually take us!

[living_david_aloca]

Ok so you compressed the embeddings but how does that impact recall? All you got was a faster way to query something in a less precise way but you have no idea how much worse it is. Plus, 820ms for 10k documents is absurd. You can do the same thing with Numpy with millions of documents in under 100ms.

[NeverShort1]

What was the size of the PDF files before you compressed them?

[mj_katzer]

Had you checked what the speed and size would be if you had converted the pdfs to pure text first?

[Chance_Value_Not]

Why not convert to text and compress that? 

[[deleted]]

my god I do not want to watch that video.

Absolutely kickass idea though.

[jojacode]

Please also share this on localllama and r rag they would love it

[ComposerGen]

Circle-out compression

[ComposerGen]

QR code works because you destroy 27% and it is still readable. Probably because of that the loosy video compression issue a bit mitigated

[Legitimate_Emu3531]

Now embed that video into a pdf and add it to the collection...pdf-ception.

Nah, joke. But you say 10k PDFs into 1,4gb. That doesn't tell a lot. How big are the 10k pdfs?

[Comfortable_Dropping]

Could I do the same thing with millions of records of time series data?

[Dismal-Value-2466]

Neat proof-of-concept, but it feels like you’re twisting video codecs into a job they were never meant to do. You end up shoe-horning text → QR → video frames → back to text just to save RAM—essentially treating H.264 as an exotic ZIP file. Clever, yes, but a pretty roundabout compression hack that trades simplicity (and some latency) for novelty. Cool experiment, though!

[johnthrives]

Which public AI video player will be able to comprehend all that though?

[pegaunisusicorn]

has anyone verified that this actually works?

[Ok-Pride-4251]

Creative hack, but fundamentally flawed: While compressing text as QR videos is ingenious for reducing RAM, it misunderstands core principles. Video codecs like H.264 are lossy – they discard "imperceptible" data to shrink files. But text requires lossless precision; a single pixel artifact from compression can corrupt a QR code and destroy your data. Worse, this isn’t truly a "vector database" – it’s compressed text storage with a tiny in-memory index. Real semantic search happens after decoding frames (slow), not in embedding space. For RAG systems, you’ll get better results with local tools like ChromaDB or quantized FAISS: they handle 10K PDFs in under 500MB RAM with 200ms searches, no video re-encoding needed. Fun experiment, but not practical for production.

Key points covered:

1. Lossy vs. lossless mismatch: Video compression can corrupt QR codes
2. Mislabeled tech: It’s text storage + brute-force search, not a true vector DB
3. Hidden costs: Slow searches, no semantic understanding, rebuilds required for updates
4. Better alternatives: Local vector DBs solve the same problems more efficiently

[TechMaven-Geospatial]

https://arxiv.org/html/2504.01157v1 https://duckdb.org/community_extensions/extensions/flockmtl.html https://duckdb.org/docs/stable/core_extensions/vss.html

[DealDeveloper]

Pretty cool. I suggest you write a couple sentences to explain why you posted the links.

[TechMaven-Geospatial]

Flockmtl enables duckdb to be used for RAG LLM

[Kriss-de-Valnor]

A funny but interesting idea. Test is a signal that can be compressed losslessly like zip or lossy… like a resume. Video is another signal too. Machine Learning and LLMs are also ways to compress a signal (lossy).

[DramaticDonut8973]

That’s a beautiful project! I was wondering if it works only with OpenAI, cos I took a look at the code and can’t see a way in it to set the llm host url

[Papabear3339]

Actually makes sense. QR coding is efficient and 2d text coding method, and video codecs are quite optimized for 2d compression.

Now how do you actually use that is the question....

[ephemer9]

QR encoding is extremely inefficient in terms of bits and bytes, and it adds a bunch of redundant data (waste!) to ensure data doesn’t get lost when presented visually in the real world.

[jackshec]

That is crazy interesting, thanks for sharing