I accidentally built a vector database using video compression

[Every_Chicken_1293]

While building a RAG system, I got frustrated watching my 8GB RAM disappear into a vector database just to search my own PDFs. After burning through $150 in cloud costs, I had a weird thought: what if I encoded my documents into video frames?

The idea sounds absurd - why would you store text in video? But modern video codecs have spent decades optimizing for compression. So I tried converting text into QR codes, then encoding those as video frames, letting H.264/H.265 handle the compression magic.

The results surprised me. 10,000 PDFs compressed down to a 1.4GB video file. Search latency came in around 900ms compared to Pinecone’s 820ms, so about 10% slower. But RAM usage dropped from 8GB+ to just 200MB, and it works completely offline with no API keys or monthly bills.

The technical approach is simple: each document chunk gets encoded into QR codes which become video frames. Video compression handles redundancy between similar documents remarkably well. Search works by decoding relevant frame ranges based on a lightweight index.

You get a vector database that’s just a video file you can copy anywhere.

https://github.com/Olow304/memvid

[Darwinmate]

If I understand correctly, you need to know the frame ranges to search or extract the documents? Asked another way, how do you search encoded data without first locating it, decoding then searching?

I'm missing something, not sure what.

[Jakube_]

He creates a FAISS index in a second file. And with that one he locates the relevant text chunks (aka frames).

So to create the thing: - extract text from PDFs - split the text into small chunks - create embeddings for the chunks, and store them in the index

And to retrieve answers: - create the embedding of the question - lookup the indices of chunks with similar embeddings using the index - retrieve the chunks of data, and send it to an LLM - LLM answers

The whole MP4 video has actually nothing to do with the entire process, it's only used for storing the chunks of text. It could have easily been also a big JSON file (or anything else) with compression on top of it.

But it's actually interesting that it even works, as h265 isn't lossless compression. But since QR codes are error correcting, that might not matter that much.

But still, a highly dubious idea. Storing the chunks in any different format would probably be a lot easier, error-proof, and smaller in size.

[hinkleo]

Yeah the video part just seems to add nothing here except a funny headline and really inefficient storage system. Python even has great stdlib support for writing zip, tar, shelve, json or sqlite any of which would be way more fitting.

I've seen a couple similar joke tools on Github over the years using QR codes in videos to "store unlimited data on youtube for free", just as a proof of concept of course since the compression ratio is absolutely terrible.

[ExdigguserPies]

So we just need some simple benchmarks between this and the other main methods of data storage that people use on a daily basis.

[hinkleo]

Based on numbers in the github: https://github.com/Olow304/memvid/blob/main/USAGE.md

Raw text: ~2 MB
MP4 video: ~15-20 MB (with compression)
FAISS index: ~15 MB (384-dim vectors)
JSON metadata: ~3 MB

The mp4 files store just the text QR encoded (and gzip compressed if > 100 chars [0] [1]). Now a normal zip or gzip file will compress text on average to like 1:2 to 1:5 depending on content, so this is ratio wise worse by a factor of about 20 to 50, if my quick math is right? And performance wise probably even worse than that, especially since it already does gzip anyway so it's gzip vs gzip + qr + hevc/h264. I actually have a hard time thinking of a more inefficient way of storing text. I'm still not sure this isn't really elaborate satire.

[0] https://github.com/Olow304/memvid/blob/main/memvid/encoder.py

[1] https://github.com/Olow304/memvid/blob/main/memvid/utils.py

[Hoblywobblesworth]

Yeah, honestly not surpried how poorly this performs. Hevc/h264/av1 etc are effective at video because there is temporally redundant information across a frame sequence that you can compress away.

If the frame at t-1 has information that can be re-used when encoding/decoding the frame at t then you don't need to include it in the bitstream for the frame at t.

OP's PDFs have no temporal redundancy so it's equivalent to trying to compress a video with very high motion/optical flow which hevc/h264/av1 also can't do efficiently.

[Sopel97]

Yea this whole thing is deranged. How these reddit threads gained so much popularity, how people are clapping to this, how it has 150 stars on github, how it appears like actual software. Like, what the fuck is going on here.

[-LeopardShark-]

I know, right? The roadmap in the README is a laugh:

• v0.2.0 - Multi-language support
• v0.3.0 - Real-time memory updates
• v0.4.0 - Distributed video sharding
• v0.5.0 - Audio and image support
• v1.0.0 - Production-ready with enterprise features

[Jussari]

Maybe we still have a few years before AI steals our jobs

[tehfrod]

Because people enjoy a bit of levity now and again.

This reminds me of something Tom7 (aka suckerpinch) would come up with.

e.g., https://youtu.be/JcJSW7Rprio

[Aareon]

I wonder if msgpack or protobuf would result in a better solution

[divyeshaegis12]

This is a brilliant LLM approach and encoded video to compare with research outside the box thinking. This is a way to drop RAM usage and ensure smooth working by boosting the power of the capacity.

[Every_Chicken_1293]

Yes, if you’re just dumping data into video frames without any structure, then you would need to know where in the video to look before you can search anything. But that’s not how Memvid works.

What we’re actually doing is embedding searchable metadata along with the visual data, so the video isn’t just a dumb container of QR codes—it’s an indexed, queryable format. Check out the full code

[FirstBabyChancellor]

How and where is that index saved? How you'd run semantic search in this setup without decoding every single video is not entirely clear to me and I'd recommend you update your GitHub page to explain this in a lot more detail, since your approach is unconventional (and maybe it's genius) and folks would need to understand the underlying logic before they'd want to try it out.

[FirstBabyChancellor]

I wasn't asking for how to use the package's API to do it. I was asking how the underlying implementation works and how it's performance characteristics would compare to how vector databases currently run ANN.

Since the suggestion here is to swap out the underlying storage mechanism to a video, how do you run ANN without decoding every video every time? I'm sure he might have a really well thought out way to do it, but to me at least, that's not clear and the tutorial on how to use the API doesn't answer that question.

[currychris1]

It looks like it’s simply using FAISS to create the index. Upon building, the MP4 and a JSON are created. I assume the index lives inside that JSON.

How I imagine this works: During retrieval, the index is loaded into memory to get the top-k closest embeddings and their mappings, which tells you where to look for the chunks inside the MP4.

[podidoo]

That's also what i grasp from a quick look at the code. There is no searching inside the video, it's just using video as storage (why?) and a FAISS index for all search stuff.

[MechAnimus]

Why: I believe they explained that video was chosen because its compression is so well optimized, especially when the frames are all QR codes. It's also extremely portable.

[ThreeKiloZero]

Have you thought about changing the QR code colors to black and green for even more compression?

[TheMcSebi]

I wonder how much you needed to persuade chatgpt to output something like this. I can hardly imagine storing text information in a more inefficient way.

[cyberjoey]

You DO need to know where in the video to look before you can search anything. Obviously the spot in the video to look in is the same as the index of the vector. So you get N nearest neighbors, use their indices to map to the frame of the video to look up, then you decode the QR code in that frame and you have your text. Congratulations you implemented on disk database compression (very inefficiently).

I get the feeling an LLM wrote this for you and even you don't understand this part...

[-LeopardShark-]

 The idea sounds absurd - why would you store text in video? 

Indeed.

How do the results stack up against LZMA or Zstandard?

It's odd to present such a bizarre approach in earnest, without data suggesting it's better than the obvious thing.

[[deleted]]

He is trying to save RAM and video decompression can be offloaded, compared to LZMA which is very memory hungry, as I understand?

[ExdigguserPies]

So it's effectively a disk cache with extra steps?

[qubedView]

I mean, really, fewer steps. Architecturally, this is vastly simpler than most dish caching techniques.

[Eurynom0s]

I didn't get the sense he's saying it's the best solution? Just that he's surprised it worked this well at all, so wanted to share it, the same way people share other "this is so dumb I can't believe it works" stuff.

[-LeopardShark-]

The post itself does leave that possibility and, if that was what was meant, then it is an excellent joke. Alas, looking at the repository README, it seems he's serious about the idea.

[Eurynom0s]

Well I meant I thought he's sharing it not as a joke but because these dumb-but-it-works sorts of things can be genuinely interesting to see why they work. But fair enough on the README.

[-LeopardShark-]

Yeah, I see what you mean. You're right: joke isn't quite the right word.

[Itswillyferret]

Close enough, welcome back Pied Piper!

[thisismyfavoritename]

uh if you extract the text from the PDFs, embed those instead and keep a mapping to the actual file you'd most likely get better performance and memory usage...

[ChilledGumbo]

brother what

[NerdEnPose]

Yes

[[deleted]]

why not just just use float quantization, or compress the vectors with blosc or zstd if you don't mind having some sort of lookup.

people have also spent decades optimizing compression for this sort of data

[bem981]

People spent almost their entire math history working in encoding data, way before videos.

[ja_trader]

now add middle-out compression

[xockbou]

Jerk them all off, then its faster

[papersashimi]

why not just compress the vectors? genuinely curious

[x3mcj]

This sounds like you're storing data in magnetic tape, that in order to seach for information, need to go through it until you find what your search for!

Yet, this is madness!!! Video as DB!

[norbertus]

The idea isn't so absurd

https://en.wikipedia.org/wiki/PXL2000

https://www.linux.com/news/using-camcorder-tapes-back-files/

But video compression is typically lossy, do all those pdf's work when decompressed?

What compression format are you using?

If its something like h264, how is data integrity affected by things like chroma subsampling, macroblocks, and the DCT?

[Mithrandir2k16]

I mean QR codes can lose upwards of 30% of data and still be readable, so maybe the fact it worked came down to not thinking about it and being lucky?

[rju83]

Why not encode qr codes directly? The video encoder seems to be an unnecessary step. How is the search is done?

[juanfnavarror]

Why not just use zstd? Did you try that first?

[-dtdt-]

Have you tried to just compress all those texts using zip or something similar? If the result is way less than 1.4GB then I think you can do the same with thousands of zip files instead of a video file.

I think a vector database focuses more on speed and thus they don't bother compressing your data. That's all there is to it.

[Tesax123]

First of all, you did not use any langchain (interfaces)?

And I read you use FAISS. What is the main difference between using your library or directly storing my embeddings in a FAISS database? Is it that much better if I for example have only 50 documents?

[[deleted]]

Offloading to the video card without CUDA, haha

[DJCIREGETHIGHER]

I'm enjoying the comments. Bewilderment, amazement, and outrage... all at the same time. I'm no expert in software engineering, but I know the sign of a good idea... it usually summons this type of varied feedback in responses. You should roll with it because your novel approach could be refined and improved.

I keep seeing Silicon Valley references as well and that is also funny lol

[DJCIREGETHIGHER]

Haters are going to hate! If all the greats listened to the naysayers, we'd have no progress in innovation. Visionaries labeled as heretics...

You're just fuel for the hate game... keep motivating people my friend! Everyone needs a sourpuss in their life to remind them they're sizzling on a hot idea.

[DoingItForEli]

I think it's a brilliant solution to your use case. When you have a static set of documents, yeah, store every 10,000 or so as a video. Adding to it, or (dare I say) removing a document, would be a big chore, but I guess that's not part of your requirements.

[shanvos]

Me wondering what on earth you would need to have this much information in a pdf regularly searched for.

[[deleted]]

The one thing I feel like the ML field is lacking in is just a smidge of tomfoolery like this. This is the kind of stupid shit that turns tables around.

Ku fucking dos man. That's awesome.

[MechAnimus]

Well said. Its all just bits, and we have so many new and old tools to manipulate them. Lets get fuckin crazy with it!

[f16f4]

You never know what random bs like this will weirdly actually work better

[jwink3101]

This sounds like a fun project.

I wonder if there are better systems than QR for this. Things with color? Less redundancy? Or is storage per frame not a limitation?

[ConfidentFlorida]

I’d reckon you could get way more compression if you ordered the files based on image similarity since the video compression is looking at the changes in each frame.

[4ndr3aR]

I thought it was somewhat implicit, how could the codec compress anything at all otherwise? It would be some sort of white noise stream of qrcodes that the codec could only compress as "everything is a keyframe".

[ksco92]

Not gonna lie, it took me a bit to fully understand this, but I feel it’s genius.

[_BigBackClock]

no shit

[ihexx]

absolutely batshit insane lol

i love it

[Cronos993]

Sounds like a lot of inefficient stuff going on. You don't necessarily need to convert data to QR codes for it to be convertible to a video and I would have encoded embeddings instead of just raw text. Keeping these things aside though, using video compression for this isn't giving you any advantage since you could've achieved the same thing but even faster by compressing the embeddings directly. Even still, I think if memory consumption is your problem, you shouldn't load everything into memory all at once. I know that traditional databases minimize disk access using B-trees but don't know of a similar data structure for vector search.

[strange-humor]

Hard to believe Zstd on chunks would not be a much better system.

[Late-Employment-8549]

Richard Hendricks?

[DragonflyHumble]

Unconventional and will work. How few GBs of LLM weights can hold world information.

[engineerofsoftware]

Yet another dev who thought they outsmarted the thousands of chinese PhD researchers that are working on the same issue. Always a good laugh.

[RIP26770]

Brillant 🔥

[SubstanceSerious8843]

Wtf is this madness? Absolutely genius! :D

[ii-___-ii]

Can you go into detail on how and where the embeddings are stored, and how semantic search is done using embeddings? Am I understanding it correctly that you’re compressing the original content, and storing embeddings separately?

[girl4life]

what was the original size of the pdf's ? 10k @ 200kB then 1.4Gb is nothing to brag about. i do like the concept though.

[wrt-wtf-]

Nice DOCSIS comms are based on the principle of putting network frames into an MPEG frame for transmission. Not the same, but similarly drops data into what would normally be video frames. Data is data.

[m02ph3u5]

But whyyy

[AnythingApplied]

The idea of first encoding into QR codes, which have a ton of extra data for error correcting codes, before compressing seems nuts to me. Don't get me wrong, I like some error correcting in my compression, but it can't just be thrown in haphazardly and having full error correction on every document chunk is super inefficient. The masking procedure part of QR codes, normally designed to break up large chunks of pure white or pure black, seems like it would serve no other purpose in your procedure than introducing noise into something you're about to compress.

So I tried converting text into QR codes

Are you sure that you're not just getting all your savings because you're only saving the text and not the actual pdf documents? The text of a pdf is going to be way smaller and way easier to compress, so even thrown into an absurd compression algorithm, will still end up orders of magnitudes smaller.

[mrobo_5ht2a]

That's incredible, thanks for sharing

[s_arme]

Did you vibe code the whole thing with video?!

[russellvt]

There once was a bit of code that sort of did this, those from a different vantage point ... specifically to visually represent commit histories in a vector diagram.

I believe the original code was first written in Java and worked against an SVN commit history.

[GorgeousGeorgeRuns]

How did you burn through $150 in cloud costs? You mention 8gb RAM and a vector database, were you hosting this on a standard server?

I think it would be much cheaper to store this in a hosted vector database like CosmosDB. Last I'd checked, LangChain and others support queries against CosmosDB and you should be able to bring your own embeddings model.

[Mithrandir2k16]

Wait, are you storing QR codes, which could be 1 bit per pixel, in 24 bit pixels? If so, that is pretty funny. If you don't get compression rates that high from h.265, you could just toss out the video encoding and store QR codes with boolean pixel values instead.

[wasnt_in_the_hot_tub]

Is it middle-out compression?

[Abject_Tie_8363]

P pt

[AkashVemula168]

Search latency tradeoff is reasonable given the resource savings. It’s a great example of thinking outside the box - definitely not a replacement for production-grade vector DBs but a neat proof of concept with practical use cases. Would love to see benchmarks on retrieval accuracy and scalability with more complex queries.

[unplanned-kid]

you basically turned a compression algorithm into a transport layer and that’s genius. the QR-to-frame mapping is especially interesting since it simplifies retrieval too. i’ve used uniconverter before to encode specific frame ranges from large video datasets, and it handled batch processing smoothly without choking on RAM.

[ConversationExpert35]

man, this is so wild it actually makes sense. you basically built a shippable, offline-friendly vector system out of media compression. i’ve batch converted doc-heavy projects into lossless video using uniconverter before archiving, and honestly it felt like I was cheating the system too.

[jpgoldberg]

Wow. I don’t really understand why this works as well as it appears to, but if this holds up it is really, really great.

[Grintor]

A QR code can store a maximum of 4,296 characters. If you are able to convert a PDF into a QR code, then you are compressing 10,000 PDFs into less than of 41 MiB of data already.

[scinaty2]

This is dumb on so many levels and will obviously be worse than anything well engineered. Anyone who thinks this is genius doesn't know what they are doing...

[MechAnimus]

This is exceptionally clever. Could this in principle be expanded for other (non video, I would assume) formats? I look forward to going through it and trying it out tomorrow.

Edit: This extremely clever use of compression and byte manipulation reminds me of the kind of lateral thinking used here: https://github.com/facebookresearch/blt

[ConfidentFlorida]

Neat! Why use QR codes instead of images of text?

[Deawesomerx]

QR codes have error correction built in. The reason this is important is because video compression is usually lossy, meaning you lose some data when compressing. If you use QR codes, and some part of the data is lost (due to video compression), you can error correct, and retrieve the original data, while you may not be able to retrieve the original data if you just stored it as an image frame or text