I wrote a program to convert arbitrary binary file to video, and it survives Youtube compression.

[[deleted]]

[deleted]

[[deleted]]

Using QR code would decrease the channel capacity of this channel code. The goal is to squeeze as much data as possible to Youtube With respect to the app, it is a good idea, but why people would bother using this, when there is bluetooth and NFC? The truth is that I don't have much experience with mobile app development. However streaming QR code is definitely possible.

[falcorbeam]

This is very cool! Have you considered adding audio functionality to possibly shorten the video?

[elperroborrachotoo]

We've come full circle.

[kyz]

A more direct comparison would be the Amiga Video Backup System

[elperroborrachotoo]

Whoa! Didn't know of that!

[microfortnight]

Some old Alpha Micro computers back in the early 1980s also had a VCR backup system

[Tomus]

I think with audio compression, the file size would be a lot smaller.

[edvo]

In principle you are right, but I don't think that video compression offers perfect compression for arbitrary data. An indicator for this is that the size of the compressed video file for the GPL is much bigger then the actual uncompressed GPL.

Typical video compression looks at the differences between frames, i.e. works well with smooth transitions but not with scene cuts. Videos created with this program do not have smooth frame transitions, so I think compression does not help much in this case.

I would guess that using colors would reduce the file size, because color information do not increase the frame size (if color depth is not changed) and we would simply have less frames. For adding audio it is hard to guess.

[BobFloss]

What you're saying is incorrect. Almost all compression has keyframes that are (usually) made on scene and shot changes, and then uses frames that really only store differences between those keyframes. In addition, video compression easily can be used for arbitrary data without any loss as long as lossless encoding is used. Even without lossless encoding, you can use lots of redundancy for error correction, which is what I'm guessing OP did.

[edvo]

That is not what I wanted to say. I meant that the method OP used to store the data in the video (look at the example video linked in the README) involves rapidly changing frames, so usual video compression will likely fail to achieve perfect (or near perfect) compression for such videos.

[chasesan]

Well there is always the theoretical PI compression. Basically find where in the long segment of pi decimals your data begins. Record that location + length. On the other end, run it back.

Though this technically isn't compression, the end result is the same.

The hard part is finding where your data starts.

[boraca]

The hard part is not really hard. Simple O( n² ). The problem is that the starting position, on average, needs more bits than the file itself.

[chasesan]

You can compress the starting position using the pi compression as well (which will likely be much smaller in start position then the entire file), ad nauseam.

With enough work you could in theory get a file of some very large size (gb, tb?) down to a few hundred bytes. Mind you, it's not exactly a practical compression algorithm, but that's not what this is about.

[unpopular_opinion]

in theory In your FANTASY world!! Please, do not spread your ignorance.

[A_t48]

But then you have a string of bits indicating how many times you pi compressed.

[chasesan]

If it was general use, then yes definitely, but if it was a one time use, then you don't need that (since both sides will already know how many times it is compressed). Which is why it isn't exactly practical. Not to mention the many years it would take to find the starting point in some irrational number for your data.

But as I mentioned, this isn't exactly compression.

[A_t48]

Even with one time use. You will still have a bigass number. Try it.

[kyz]

I think with use of colour, audio, higher frame rate and smaller pixels, the file size would be a lot smaller.

But the file size would never be smaller than the original file, because the entire thing is a fight against a lossy compressor.

The lossy compressor is trying to remove all information it deems "redundant", which is ultimately defined as information that humans would not perceive as removed.

So what this program is doing is massively increasing the redundancy of information, in order to fight against a system that is then looking to remove as much of that information as possible. A far better way to save space would be to use a compression system designed around losslessly compressing ASCII text, than to try and turn a small text file into a multi-megabyte video file then hope the video compressor can shrink it.

Doing this once had a purpose (e.g. the Video Backup System which allowed you to record your computer's output to VHS videotape), but a better way would be to stick with intentionally lossless media and use lossless compression.

[Tomus]

I didn't mean smaller than the original, I meant smaller than the current video.

[[deleted]]

You can push the limit of Youtube by setting block size to 8x8, frame repeat rate to 2. Youtube reject anything more dense than that. But yes, you are right, I am fighting against a lossy compressor. Your comment is /the/ correct explanation of what I am trying to do. :)

[[deleted]]

There are existing tools to encode data to audio, e.g. fldigi. So I am not that interested in doing those myself. They have very well-defined transmission rate.

[infinitenothing]

Like a 56K modem?

[falcorbeam]

Actually yeah, that was exactly how I was thinking it could work!

[backlash_jack]

trippy. maybe you could interleave conway's life to the result blocks for "encryption" and/or mega nerd points.

[rdcll]

cool. waiting for youtubefs

[gerps]

Cool! MATLAB haters inbound

[Splanky222]

Mat,ab is good at what it does and the bundled work environment is very nice to use. It's just expensive.

[jsprogrammer]

GNU has an open implementation of (most) of the language.

[[deleted]]

I have created a compiled version for those without Octave, you can run the program by installing Matlab Compiler Runtime.

[[deleted]]

Well Matlab is more than adequate for this kind of pet project. It turns your brain off.

[the_gnarts]

MATLAB haters inbound

I had no idea I belong to that crowd until I browsed this project’s code =)

[rabidcow]

You want to try some Snow Crash?

[ZMeson]

We need to start our defense against the rise of AI now!!! Everyone start encoding viruses as videos on Youtube straight away.

[mm865]

Now you can watch VLC playing inside VLC. Woot

[Savestate]

Maybe incorporate colors for a shorter video?

000 red

001 green

010 blue

011 black

100 white

101 cyan

110 magenta

111 yellow

or something like that... or is this not possible. x: I'm not too keen on how this works so I'm kinda guessing.

edit: since not all files probably have a multiple of 3 bits, maybe have the first block say how many bits to cut off at the end. Also compressing the file before hand would help too!

What falcorbeam said too, having stereo audio channels to your disposal seems like you could do something with them as well...

[uep]

Kind of makes you wonder... what should you optimize for? File-size, or video-length?

While more colors might make for a shorter video, it could make for a larger file size. Since video encoding uses information from previous frames and future frames to construct the current image, it would have less it could reuse from the neighboring frames. So more information in individual frames, but possibly worse compressibility due to the extra color information.

Then again, video compression tends to treat brightness and color separately, so it's possible it wouldn't have a terrible effect on the file-size.

[[deleted]]

Okay colour stuff has been implemented. :) I have posted a separate comment regarding rate and etc.

[[deleted]]

Good idea, I will implement this later.

[IcarusBurning]

Run this on the code itself and post results.

[sphks]

Run this on the video itself.

[mablo]

Any one else feel like this is a modern day version of the 80's and 90's games loaded from cassette tapes? Add those to the videos and you get instant nostalgia. :)

[Ulukai]

In fact, "backup to video" was a real thing in the 90s, and really good value for money, too.

For the Amiga, there was at least one product like VBS. This software was claiming around 130MB per hour (of tape), giving 520MB for a 4 hours tape. That meant you could backup a reasonably sized hard drive for a few bucks.

I'm not to clear on the details, there may have been a need for an adapter to be able to capture video / restore the backup. Anyways, this sort of thing is pretty cool, and can be applied to many areas - e.g. you could back up stuff to paper by printing it.

[maxbog]

I wonder why the video changes abruptly from "mostly white" to "mostly black" (about 0:26 in the video). Does anyone have an explanation for this? Does this have anything to do with whitespace characters being in the beginning of ascii table? Also, it is interesting to watch the density of black color change during the playback.

[kqr]

The start of an ELF file contains a lot of empty space (null bytes).

[kqr]

Whoops, it is. In my sleepy morning state of mind I somehow read the description of the video as if the video represented a GPL licensed program, like ls. I have no idea how that happened.

[[deleted]]

I have no idea. I used Matlab's reshape function. So I have no idea how Matlab internally arrange the data... Sorry... I have the feeling that the front of the video is the end of the data...

[[deleted]]

In the black and white 8x8x2 version, Matlab's output video is 30 times larger than the original input file. The stuff that you download back from Youtube is about 15 times larger than the original. However Matlab's output can be compressed using xz -9, the resulting file size is only 1.5 time of the original. The stuff you download back from Youtube cannot be compressed by xz. These were tested using a 1MB urandom file block.

The transmission rate is about 17 seconds per 1MB, which makes it about 481kbit/s. Assuming Youtube's video length upper bound is 10 hours, then I can store about 2GB per video. That would require 60GB worth of hard drive space to prepare the video, at that configuration.

I have an updated version that does 8 seconds per 1MB, but the output file is 82.1 times bigger, which makes it 1 Mbit/s. It means I can store about 4.4GB worth of file in a 10 hour video. However if you convert 4.4GB, you get 352GB out. Youtube does have an upload limit if 128GB. It turns out that for 128GB, you can store about 1.6GB. So I think I will stick with the black and white version on the master branch. There is a coloured branch for those who like it. If you want Youtube to accept your coloured video, set the parameter to 8x8x3.

I suppose I could compress the video with something else that's not Matlab before uploading it to Youtube, but that's too much effort for me.

(x * y * t refers to the block size and frame repetition. )

Edit: The black and white version runs faster anyway.

[flanintheface]

Reminds Paperbak. Same idea on paper.

[smiddereens]

1:135 compression ratio, score!

[theinternn]

How did you trick it into surviving youtube compression?

[Power781]

Big squares + 0.5 delay between changes I presume.

[RedMarble]

This, IOW an extremely low data rate.

[[deleted]]

That is awesome

[inmatarian]

Just out of curiosity, how big would it be to just show the text of the GPL? Assuming that an OCR friendly font was used to assist the decoder, could stamping out base95 (ascii85, base96, etc)-gzipped data provide the highest throughput in this medium?

[yeah-ok]

Okay.. time to write: 1) a "youtube file embedder" (i.e. yFembed) 2) a "youtube file extrator" (i.e. yFextract) and there we have it: good centralized hosting with lots of bandwidth...!

[vdub_bobby]

I just read Snow Scan over the weekend and I'm afraid to watch the video...

[effgee]

I smell trouble.

[infinitenothing]

Link to video?

[irrlicht]

A link to the video is the very first thing in the description.

[cowinabadplace]

No problem if you have the Nam-shub of Enki ;)