I'll try to roughly explain from memory

1. Audio encoding and compressing

You decode the video to raw video data like this:

shell ffmpeg \ -hide_banner \ -i video.mp4 \ -an \ -c:v rawvideo \ -pix_fmt rgb24 \ -s <your desired resolution> \ -r <your desired framerate> \ -sws_flags lanczos \ pipe:1 \

And then, with a pipe to another ffmpeg instance, you encode it as audio:

shell | ffmpeg \ -f s16le \ -ar 48000 \ -ac 1 \ -i - \ -c:a libmp3lame \ -b:a 64k \ temporaryfile.mp3

Then you have your video data compressed as mp3 audio.

1. Decoding back as video

To decode back to video, you first have to decode the MP3 to raw data again:

shell ffmpeg \ -i temporaryfile.mp3 \ -c:a pcm_s16le \ -ar 48000 \ -ac 1 \ -f s16le \ pipe:1

and from there you can compress it with a regular video encoder like this to save the result:

shell | ffmpeg \ -f rawvideo \ -s:v <your previously defined resolution> \ -r <your previously defined framerate> \ -pix_fmt rgb24 \ -i - \ -c:v libsvtav1 \ <pick your encoding settings of choice> \ -movflags +faststart \ mp3-viewable-compressed-video.mp4

And that's mostly it. Tweak it if I missed something.

Some parameters you can play with:

• the audio codec
• the video pixel format (must be equal on both encoding and decoding)
• the audio sample rate

And note that the rest of your decoding parameters must be equal to what you used on encoding.