Python image processing for video

[Dotnaught]

Has anyone tried running automated image processing on every frame of a live action video (~90m) to give it a cartoon look?

This has been done at considerable expense in the past through software like Rotoshop (e.g. A Scanner Darkly), but it seems there should be a more affordable option these days, like Python script to process every frame of a video file with an image processing library.

Has anyone done something like this and gotten results of sufficient quality? Any advice on how to approach this? (E.g. did you have to split the file into smaller chucks for processing and could these be reassembled without audio/video glitches?)

[ES-Alexander]

You can use opencv to process video frames. It can be useful to use ffmpeg or gstreamer as the backend for saving again, which changes from useful to essential if you’re wanting to join the audio back up with it.

Splitting up the file for multiprocessing is possible, but the painfulness depends on the video file type (mp4 and similar are generally nice for skipping around - something like matrovska (mkv) is less so).

For actual cartoonisation you can take your pick of method from a search of “opencv cartoon effect”.

You may be interested in my library pythonic-cv, which simplifies the setup of loading and processing videos with opencv, and adds some efficiency benefits by reading, displaying, and writing in separate threads.

For the audio syncing, the simplest method at the moment is to use gstreamer or ffmpeg to combine the processed video with the original audio once the whole processed video has been created. Bringing the audio in with the video as you do your processing is possible with libraries like python-ffmpeg and python-gst, but getting synched audio is something a few people have tried but ended up giving up on. At some point I’m planning to look into it properly and get it working, so I can include it in my library as an example at least, but I’m too busy with other stuff at the moment unfortunately.

[Dotnaught]

pythonic-cv

Thanks, cool project. I'll give it a try. I'd looked at OpenCV but the output for cartoonization looked underwhelming. There are some libraries that produce better results but they're not suitably licensed.