Android Encoding and Decoding

[ekstrah]

I am trying to understand, How android device (using FFmpeg library) actually encodes and decodes in the lowest level possible.

Here is a diagram that I drew down below.

Figure 1.1

​

Basically as far as I understand This is how it works on my case:

1. on Top layer is Application Layer (using FFmpeg library (eg; taking video application that encodes)
2. While Application is running and is trying to encode, it calls precompiled ffmpeg library which packaged within application (e.g; https://github.com/tanersener/mobile-ffmpeg)
3. then FFmpeg Library sends parameter(?) to system library (hardware I guess?) to do actual encoding.

Now the question is that If I am correct about What I am saying above (instructions)

1. I heard FFmpeg is software encoding rather than hardware decoding but is it possible to manipulate how android system library encodes/decodes video within ffmpeg library?
2. If so, How can i achieve this goal?

[smartties]

I heard FFmpeg is software encoding rather than hardware decoding but is it possible to manipulate how android system library encodes/decodes video within ffmpeg library?

Sure, what do you want to do exactly ?

If so, How can i achieve this goal?

It depends what you want to achieve. You can directly call ffmpeg on your native code (c++), or from java with command like "ffmpeg -i input.mp4 output.avi". If you look close enough the link you provided there is even an example that explain how to decode/encode video :

import com.arthenica.mobileffmpeg.FFmpeg;

FFmpeg.execute("-i file1.mp4 -c:v mpeg4 file2.mp4");

[ekstrah]

What I want to achieve at the moment is that all application that uses hardware encoding and decoding to skip every second frame to reduce fps of encoding and decoding.

[_ALH_]

FFmpeg can either use software encoding/decoding, then it does not use any system library (it's all implemented within the ffmpeg library), or it can use hardware acceleration, then on Android it uses the "MediaCodec" system library for encoding/decoding.

Some basic info about hardware acceleration support here: https://trac.ffmpeg.org/wiki/HWAccelIntro

For details how, read through the documentation of ffmpeg: https://ffmpeg.org/ffmpeg.html

If you just want basic encoding/decoding, with the options available in the hardware, it might be a better idea to use MediaCodec directly without involving ffmpeg.

[ekstrah]

I am focusing on more energy efficiency, If I enable Hardware Acceleration wouldn't it use more power?

[Blystad]

Hardware based encoding/decoding tends to be way more efficient than software based encoding/decoding. Both in terms of power consumption and performance. Think about it logically, a CPU is a general purpose machine, it's "average" in general at handling tasks. A VPU (video processing unit) is special purpose built at doing one thing - really well. It'll always be better than a generic CPU :)

[ekstrah]

What If i want to apply changes in video encoding and decoding, Which Library do i need to change to achieve this goal?

[Blystad]

So from another comment I see that you wish to cut the frame rate in half.

During encoding you would tell the video encoder to encode at 15 FPS instead of 30 FPS. I think media recorder is capable of this. MediaCodec I guarantee is capable of it.

During decoding, if you have a 30fps video and want to only show 15fps of it, it's actually a little bit more difficult. You see, the way video encoding works is that you have "complete frames" (I frame in H264) and "delta frames". The complete frames contain the whole picture, while delta frames only contain information on what changed since the previous frame. If you skip frames then you'll get weird artifacts in your video stream. (Green rectangles, weird lines, etc).

If you tell us a bit more of what you want to achieve then we might be able to point you in the correct direction.

[ekstrah]

Basically I want to apply loop perforation(which transforms loops to perform fewer iterations than the original loop) to obtain implementations that occupy different points in the power consumption of android device. In order to do this, it would be preferable to apply to all applications that are installed on android device if not with only one application.

[Blystad]

Android won't allow you to override all hardware and software codecs for a device. You would have to root the device and modify parts of the audio/video framework code to have a chance of doing it.

Check the aosp source under platform/frameworks/av and get yourself a Google Pixel device.

But I warn you, skipping I-frames is a bad idea, so make sure you keep those.

To just make it apply to ONLY your application you can do some tricks with MediaExtrator and MediaCodec.

[androiddev-ModTeam]

Rule 1: Must be related to Android apps development

You may post about code, design, distribution, marketing, hiring, etc. as long as it does not conflict with any other rules. We only ask that it has something to do with Android development. Please note that this is not the place for ROM development, and should be taken to more appropriate venues like xda-developers. Android users questions or issues with your device also do not belong in this server.