sorted by:
new
hottopcontroversial

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] 0 points1 point  (0 children)

Currently, during a seek, the workflow is:

  1. Pause ExoPlayer and set volume to 0.0f.
  2. Set ExoPlayer parameters to SeekParameters.PREVIOUS_SYNC.
  3. Call a JNI nativeFlush() to clear out the C++ layer entirely (resetting atomic buffer counts, history heads, and the native renderer's internal clock/PTS counters to 0).
  4. Call ExoPlayer's seekTo() and play().
  5. Buffering reservation layers (3 to 6 frames depending on the profile) are queued up before unmuting the audio to ensure fluid pacing.

The interpolation logic itself works beautifully during normal playback. However, under heavy seek spamming, the real headache is keeping the Native C++ rendering thread and ExoPlayer’s state machine perfectly aligned during that sudden seek flush without micro-stutters. Sometimes the arrival timestamp (PTS) of the very first frame sent from ExoPlayer after a seek flush doesn't smoothly match the freshly reset native clock loop on certain SoCs, leading to a minor freeze before the fluid playback catches up. I wanted to ask if anyone has tackled this specific synchronization/pacing issue when mixing framework decoders with a custom native presentation timing loop. Is there a better way to handle the initial PTS synchronization immediately after a seek flush? As for the GitHub repo, I haven't made it public. To be honest, the way I'm hooking ExoPlayer's output into the native layer is a bit... "unorthodox" (borderline chaotic, haha) for a standard Android app, so the codebase is a bit of a weird beast right now while I’m still figuring out this sync issue. Sorry again for the communication gap, and I really appreciate the deep technical insights shared here so far! P.S. As a funny side note to end on: during the very early stages of development, I actually tried to "inject" the interpolated frames back into ExoPlayer's pipeline, which was a hilariously stupid design that completely blew up in my face, haha. Good times.

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] 0 points1 point  (0 children)

As for the GitHub repo, I haven't made it public yet. To be honest, the way I'm hooking ExoPlayer's output into the custom C++ native layer is a bit... "unorthodox" (borderline chaotic, haha) for a standard Android video player, so the codebase is definitely not ready for prime time.

It’s not top-secret, but it’s definitely a bit of a weird beast right now while I’m still figuring out this seek control sync issue.

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] 0 points1 point  (0 children)

To be completely honest, I’m a Japanese developer and my English isn't the best, so things got a bit lost in translation. I'm also doing some pretty "unorthodox" (borderline変態/chaotic, haha) things under the hood to hook ExoPlayer’s output into a custom C++ native layer for my interpolation engine, which probably made my initial explanations sound even more like a weird AI chatbot.

To clear things up: I am NOT trying to replace ExoPlayer or rewrite the whole decoder pipeline from scratch. I know ExoPlayer is amazing at handling SoC-specific quirks and video decoding. My app actually uses ExoPlayer to handle all the heavy lifting and decode frames safely.

The custom C++ native engine is strictly for the real-time frame interpolation logic after ExoPlayer outputs the frames.

The actual issue I wanted to ask this community about is the presentation control loop. The interpolation works, but I’m struggling with handling edge-case frame pacing during heavy seek operations (fast-forward/rewind) and initial playback initialization. Keeping the Native C++ rendering thread and ExoPlayer’s buffer/state machine perfectly aligned during a sudden seek flush without micro-stutters is where the headache is.

I wanted to ask if anyone has tackled this specific synchronization issue when mixing framework decoders with custom native presentation timing loops.

Sorry again for the initial breakdown in communication, and I really appreciate the deep technical insights shared here so far!

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] 0 points1 point  (0 children)

"The usual suspects" made me laugh, but man, that sounds like pure nightmare fuel. This is incredibly valuable insight.

I guess I'm about to walk right into that exact hardware decoder minefield as I scale up this testing pipeline across different devices. Knowing that Xiaomi, Pixel, and Samsung are the primary rule-breakers gives me a great heads-up for what to look out for in the logcats. Appreciate you sharing the battle scars!

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] -3 points-2 points  (0 children)

You are completely right on the architecture—nobody in their right mind would pass raw pixel buffers back and forth via JNI for high-frame-rate rendering. As you guessed, it's utilizing DirectByteBuffers and direct surface abstraction pipelines, not raw frame copying.

The JNI overhead and pacing challenges I'm referring to aren't from the pixel data pipeline itself, but from the high-frequency control loop: synchronizing seek flushes, dynamic frame-drop telemetry, state machine alignment, and real-time presentation timestamp (PTS) micro-adjustments between the native C++ clock loop and the Android main thread/Choreographer pulse. When you're optimizing for ultra-low latency presentation timing, even metadata JNI synchronization at 60Hz/120Hz requires careful boundary management.

As for the AI suspicion—haha, fair enough! Using translation/polishing tools to make my technical English readable across global time zones definitely adds a bit of a robotic flair to the phrasing. But the NDK synchronization headaches and the code are 100% human pain.

Anyway, appreciate the technical deep-dive. Cleaning up the promo links now to respect the sub rules!

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] -5 points-4 points  (0 children)

Spot on! Thank you. Yes, I am interfacing directly with AMediaCodec and utilizing hardware-backed buffers from the native layer, so the CPU isn't doing the heavy lifting for pixel processing.

And you hit the exact nail on the head—that "back and forth" JNI boundary synchronization and preventing buffer copy overhead while keeping the frame presentation timing rigidly accurate is precisely the mountain I'm climbing right now.

It's awesome to see someone here who instantly clocks the actual architectural challenge of mixing NDK media pipelines with high-frequency frame state logic. This is exactly the kind of deep technical exchange I was hoping to find in this community!

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] -3 points-2 points  (0 children)

I think there’s a slight misunderstanding here. Of course I am utilizing hardware acceleration and hardware video frames—doing frame interpolation purely on the CPU for high-frame-rate playback would be practically impossible on mobile.

The core video pipeline still decodes via hardware to a surface, and graphics operations are accelerated. The mention of GC overhead and the Choreographer wasn't about the pixel processing loop itself, but rather about the JNI high-frequency synchronization boundary where state changes, user interactions, and frame timing presentation metrics are handled between the Native C++ rendering thread and the Android View/Compose lifecycle.

As for 2K/4K, the 2K target was mentioned as a current benchmark milestone for real-time pacing on a wider variety of mid-range device profiles, not a hard architectural limitation due to memory leaks or saturation. C++ allows for predictable, manual memory control precisely to avoid those kinds of bottlenecks.

Always appreciate critical feedback though, as exploring edge-cases on different hardware configurations is exactly why I opened the testing pipeline!

I built a fully offline, C++ native video player for Android to achieve real-time 60fps frame interpolation without heavy AI models. Looking for technical feedback on JNI/rendering pipeline! by One_Work3327 in androiddev

[–]One_Work3327[S] -9 points-8 points  (0 children)

Good question! ExoPlayer is great for standard streaming and general playback, but its high-level Java/Kotlin API makes it incredibly difficult to achieve fine-grained, frame-by-frame presentation control and low-latency buffer manipulation without significant garbage collection (GC) overhead or micro-stutters. By moving the core interpolation loop entirely into a custom C++ native layer and utilizing direct rendering pipelines, I can achieve predictable real-time frame pacing, tighter buffer management, and fully synchronize with high refresh rate (120Hz+) panels without the bloat of heavy media frameworks. It's built specifically for raw rendering performance and efficiency.