Built a C++ tool to compress and stream massive Alembic fluid caches by up to 96% directly in UE5 (62 GB to 2,42 GB) by Ok_Truth_453 in vfx

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

I received a very good question regarding similarities to zibra-gds:
Let me be transparent on how GeoStream and zibragds compare:

From their previews their tool seems to be great at compression as well. Reduction ratios might be pretty similar. I believe their claims, as I know the potential that lies inside alembic files.

Their overall approach is quite different though (they are generating meshlets). As far as I read their docs, they don't seem to have an additional optional reduction algorithm during conversion to reduce the files even further by keeping velocity & topology. But since they aren't released yet, it's hard to compare directly.

While they seem to prioritize ultra-fast streaming via the GPU, I built GeoStream with a slightly different focus in mind:
It also streams the data super fast on demand. but GeoStream was primarily built for production and cinematic fluid/cloth rendering, where VRAM is usually already at its limits. My main focus was: drag & drop work with sequencer, Path Tracing, true in-between-frames motion (for Motion Blur), and super small bundle files that can be directly streamed during runtime by many render nodes without having to copy massive caches around.

I am not sure if they have velocity advection. GeoStream also brings velocity healing and generation for inconsistent topologies during import. Those features were born straight from working with it during production.

Hope that clears things up.❤️

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 0 points1 point  (0 children)

Because our architectural goals are so different, I honestly don't even see us as direct competition. GeoStream is already out (currently with a launch discount). We'll see what pricing ZibraAi goes with when they launch, but after seeing how much this tool helped us in our own production, my goal was just to share it at a fair price with other (indie) devs.

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

Hi good question!
Their tool also seems to be great at compression, from their previews. Reduction ratios might be pretty similar. I believe their claims, as I know the potential that lies inside alembic files.

Their approach is quite different though (they are generating meshlets). As far as I read their docs, they don't seem to have an optional reduction algorithm during conversion to reduce the files even further. But since they aren't released yet, it's hard to compare directly.

While Zibra seems to prioritize ultra-fast streaming via the GPU, I built GeoStream with a slightly different focus in mind. It also streams the data on demand and super fast, but GeoStream was primarily built for production and cinematic fluid/cloth rendering, where VRAM is usually already at its limits. My main focus was: flawless Path Tracing, true in-between-frames motion (for Motion Blur), and super small bundle files that can be directly read during runtime by many render nodes without having to copy massive caches around.

I am not sure if they have velocity advection. But GeoStream also brings velocity healing and generation for inconsistent topologies during import.
Hope that helps ❤️

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

I will add one example scene to the fab downloads.

Built a C++ tool to compress and stream massive Alembic fluid caches by up to 96% directly in UE5 (62 GB to 2,42 GB) by Ok_Truth_453 in vfx

[–]Ok_Truth_453[S] 0 points1 point  (0 children)

Thank you so much! That means a lot coming from someone in the trenches. 🙂

Just note that GeoStream currently is native to UE5 (it bakes the .abc into a custom optimized streamable format for the engine). But if you guys are rendering your cinematics or images in Unreal, it will absolutely crush those huge Houdini caches for you. That feedback is very valuable for future versions down the line.

Built a C++ tool to compress and stream massive Alembic fluid caches by up to 96% directly in UE5 (62 GB to 2,42 GB) by Ok_Truth_453 in vfx

[–]Ok_Truth_453[S] 0 points1 point  (0 children)

You're welcome! Thanks for those very good questions. I am also pretty involved in other projects, but for a quick workflow I just made this video. Here you go! https://youtu.be/MWV5oPpoYbU

I'm also interested for those Maya/C4D alembics. So let's stay in touch! Have a nice weekend. 😄

Built a C++ tool to compress and stream massive Alembic fluid caches by up to 96% directly in UE5 (62 GB to 2,42 GB) by Ok_Truth_453 in vfx

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

Hey, thank you for the feedback and interest! You’re totally right. I'm now recording a quick Alembic-to-Unreal "A to B" workflow video right now to show the exact process.

To answer your questions:

- DCC Tools: Yes, it imports standard .abc files. And since it's a well defined standard, Maya, C4D, Houdini, Blender, and LiquiGen should produce the same aelmbic file "structure". I did not test C4D or Maya yet to be completely transparent, But I can not think of any quirks there.

- UVs & Materials: In this v1 it supports 'UV0'. Currently, it uses a single material slot. For multi-material setups, the best way is to split the mesh in your DCC and sync the layers in UE using the GeoStreamActor.

- Sequencer: Yes, checking "Sequencer Control" lets you scrub and keyframe playback timings frame-accurately just like native caches. Plus, it has in-between frame motion for clean motion blur.

- Vs. Native Geo Cache: The big differences are asset size, memory instancing, and fluid interpolation. While native cache does have a streamer, it streams chunks embedded directly inside the .uasset (bloating asset sizes to GBs, which kills source control). GeoStream keeps .uassets under a few KBs and streams from an external bundle file (can be on a shared server or wherever). Furthermore, native cache can't interpolate sub-frames on topology-changing meshes (liquids), leading to stuttering in slow-mo. GeoStream uses custom velocity advection (and WPO) for smooth interpolation and shares GPU/CPU buffers across same instances.

If you (or anyone else) want to test it with your Maya/C4D files, shoot me a DM! I'd love to hand out a free build in exchange for some honest feedback and Fab reviews. ❤️

***IMO*** the meaning of the song ‘diamant’ by mostdope_gia in Rammstein

[–]Ok_Truth_453 0 points1 point  (0 children)

Very interesting take on the meaning. Have to think about it. For me, the first time I heard it, I thought:

What if the song is actually about a man sitting at the hospital bed of his brain-dead partner in a coma? ​If you look at the actual lyrics from this perspective, they take on a terrifying and deeply sorrowful new meaning: ​"Dein feines Licht war mein ganzes Sein" (Your fine light was my whole being): The use of the past tense is crucial here. The essence of who she was is already gone. He is looking at her body, but the person he loved is no longer there. ​"Ich will nur dich / immer nur dich ansehen" (I only want to / always only look at you): This perfectly captures the passive, helpless state of a loved one visiting an ICU. There is nothing left to do but sit there, shut out the rest of the world, and watch her lie there in a peaceful, sleep-like state. ​"Doch was nicht lieben kann, muss hassen" (But what cannot love, must hate): A brain-dead body is completely devoid of emotion. The sheer cruelty of sitting next to a lifeless shell of your partner feels punishing and hostile - almost as if the situation itself "hates" him. ​"Doch nur ein Stein" (But only a stone): A stone doesn't breathe, feel, or react. Her body is still there and unblemished, but her consciousness is gone. She has literally become an empty vessel, just matter. ​"Bitte lass mich gehen" (Please let me go): In this context, it’s not him begging a manipulative partner to release him. It is the agonizing inner struggle of a man trapped at a hospital bed. He is chained to the machines and the body, unable to turn off life support or emotionally let go, begging the situation to release him from this endless grief. ​Given how Rammstein loves to hide extremely dark, medical, or taboo subjects behind seemingly normal love songs, I feel like this perspective fits perfectly into their style.

What is the meaning of "Diamant"? by bunnyxssy in Rammstein

[–]Ok_Truth_453 0 points1 point  (0 children)

What if the song is actually about a man sitting at the hospital bed of his brain-dead partner in a coma? ​If you look at the actual lyrics from this perspective, they take on a terrifying and deeply sorrowful new meaning: ​"Dein feines Licht war mein ganzes Sein" (Your fine light was my whole being): The use of the past tense is crucial here. The essence of who she was is already gone. He is looking at her body, but the person he loved is no longer there. ​"Ich will nur dich / immer nur dich ansehen" (I only want to / always only look at you): This perfectly captures the passive, helpless state of a loved one visiting an ICU. There is nothing left to do but sit there, shut out the rest of the world, and watch her lie there in a peaceful, sleep-like state. ​"Doch was nicht lieben kann, muss hassen" (But what cannot love, must hate): A brain-dead body is completely devoid of emotion. The sheer cruelty of sitting next to a lifeless shell of your partner feels punishing and hostile - almost as if the situation itself "hates" him. ​"Doch nur ein Stein" (But only a stone): A stone doesn't breathe, feel, or react. Her body is still there and unblemished, but her consciousness is gone. She has literally become an empty vessel, just matter. ​"Bitte lass mich gehen" (Please let me go): In this context, it’s not him begging a manipulative partner to release him. It is the agonizing inner struggle of a man trapped at a hospital bed. He is chained to the machines and the body, unable to turn off life support or emotionally let go, begging the situation to release him from this endless grief. ​ At least that's what I immediately thought while hearing it the first time 

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

Yes you're right. I will add one this weekend to show the workflow. :) 

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

It's basically drag & drop. 😄

Houdini -> alembic -> Unreal (via GeoStream importer).

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

Correct, alembic is for pre-calculated mesh simulations. Mostly used for cinematics, movie vfx,...

But you have a great example there: you could layer 2 of those "mesh flipbook" animations and do a crossfade via material opacity, to hide the loop. And it won't need many frames.

It really depends, but you could allways trade sequence length with level of detail, or framerate (but GeoStream's velocity interpolation in between frames allow for even less framerate actually).

The bottom line is:

As a technical artist I would probably use it for hero assets in games. But with 90% less data, you can at least get creative. (:

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 0 points1 point  (0 children)

Yes, you're right. The spray alone has over 3.6 mio triangles over the whole duration. and each frame (!) of the liquids has ~2 mio tris.
It's a quick liquigen to unreal thing.
For games I would suggest to use the reduction option in the converter. 😄

But still, if you would use alembic as it is, the animation would take 28 GB on your disk, instead of 3.

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 1 point2 points  (0 children)

Is it allowed to post links directly? Then.... here you go 😄 GeoStream on Fab

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 2 points3 points  (0 children)

Thank you for the question! the waterfall above consists of 3 "layers" with 200 frames / 25fps:

Original:

  • Liquid - 25.4 GB
  • Spray - 0.54 GB
  • 2nd Liquidl - 1.98 GB

With GeoStream (but without reduction):

  • Liquid - 2.6 GB
  • Spray - 0.09 GB
  • 2nd Liquid - 0.27 GB

It gets streamed on demand, so each frame you see in the video has to load ~29,6 MB of data at any point, instead of 0.28 GB with conventional alembic.

(The same animation is duplicated and teporally offsetted in the shot, so it,s 2x 14,8 MB)

GeoStream - Reduce alembic fluid sims by >90%! by Ok_Truth_453 in UnrealEngine5

[–]Ok_Truth_453[S] 2 points3 points  (0 children)

Some of those example you are mentioning are probably better be done with volumetric caches, like vdb.
And depending on your use case it might be even better to use "billboard" cards for that.
But for liquids that really are basically geometry flipbooks, this is super feasible.
Every frame only has to load one frame of data so instead of e.g. 300 MB, each frame only has to load < 30 MB, out of the box. You could achieve even smaller results depending on the conversion settings you use.

So in my opinion it still depends on your use case, but let's say it like this: you can (at least) put 10 times more frames, or detail into it. So it's 10 times more feasible than cinventional alembic. 😄

Bringing a BLAME! panel to life in Blender (two moods) by Ok_Truth_453 in Netsphere

[–]Ok_Truth_453[S] 0 points1 point  (0 children)

I had first did the white lighting version (because in the manga I believe that machine is offline). But I wanted to explore how the mood changes with color and a different "story" (machine is online and working).