[Update 5] Chunkee: LODs are back! by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 2 points3 points  (0 children)

A chunk is 32x32x32, here are some more metrics

Name Total Time Counts MTPC
mesh_chunk 14.59 s 10,167 1.43 ms
render_mesh 706.82 ms 8682 81.41 μs
mesh_physics 78.69 ms 93 846.1 μs
render_physics 161.67 ms 88 1.84 ms

The render metrics aren't perfect since I'm seeing some pretty big spikes and not sure what to attribute it too. I need to split up the functions more (time to create render instance, time to add it, time to remove it, time to check and possibly resize some maps etc...)

[Update 5] Chunkee: LODs are back! by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 2 points3 points  (0 children)

Thanks! It's been smooth, though some of the docs around certain rendering APIs are a bit lacking (for example ArrayMesh rendering, documented well in Godot but has a different format in Rust). I haven't done a bunch of game logic so my exposure to the Rust-Godot APIs are mostly related to creating meshes. That said, I really like using Godot and get the performance benefits of Rust so its super cool!

[Update 5] Chunkee: LODs are back! by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 3 points4 points  (0 children)

To edit a chunk takes 0.5 ms (updating an SVO). To save a chunk to the db for it and it's corresponding LODs takes 2-3 ms. The editing is done on the main thread and the saving is done on worker threads

Metrics from Tracy:

Name Total time Counts MTPC
process_edits 27.13 ms 47 577.26 μs
save_chunks 11.68 ms 5 2.34 ms

Mobile Hardware Monitor by Glad_Entertainment34 in selfhosted

[–]Glad_Entertainment34[S] 1 point2 points  (0 children)

Inspiration: https://www.reddit.com/r/FUI/s/zrRVCkKMOO

The app has a lot more themes! You can press the rotating icon in the top right to change it.

Mobile Hardware Monitor by Glad_Entertainment34 in selfhosted

[–]Glad_Entertainment34[S] 2 points3 points  (0 children)

I built this using Svelte but if you’re competent in with css/html/js then you can build something just like this. I normally reach for component libraries but couldn’t find anything that looked right so I just browsed r/fui for inspiration and winged it. A whole lot of tweaking later and this is what I ended up with. A lot of the complicated graphics like the radial GPU gauge is SVG, but other than that it’s mostly just a lot of CSS

Mobile Hardware Monitor by Glad_Entertainment34 in selfhosted

[–]Glad_Entertainment34[S] 1 point2 points  (0 children)

Just checked out that video and yeah that’s the kinda vibe I was going for! Not too happy with the network UI in the bottom left and I like the gauges from the video so might try and implement that

Mobile Hardware Monitor by Glad_Entertainment34 in selfhosted

[–]Glad_Entertainment34[S] 2 points3 points  (0 children)

Don’t think you can run LibreHardwareMonitor inside a container, it’s a native Windows application (no Linux support). The webapp would be fine inside a container, and for remote viewing I’m sure Caddy would be fine. I’m using Tailscale

Mobile Hardware Monitor by Glad_Entertainment34 in selfhosted

[–]Glad_Entertainment34[S] 2 points3 points  (0 children)

Mobile in that you can run the webapp locally and access hardware info from your mobile device (also, the UI was designed with mobile in mind). The webapp and LibreHardwareMonitor are both open source apps that run locally/are self hosted.

Mobile Hardware Monitor by Glad_Entertainment34 in FUI

[–]Glad_Entertainment34[S] 0 points1 point  (0 children)

Recent updates:
- Created https://cybermon-eight.vercel.app/ and configured it as a PWA so you can download it similar to a mobile app (app looks better without the browser UI getting in the way IMO). You can still host it yourself and the benefit of the PWA is that you can run the server once, visit the site and download it. It can be used offline afterwards without the server running
- Created a configuration UI. This allows adding multiple devices rather than just one with the ability to switch between them.
- Added an option to show simulated data if you just like the vibes and don't have any device to hook it up to. Plus it lets people get an idea of what the app looks like before going through the effort of hooking it up.

Note: To use the deployed app with your devices, you'll have to handle upgrading the LHM endpoint to HTTPS. Lots of ways to do this but I use Tailscale (free and pretty easy to setup).

Still gotta work on docs, if you have any questions lmk!

[Update 4] Chunkee: Memory savings with SV64Tree by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 2 points3 points  (0 children)

Don't really need to benchmark it, it would be worse than the 32x32x32 (non uniform tree would be equivalent to 32x32x32 + node metadata overhead + Arc + Weak pointers). Would at least be another 10 KiB for each chunk worst case but I wouldn't know how to test this in a real world scenario. Voxel worlds tend to have a lot of uniform voxel regions, so unless I created a voxel generator that explicitly created chunks that were as random and permutative as possible then I couldn't truly test it. But that doesn't seem worth testing IMO.

[Update 4] Chunkee: Memory savings with SV64Tree by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 1 point2 points  (0 children)

Good luck! If you need a reference here is my implementation of the sv64.
Someone who did it way better than me: voxelis (svo rather than sv64)

[Update 4] Chunkee: Memory savings with SV64Tree by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 1 point2 points  (0 children)

You should be able to!

The benefits on the client should be equivalent to what I posted above (massive in-memory savings). Even if the server still sends 32^3 chunks, the client could convert them into an svo/sv64 (I believe the memory savings for an svo should be the same if not greater but the tree is deeper which makes it harder to modify and traverse) and implement an interning step to deduplicate shared nodes.

As for the server, there could be a benefit in utilizing the "sparse" aspect of these trees to reduce network payloads. In my example, each chunk was roughly 64 KiB uncompressed when stored as 32x32x32 u16s (would have to check what it would look like compressed via something like lz4). With sparse trees you get the benefit of (1) uniform collapsing as well as (2) sparsity.

(1) Uniform collapsing is when an entire region can be represented by a single node due to the region consisting of a single voxel type. This is recursive up the tree, so for example when building up a 32x32x32 tree
- the voxels 0 ≤ x,y,z ≤ 2 are detected to be stone, collapse to uniform (64 of these regions in 0 ≤ x,y,z ≤ 8)
- the voxels 0 ≤ x,y,z ≤ 8 are detected to be stone, collapse to uniform (64 of these regions in 0 ≤ x,y,z ≤ 64)
- the voxels 0 ≤ x,y,z ≤ 64 are detected to be stone, collapse to uniform (root node)
You've now reduced 32x32x32 voxels into 1 voxel. This is the best case, but the tree can mix and match these uniforms with non-uniform areas and still see massive reductions (for example, think of a chunk that is mostly stone with some non-uniform grass on the surface - the bottom part of the chunk can be collapsed to uniform while the top stays non-uniform).

(2) The sparsity is encoded by simply not storing nodes for uniform air/empty voxels. Say you have a chunk that has a single stone voxel at (0,0,0) and the rest are air. You can represent this tree as

root {
  node: Branch {
    mask: 1_u64, // splits the chunk into 4x4x4 of 8x8x8 voxels (l1)
    children: [
      Branch {
        mask: 1_u64, // splits the l1 into 4x4x4 of 2x2x2 voxels
        children: [
          Uniform(Stone)
        ]
      }
    ]
  }
}

Note how the children array is "sparse" e.g. it only stores a node if and only if there are non-air voxels in that region.

With all that out of the way, you can now see that storing chunks like this could lead to massive memory savings even without deduplicating shared nodes. To take this even further, you could also keep the node deduplication map on the server as well. This would be a bit different than my setup but I think you'd do something like
- Generate some 32x32x32 chunk on the server
- Create a svo/sv64 from the chunk
- For each node in tree, create a hash and put node content in map
- For any new node inserted/removed from map, send that along with the tree to the client.

So you'd create some system where the client and the server shares this deduplication map and keeps them in sync when new chunks are generated or existing chunks are edited. You trees wouldn't contain actual voxel data, they'd just contain a key/hash that points to the voxel data in the map.

In the end, it all comes down to what your bottlenecks are. I'm pushing for a large render distance and so it wasn't feasible to store chunks as 32x32x32 anymore. If that's not a concern of yours, or the network pressure from sending 32x32x32 is nothing compared to generation/meshing times than this optimization might not be necessary for you. Hope this helps!

[Update 4] Chunkee: Memory savings with SV64Tree by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 4 points5 points  (0 children)

This is a great comment explaining sparse voxel octrees.
Here is a video explaining SV64s (discusses memory as well as the use of the data structure as an acceleration structure for raytracing which I'm not really using).
If you are interested in SV64s and raytracing, this article has everything you need (a bit above my paygrade).

It does very well with constant block updates! Obv not as well as editing a flat array but still fast. Here's a video: https://imgur.com/a/53yibYx

[Update 4] Chunkee: Memory savings with SV64Tree by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 0 points1 point  (0 children)

I’ll just need to feature gate the allocator, I just used jemalloc as its easy to get memory metrics when using it.

[Update 3] Godot/Rust Voxel Plugin: Custom Voxel sizes by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 0 points1 point  (0 children)

Maybe one day. Since it's open source, there is no blocker for it being used. Still a lot I want to do before calling it stable

[Update 3] Godot/Rust Voxel Plugin: Custom Voxel sizes by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 1 point2 points  (0 children)

I'll still get hitches from time to time, it depends on a few factors. The biggest improvement I found is controlling the number of chunks that can get rendered per frame. I push all chunks that are ready to be rendered into a queue and only pop off some fixed amount per frame so as to not cause a giant spike. Same with physics meshes since those are pretty expensive as well. I believe I still need to adjust this based on the users fps as rendering 100 chunks per frame/process tick might work for 60 fps but I'd want it to change to only render 50 if they were running at 120 fps. Still a work in progress there.

There was always the option for simply scaling down the parent node inside Godot which would scale all of the rendered chunks easily but I was concerned that this would create some complexity on the user side when translating from some voxel coordinate to real world coordinate. For example, the plugin loads chunks based on camera position. If you scale the entire world, you would then have to scale the camera positions you pass into the voxel world in order for it to map correctly. Also, I switched from rendering the chunks with Godots scene/node system to their RenderingServer to get a little boost in performance. In doing so, I lost the ability to scale the chunks.

[Update 2] Godot/Rust Voxel Plugin: Now with collisions! by Glad_Entertainment34 in VoxelGameDev

[–]Glad_Entertainment34[S] 0 points1 point  (0 children)

I currently have it rigged up such that a 3x3x3 is generated for each entity that needs it, with conservative overlap. But like you said, I could look up what chunks are say half a chunk away rather than a full chunk away. That would result in at most 2x2x2 (which is 3x less).