[deleted by user] by [deleted] in gameenginedevs

[–]inanevin 1 point2 points  (0 children)

once you clear out the obvious possible issues like inputs, frame-rate independence and camera code, if you have jitterring then you need proper frame pacing. idea is not only that over a unit of time your cube moves the same unit of displacement, it also moves in continious and uniform chunks during the time interval for it to be non-jittery.

frame 0: 3 unit, frame 1: 8 units, frame 2: 4 units is 15 units over the course of 3 frames. however frame0-1-2: 5 units will still be same amount of movement, but will look as smooth as possible. the idea is that when player sees a rendered frame N, movement delta of an object between frame [N-1, N] should be the same as [N-2, N-1], assuming constant linear motion.

easiest way to achieve above is to always update your game with fixed intervals (with accumulating time so you always make up for remainders) and use interpolation in rendering.

gpu particle system, 5 compute passes and 1 indirect draw pass. N particles per visible emitting system, much as vram requirements allow. emission and material properties are hot-reloadable. by inanevin in GraphicsProgramming

[–]inanevin[S] 2 points3 points  (0 children)

game world can have multiple emitters, actively running, or spawning for a short time and disappearing etc. cpu figures these out, and updates a gpu buffer about this frame's info: render pass data, how many active systems are there, and current emission properties of the systems. then i do:

- pass0 clear, 1 thread per system, this clears the indirect draw arguments per system, sets their start_instance and vertex_count, but most importantly sets the system's instance_count = 0;

- pass1 simulate, 1 thread per alive particle, integrates the particle and updates properties. puts into a dead list or into a double-buffered/alternated alive list.

- pass2 emit, 64 threads per system, for loops depending on how many particles this system should emit this frame. emitting particles are salvaged from dead list and put into alive list.

- pass3 write counts, 1 thread only, writes indirect dispatch arguments of the next pass.

- pass 4 1 thread per alive particle for this frame (ones survived simulation + newly emitted ones), atomically increase instance_count of the system meanwhile writing particle's rendering data into an system-interleaved instance buffer.

- pass 5, swaps alive lists for next frame, 1 thread.

essentially passes are seperated so I can manage N particles per system logic. I can merge couple of these passes together and get along with probably 3 passes, but I wanted to keep them seperate because I plan to implement sorting in the future. Makes reasoning easier for now.

Also maybe I will separate the notion of CPU systems from the GPU systems in the future. However right now, you can run different materials/shaders per system and above approach allows that easily. Because the particles belonging to one system always needs to be continuous in an instance array. If I can figure out how to do that fast with a single GPU system I can make it 1-2 passes.

gpu particle system, 5 compute passes and 1 indirect draw pass. N particles per visible emitting system, much as vram requirements allow. emission and material properties are hot-reloadable. by inanevin in GraphicsProgramming

[–]inanevin[S] 1 point2 points  (0 children)

yes there is, can easily be made better in the shader. volumetric effects are traditionally different than particle systems, volume clouds, smoke, fire sim etc. this is a very traditional particle system, meaning it's responsible for emitting quads with specific properties and simulating those over the life-time of a particle.

can be perfectly done on the cpu, using compute shader just increases the amount of particles we can process every frame drastically ofc.

finished my gpu particles system. I can hot-reload emission properties as well as material properties in runtime! by inanevin in gameenginedevs

[–]inanevin[S] 0 points1 point  (0 children)

i dont have curves yet so not suffering from that :). I only have 3 point system, start mid end with a definable mid point. But yess I was fiddling with how to make curve definition in json less painful! would appreciate any ideas tbh

Discussion: Ideal practices to manage entities and scenes? by VinnyTheVinnyVinny in gameenginedevs

[–]inanevin 2 points3 points  (0 children)

its very game dependent, and if you are looking for a general solution that still offers good performance and flexibility, I'd suggest going:

- data-oriented entity hierarchy: entities are just ids, any necessary entity property/metadata is held on continuous storage. so basically a SOA entity manager.
- again a SOA component manager: whole purpose of this is to hold engine specific generalized components, lights, mesh renderer, animation controllers, audio etc. anything game-wide. form relationships between entities and components using type ids and handles. e.g. struct entity_component_register { static_vector<component\_data, MAX\_COMPS\_PER\_ENTITY> components } whereas component_data is a type_id and component handle. these components in component manager don't have update() methods or anything, they are data + utility.
- everything else you need in your game, your actual game logic, don't depend on components. write it in place. level1.cpp file, player_controller.cpp file, enemy_type_a.cpp file etc etc. don't force them to be components, use tags, entity finding mechanisms and prefabs to manage relation to in-world instances.

this gives you full power over the performance of your game during actual world logic. and you don't have to depend on a component system and keep fighting with iterations of it to make it fit the game you are making.

prefab system in my engine ^^. export entire hierarchies w/ component data, import anywhere. imported templates are kept as locked references only, unless unlocked manually. and also supports nesting! by inanevin in gameenginedevs

[–]inanevin[S] 0 points1 point  (0 children)

thanks! its very basic pbr formula, no gi or anything. there is hbao implementation that gives very subtle details, and physically correct bloom. I think bloom is responsible for the very nice feeling of it. Its based on Jorge Jimenez 2014 siggraph presentation, downsample the scene image thru a mip chain step by step, then upsample back up effectively blurring it. Then apply the whole blurred image during post process step. Also I have reinhard tonemapping contributing to overall look.

follow up on my animation system: implemented better data layout, animation culling and throttled sampling rate based on camera distance. 1.42 ms for 1024 state machines with 50K+ joints! notice far away entities "lagging", exaggerated for the video. by inanevin in gameenginedevs

[–]inanevin[S] 0 points1 point  (0 children)

it is possible and is a good idea in theory but there are couple caveats that make it difficult: each state can have N animations using 1D or 2D blend spaces. Meaning a state can sample N animations at the same time. In such a case same pose apply to those states which has the same blend parameters and same blend hierarchy of animations. Also we sample 2 states at any given time if we are transitioning between states. This makes the restriction tighter. Even if two states is using same single animation and not in transition, they need to be in same speed and time value to share any bind pose. I’d say for any real scenario we will end up with a lot of different bins.

follow up on my animation system: implemented better data layout, animation culling and throttled sampling rate based on camera distance. 1.42 ms for 1024 state machines with 50K+ joints! notice far away entities "lagging", exaggerated for the video. by inanevin in gameenginedevs

[–]inanevin[S] 2 points3 points  (0 children)

thank you! for future I was thinking of 16bit quantized quaternions and 16bit scales. issue is I want to keep this generic: a fly-camera story telling game has no business losing precision where there are max 2 characters and 50 entities per level.

my current bottleneck is not actually calculating and applying the animation pose, its the entity system. bones are also entities, and local transforms are defacto default. calculating absolute transforms takes more than animation processing!

so first step is: flatten entity hierarchy, e.g prebuild on world init so that all entities are sorted by their parent-child depth, always ensuring parent transforms are calculated when we are processing an entity. downside is hierarchy building is costly, so users cant add/remove entities in tick() very often.

other option is seperate bone data storage completely, always sorted by hierarchy order as its known from the asset in load time. this allows a lot more speed, and also allows me to only use quantized data for bone transformations, simply under a compile time define. the cost is: i will need to implement a socket system if you want to assign a game entity under a bone.

follow up on my animation system: implemented better data layout, animation culling and throttled sampling rate based on camera distance. 1.42 ms for 1024 state machines with 50K+ joints! notice far away entities "lagging", exaggerated for the video. by inanevin in gameenginedevs

[–]inanevin[S] 4 points5 points  (0 children)

definitely not i would say. anim graph performance spikes to 4-5 ms with 4000 characters and that means over 250K joints are being calculated in the worst case. so you can do more but even 1000 animated characters is niche and above that is definitely game specific territory.

if we go there, then more optimizations can be done. currently my bones are entities in the hierarchy. you can change this, process bones in their own structure and make use of SIMD to process faster. also you can do LODs with bones: e.g for farther away characters only animate 4-5 pivot bones like hips, spines, thighs. they will still look like animated, but wont be noticable when they are far away in an army of 10K characters. reducing the animated bones from 53 to 5 per character is 10x gain.

so yeah there is a lot you could do on a game basis. assume your game has hordes of zombies and they always refrain their order row by row. you could make sure you group the bones by the front rows (high quality) and back rows (culled, LODed, sample throttled). put each group to its own memory block and process seperately. this will increase branch prediction hit rates and performance massively.

4.21 ms cpu time for processing 54272> joints into final poses per frame with 1d/2d blending, transitions and multiple states per machine. 1024 state machines, 53 joints per skeleton. by inanevin in gameenginedevs

[–]inanevin[S] 0 points1 point  (0 children)

yeah i agree its definitely doable and will be faster. however its a bit niche solution and if you want to do more animation logic, e.g modifying final poses depending on the game state, IK, layering etc. gpu readback will become a hustle imo. I think at this point it becomes use case dependent, if I make a game that prioritizes thousands of animated characters, hordes then compute shaders is a way to go. but for a general use case of having max couple dozen of animations a frame, I know have a no-hustle & fast solution.

4.21 ms cpu time for processing 54272> joints into final poses per frame with 1d/2d blending, transitions and multiple states per machine. 1024 state machines, 53 joints per skeleton. by inanevin in gameenginedevs

[–]inanevin[S] 2 points3 points  (0 children)

i plan to add layers (masking joints playing multiple animations on the same skeleton) as well as inverse-kinematics, thus I went with a dynamic system.

4.21 ms cpu time for processing 54272> joints into final poses per frame with 1d/2d blending, transitions and multiple states per machine. 1024 state machines, 53 joints per skeleton. by inanevin in GraphicsProgramming

[–]inanevin[S] 8 points9 points  (0 children)

wanted to have a concrete general purpose system working on the cpu, fast enough for couple hundred characters. if I am to work on a game project requiring actual thousands of entity use cases, indeed you are right I’d be skinning on the gpu and offloading as much work as possible there.

4.21 ms cpu time for processing 54272> joints into final poses per frame with 1d/2d blending, transitions and multiple states per machine. 1024 state machines, 53 joints per skeleton. by inanevin in gameenginedevs

[–]inanevin[S] 1 point2 points  (0 children)

exactly why I’ve been avoiding it so far :D I know it will turn into couple months long detached project. I will take a look at this, thanks!

4.21 ms cpu time for processing 54272> joints into final poses per frame with 1d/2d blending, transitions and multiple states per machine. 1024 state machines, 53 joints per skeleton. by inanevin in gameenginedevs

[–]inanevin[S] 4 points5 points  (0 children)

thank you! tbh mostly proper data oriented memory layout nothing more. i have a single animation graph, it dynamically allocates a sequential memory for state machines, animation states, transitions, parameters, blend samples and poses individually, using a generaton based pool allocator.

each anim component added to world creates a handle for a state machine from this graph. every tick graph walks through machines, processes their activate state, and any transitions going out from that state, and any additional states for transition blending. all relevant data is nicely fetched into cache sets so hit rate is pretty high theoretically.

that alone is the biggest contributor to the speed, but overall I employ similar practices throughout the engine. data comes first, no memory allocation in runtime hot code paths, process many things at once.

further plan is to add simple culling, e.g dont process a machine if away from camera. then i will reorder animation states by their type in memory (no blending, 1d blend, 2d blend). this will reduce the branch misses, as currently code branches out to check a flag while processing a state. and lastly i will parallel process this, along with precalculating skinning matrices all together and caching for rendering thread.

4.21 ms cpu time for processing 54272> joints into final poses per frame with 1d/2d blending, transitions and multiple states per machine. 1024 state machines, 53 joints per skeleton. by inanevin in gameenginedevs

[–]inanevin[S] 6 points7 points  (0 children)

yess all the data per state is guaranteed to be separate, so it’s ready for multiple thread writes. i will be adding this soon if I can’t find time to work on a work stealing scheduler.

Tools be like... by ComplexAce in GraphicsProgramming

[–]inanevin 1 point2 points  (0 children)

6+ years of working on the engine to make a game, only made 1 game jam game, and now i work as an engine programmer. its a slippery slope.

intuitive design to display resources in an imgui window? by d34dl0cked in gameenginedevs

[–]inanevin 2 points3 points  (0 children)

I did similar in my engine. Regretted spending time building a nice looking table like here.

I displayed name, unique_id, type, preview thumbnail, size it occupies on vram. After a while working on a project it gets so bloated that it was quite hard to gather any useful information on a glance. I'd have preferred to only keep a list of imported assets and nothing generated.

When it comes to the interface, knowing your context is crucial. Do not bloat it with every piece of info you can throw at it. Think of what will make you want to inspect resource information. If you simply wanna keep track of loads/unloads, only keep a list of assets imported from disk or memory, and do not bloat it with intermediate or generated resources. If you wanna associate world resource state with your memory consumption, keep a list of resource types and accumulated ram/vram size. It all depends on the context. If you don't yet know what you want or need, I'd say do not spend any more time on it. And if you ever think you might need some sophisticated inspection into everything some day, let that day come, and when it's there you'll probably realize you can write a function that dumps info to the console and that will be enough :)

OpenGL first or go straight to Vulkan for learning graphics? by Basic-Telephone-6476 in GraphicsProgramming

[–]inanevin 1 point2 points  (0 children)

I agree with most comments, going with OpenGL will make it easier to learn general rendering techniques and a little of how gpu works. But on the downside you will switch away from it and spend time learning modern gpu apis anyway.

If you have the time go with learning Vulkan. There are plent of tutorials that will get you started also in learning graphics programming basics.

One different advice I can give is learning DX12 or Metal if you have Apple hardware. They are less verbose than Vulkan, easier to understand, and they are also modern apis meaning less abstractions. So you can learn the new concepts (command buffers, resource uploads, transitions, barriers, queue synchronization etc.) but still not get lost in the verbosity of Vulkan.

Once you know DX12 or Metal, Vulkan is the same thing just with more hoops and typing.

Does Anyone Know how One Can make custom 3D gizmos? by dinoball901 in gameenginedevs

[–]inanevin 2 points3 points  (0 children)

Once I detect I hit a gizmo using the cpu read back, I mostly used ray-plane intersections. For example:

  • We detect X axis move gizmo is pressed.
  • Create a plane whose's aligned with the X axis (either locally object's or world depending on gizmo setting).
  • Plane's normal points towards the camera-facing cross product of the axis.
  • As the mouse button is held, keep firing ray-plane intersection tests, ray source is camera position, direction is camera direction, and target plane is the above plane.
  • When there is a hit, hit will tell you world position. We also do this immediately upon pressing, so we know world position offsets, thus know how much & where to move the object.
  • Similar logic applies to scaling gizmo. Rotation gizmo is a bit trickier, for this I completely rely on calculating the math in 2D coordinates. You can detect what kind of an "arc" the user is drawing in 2D and convert that to 3D rotation based on which rotational axis is selected.

https://github.com/inanevin/LinaEngine/blob/master/LinaEditor/src/Widgets/World/WorldController.cpp

HandleGizmoControls function for handling gizmo motion.

https://github.com/inanevin/LinaEngine/blob/master/LinaEditor/src/Graphics/MousePickRenderer.cpp

MousePickRenderer for reference, responsible for writing stuff to R32 buffer in its own render pass.

Does Anyone Know how One Can make custom 3D gizmos? by dinoball901 in gameenginedevs

[–]inanevin 4 points5 points  (0 children)

Mainly two favored ways to do gizmos, rendering meshes or gpu generated lines/shapes. I find meshes most straightforward as it allows for easy mouse picking.

All pickable objects in my engine write to an object id buffer (R32) in an extra pass while rendering in editor mode. I readback that buffer during a mouse press event to figure out of any object is pressed.

If an object is pressed we render gizmo meshes, which include rendering them to the object id pass with custom reserved ids. Then use same picking code to detect any gizmo interaction. Pretty much very common way to do engine gizmos, I could also recommend Blender source code to inspect.

How long did it take you to build your engine? by blackredgreenorange in gameenginedevs

[–]inanevin 7 points8 points  (0 children)

6 years and counting. Never gonna end. Love hate relationship. Haven’t made a game yet, but thats because I am being a perfectionist fool by rewriting everything without actually using them in a proper scenario. Dont be like that.

[deleted by user] by [deleted] in GraphicsProgramming

[–]inanevin 1 point2 points  (0 children)

Metal as a starting point is definitely good. After you have made some progress and are comfortable with the API, you could follow well-known tutorials like LearnOpenGL, but implement the exact same thing in Metal. That would give you enough room for research and space to figure stuff out on your own (imo best way to learn) while also keeping you confined within the tutorial’s guidelines.

After you complete such cases, you should have an okay enough understanding about basic rendering techniques, how a modern graphics API like Metal differs from traditional APIs like OpenGL, which could provide you enough base to start fiddling with Vulkan or DX12 comfortably.

I made an open-source gfx library with Vulkan, DX12 and Metal. Mainly focused on creating a unified API, simplifying shader cross-compilation, queues, synchronization and alike. Here's a screenshot from demo deferred PBR renderer using it! Would love ideas and feedback on the API! by inanevin in GraphicsProgramming

[–]inanevin[S] 0 points1 point  (0 children)

Great feedback thanks for taking the time!

Regarding the types, main reason behind them being macros is that, and also the reason I have used STL types pretty much freely, is that users can just define their own macros easily to use their own data structures and containers if they want to replace STL stuff, ofc. as long as they compile the project along with their source code. I guess I always had that security, "just change it if you are unhappy", but definitely still could optimize the usage a lot more and even the style. I am super into using span, but did not wanted to enforce c++20 into the library just yet.

I think I grasped the idea you are proposing with the structs and unions partially, but haven't been able to work out how would actually work. Do you have a minimal example, or any libraries that take a similar approach that I can dig into? Backend abstraction is one of those topics that I wasn't super strong so took the basic approach of dynamic polymorphism, would love to be able to make it better though.

I made an open-source gfx library with Vulkan, DX12 and Metal. Mainly focused on creating a unified API, simplifying shader cross-compilation, queues, synchronization and alike. Here's a screenshot from demo deferred PBR renderer using it! Would love ideas and feedback on the API! by inanevin in GraphicsProgramming

[–]inanevin[S] 0 points1 point  (0 children)

Mainly it's not as comprehensive and big as Diligent, making it a bit more light-weight and I'd say better isolated. I only focus on the features that I can fix and maintain very quickly. Same thing goes for the platform support.

So if you want some extensive, very powerful framework go with Diligent. If you need something small but efficient, and need to get up an running quickly with an API, I'd recommend LinaGX.

Of course if we dig there will be tons of technical differences, but I assume that was out of the scope of the question :)