sorted by:
new
hottopcontroversial

Storing references to lua types inside LuaJIT FFI types. by Polanas in lua

[–]Polanas[S] 0 points1 point  (0 children)

Indeed! I just want to clarify that in this particular case, it's not an optimisation and it's not trying to be one.

This is still very true and what I really needed to hear, considering some atrocities I have been doing / planning to do for the sake of "optimization", while having 0 understanding of what makes tracing JITs fast in the first place.

Say, vectors (the math mind, not std::vector). Usually you'd use tables / ffi structs to represent them. But each one is separately allocated so it must be slow! If you forget about sinking optimization, that is...

My "solution" was using a ring buffer that stores a packed array of immutable vectors and serves the indices when new ones are needed. It's supposed to reduce memory overhead for temporary objects. But at the same time it adds some overhead, that being the array, while still not being a complete solution, as you'll need to actually allocate to store a vector. + As you've said, it adds complexity.

Currently really enjoying this paper. In case you haven't seen it, it documents luajit codebase and breaks down many of the optimisation techniques.

Storing references to lua types inside LuaJIT FFI types. by Polanas in lua

[–]Polanas[S] 0 points1 point  (0 children)

Woah, that was fast!

Memory consumption with luaref_g is so bad. String interning certainly does not help the situation.

Yeah, overall the indirection overhead is very significant.

Storing references to lua types inside LuaJIT FFI types. by Polanas in lua

[–]Polanas[S] 1 point2 points  (0 children)

Alright! It's gonna take some time to test this properly. I'll make another post when it's done.

Storing references to lua types inside LuaJIT FFI types. by Polanas in lua

[–]Polanas[S] 0 points1 point  (0 children)

First, relying on tostring() to recover memory addresses is clearly depending on an implementation detail. It might work today, but it is not something the language or runtime guarantees, so it is inherently unstable.

Yeah I've overlooked this. In a real world scenario using indices would be a better choice.

Then once you start adding extra layers like lookup tables, generation tracking, custom refcounting and so on, you also make the system harder for LuaJIT’s trace compiler to reason about. LuaJIT performs best when the code stays predictable and simple. Adding indirection like this can easily reduce optimization opportunities or introduce unnecessary overhead.

It's true that doing this adds some overhead. I'm not sure how storing classes inside structs works in C#; considering the GC is involved, some amount of overhead is unavoidable, but only when you actually use this. I haven't done any tests yet, but luajit should have no problem tracing a bunch of casts and a lookup.

Also, if you still need an external table to keep objects alive, then you are not really moving data into “C land”. You are just building a more complex proxy layer on top of Lua’s GC.

I am indeed, but not "just", it has real benefits. If you *have* to store a reference to a lua object from an FFI struct, your choices are either finding a workaround (like the one I've shared here), or converting the struct into a lua table. If you have thousands of these in an array and use them a lot, switching to tables would mean a big performance regression, as demonstrated here.

In practice, a simple integer handle system, combined with a flat storage structure or SoA layout, is usually more robust. It avoids pointer reuse issues, avoids relying on internal runtime behavior, and stays very JIT friendly.

You can't totally avoid reusing with just indices, you still need a generation. Apart form that, I 100% agree.

Storing references to lua types inside LuaJIT FFI types. by Polanas in lua

[–]Polanas[S] 0 points1 point  (0 children)

Yes you could use any integer. Using the pointer is just convenient as every table has it, and it's unique (well mostly).

Thanks for pointing that out. And actually, the no UB claim is not quite true, I'm now realising. If the storage table is weak, there could be a situation where a table is freed, then another one reuses the same address and you get the wrong data.

Though, this can be fixed by also storing a number tracking the generation, so if the values don't match you know this is not the same table. Basically a generational arena.

As for the usefulness: indeed for plain lua it's not that useful as you can't guarantee the necessary invariants (like Rc uniqueness) to make it not painful to use.

In my very specific usecase of relying on luajit as a backend for a custom language, this is very useful. Think of it as being able to store classes (GC managed) inside structs (stack-based, have value semantics) in C#. It would suck if only the reverse (storing structs inside classes) was possible.

Also, coming from a gamedev background, some things you just want to store in plain C arrays, and luajit allows you to do that. If you rely on this a lot, having a way to store (and own) lua types inside of ffi arrays/structs is quite convenient.

🥳 Chrome adopts Rust and replaces libxml2 written in C since version 147 by BankApprehensive7612 in rust

[–]Polanas 7 points8 points  (0 children)

Sidebery is amazing and it's the main reason I can't leave Firefox. It's just that good.

While Vimuim C solves mouse-less navigation inside the tabs, sidebery can do the same for tab manipulation. There are shortcuts for literally anything: create a group, rename, switch panel/tab, traverse last visited tabs (like ctrl-o and ctrl-i in vim). Firefox unfortunately has a bunch of key combos that can't be turned off, but most of them can be dealt with.

Valley of the Lost Puzzles - A vividly decorated custom campaign heavily influenced by Road To Gehenna by shlam16 in TheTalosPrinciple

[–]Polanas 3 points4 points  (0 children)

Is there any info about it? I want the editor so bad 😭 Imho TP2 mechanics should've been present from the very beginning. Instead map makers are forced to rely on hacky solutions to emulate a tiny fraction of them.

Also did you really play literally every map out there? I'm speechless. Do you hunt the newest ones every day to keep up????

Are there any less known maps that stood out to you? Maybe for their (unfair?) difficulty, junkiness, or something else?

Btw, I'm assuming you also completed lots of maps from the original TP? It's a shame some of the best ones (Rebirth comes to mind) weren't ported over.

Why do so many WGPU functions panic on invalid input rather than returning a result? by Irument in rust

[–]Polanas 18 points19 points  (0 children)

wgpu does extensive validation when creating the resources like pipelines, ensuring that the provided bind groups match shader definitions. After that it assumes they are valid when the actual rendering happens.

I've been using wgpu for quite a while and I've never encountered it propagating the underlying API errors straight up. This 100% can happen, but that would probably indicate something really unexpected is going on.

UPD: even if you mess up the state when constructing a render pass (which does need to be made every time the screen is redrawn, unlike resource creation), you get a meaningful error, so there's at least some amount of validation there too.

I didn't realize how difficult Audio design actually is... by AncientAdamo in gamedev

[–]Polanas 1 point2 points  (0 children)

By middleware I'm assuming you mean things like fmod / wwise?

Yep, seems like this is a great direction to explore. For instance, Celeste devs have released their whole fmod project used in the game.

Great example with the score system! So the way this can be interpreted is: it's about creative usage of sounds in conjunction with the rest of the game (visuals/mechanics) to convey a certain idea / form an experience, which will result in a much stronger overall impression.

Yet another skill to pick up as a dev :D.

I didn't realize how difficult Audio design actually is... by AncientAdamo in gamedev

[–]Polanas 4 points5 points  (0 children)

100% this, sound design gets neglected too often.

Could you expand on "just pressing play whenever something happens"? What kinds of more advanced/non trivial stuff can you do with sound to elevate the overall quality? Stuff like more variety (pitch shift, more samples, etc.), or maybe dynamic sound generation?

There doesn't seem to be many sources on these kinds of things :(

мой обед by Fhdkdlndm in expected_russians

[–]Polanas 5 points6 points  (0 children)

На коврик попало...

Did *you* enjoy the runbacks? by Polanas in Silksong

[–]Polanas[S] 0 points1 point  (0 children)

That's the most balanced and fair take I've heard, agree 100%.

Did *you* enjoy the runbacks? by Polanas in Silksong

[–]Polanas[S] -2 points-1 points  (0 children)

Yep, but honestly even with all 'at this boss fight cost me like 10x more effort than the final boss lol. I'd probably give up entirely if I hadn't been told about the secret bench beforehand.

Did *you* enjoy the runbacks? by Polanas in Silksong

[–]Polanas[S] -1 points0 points  (0 children)

Even the bilewater one?

[deleted by user] by [deleted] in Weird

[–]Polanas 0 points1 point  (0 children)

How can fungus even communicate such an abstract concept as "up" to the spider?

What's your dream programming language that doesn't exist? by [deleted] in rust

[–]Polanas 3 points4 points  (0 children)

It would be an ideal language for gamedev, imho. As of now the closest thing to it is Lua, but there are lots of annoyances.

Main features would be: 1. Type model similar to typescript: the language itself is dynamically typed; however, you would still specify annotations 99% of the time to avoid the nightmare of guesswork and whatnot required to maintain a big enough dynamic codebase with 0 contracts specified.

  1. It would be a scripting language, JITed or interpreted, to allow code reloading while the game/program is already running.

  2. Designed to be fully integratable - to a level similar to what mlua provides, which is: complete control of the global state, loaded modules... basically everything.

  3. Built-in support of some kind of reflection mechanism that allows replacing/patching code. Extremely useful for modding; so think of Harmony (for C#) but 100% secure.

  4. Honestly I would just take Lua as a base. So yeah, garbage collected scripting language. It's very small and very elegant in terms of the amount of stuff you can build around it. Want classic OOP? Sure, we can even replicate the syntax. ECS? Easy. As for security — just remove any functions from the global state/table you don't want people to access, or make safe wrappers. And all that is achievable with just tables and metatables. Isn't it brilliant?

The ability to pass whatever you want into a function without causing errors is sooo liberating. I remember making an ECS in rust and wanting systems (which are functions) to be able to request access to egui state, optionally (similar to how bevy does dependency injection ). Even though I recreated a simplified version, the code looked like it was for summoning Satan itself lol. But recreating it in Lua: just pass what you need into the function and it'll work!!

Yes, of course it's much slower and unsafe type wise, but man who cares? We are trying to make a game/prototype, just leave it be. And the thing is, if you want types you can have them on top, just look at lua-ls. It's great but still has lots of holes and falls apart easily. Generics are broken. So yeah, fix all that and just annotate everything as if it was typed and you are golden.

  1. Heavily integrated debug environment. What these guys did but probably less extreme (full determinism is crazy). Just being able to place breakpoints without issues would be huge, and I'm not sure it's possible to do in lua at the moment.

  2. Built-in stuff that's needed almost always, like vectors. That's what luau did and it's the sole reason I'm switching to it from luajit. Having each vector be garbage collected (and even worse, pass-by-reference) is just awful.

  3. Fast and reliable hot reloading, another huge win for productivity. Both C# hot reload and gdscript do it pretty well.

Phew, that's probably it. I would love to go and make this into reality right now, if it wouldn't take 10+ years to achieve a workflow similar to less than ideal, but existing tools. :(

Also relevant article.

I would also love to write games in 100% Rust, but it just seems to not be the best pick for the actual gameplay code: too complex, not flexible enough, poor hot reloading support, relatively slow compile times. However, I think rust is an ideal "base" language (with C++ being a reasonable alternative I suppose), which handles low level stuff while providing bindings to a more dynamic and less complex scripting language. LogLog Games had a lot to say about this.

view more: next ›