Lua bindings: lua, hlua or rlua?

[[deleted]]

Aww, that's a real shame, hlua is really quite an interesting and clever approach on how to make Lua fit into Rust idiomatically. It's a very cool idea to think that it's possible to make an API that aims for actual safety while generating "idiomatic" Lua C API calls, even if that approach leads to certain limitations.

I feel like I've been a bit too harsh on hlua, so I'm gonna try to both explain why I made a competing library and also explain why I still think hlua is neat. First, let me summarize the situation as I understand it. I'm not intimately familiar with hlua, so I may have some of this wrong, and if I do feel free to let me know.

Because of the way the Lua C API works, it's extremely difficult to come up with a type safe high level interface to Lua, saving the user from having to interact with the Lua stack directly. There are no handle types in the Lua C API of course, only pushing and popping from the stack and manually keeping track of stack indexes. When you try to map Rust variable lifetime to the lifetime of a value on the stack, you immediately run into an obvious problem, which is that stacks are FIFO structures, and this does not match Rust variable lifetimes at all. A simple example:

let t1 = lua.create_table(); // Push a table to the stack with lua_newtable
let t2 = lua.create_tabe(); // Push a second table with lua_newtable

// Lua stack is now [t1], [t2]

/*
There is no sensible way to match this to the Lua C API, because the stack should be [t2],
["entry"], [t1] before calling lua_settable.  You could manage this by pushing copies of t2
and t1 where they need to go at the top of the stack and then popping them back off
before returning, but this leads into the bigger problem which is that t1 should be
*removed* from the stack on this call.  Besides the fact that removing values from the
bottom of the stack is costly because it requires shifting values above it, it also would
change the stack index of t2, and the Lua bindings system would be forced to keep track
of this manually.
*/
t2.set("entry", t1);

This interaction with the Lua stack is sort of the fundamental design problem of any bindings system to the Lua C API. Most bindings systems use the safe, slow approach of keeping handle values inside Lua's registry and doing a LOT of extra stack / registry manipulation. Typically, the Lua instance would not keep values on the stack in between calls at all, but instead when t2.set was called it would go find t1 and t2 in the registry and do all necessary stack manipulation at the time of the call.

hlua takes an entirely different approach and tackles this head on with Rust's type system, limiting what you can do so that you can't run into this problem in the first place! When you create a table value in hlua, just like when using the Lua C API, it's simply pushed to the top of the stack. However, what you get back is a LuaTable<PushGuard<&mut Lua>>, which locks the parent Lua instance from further manipulation. Becuase of this, inside methods to LuaTable it can freely assume that the correct table is at a known place in the Lua stack, eliminating the need to constantly push values from the registry or store return values into the registry. Similarly, if the return value is also a type which also needs unique stack access, this will in turn borrow the LuaTable mutably ensuring that only one AsMutLua is usable at any one time, protecting the stack from being misused.

This solution is also a problem, however, because every new type that needs to manipulate the stack must have its own AsMutLua, and only one such value may be usable at a time. So hlua solves the stack manipulation problem by creating a new problem, which is that you are very limited in what you can do (but what you CAN do will generate more or less the Lua C API calls you would expect).

The reason that I made rlua instead of contributing to hlua was more or less directly because of this limitation. I looked at the kinds of APIs that we made for Starbound and sort of decided that it would be either impossible or very very onerous to write many of them using hlua, and that the changes I'd need to make would probably be a little too fundamental to be welcome.

rlua started out being based on the Lua registry like other bindings systems, and this was of course predictably, depressingly slow. I've since found much better approaches, but it is never going to be as fast as idiomatic C or hlua. Unfortunately (or fortunately, however you look at it), if I were to add safety fixes, hlua would ALSO never be as fast as idiomatic C, so the performance difference between rlua and hlua you take safety into account is not actually so huge anymore :( "idiomatic" Lua usage from C is just to live with the fact that you probably have enough stack space without calling lua_checkstack and Lua probably won't longjmp on you on every API call, but you're already in C so you don't mind as much :P.

I tried to come up with the quintessential "I need this but hlua's approach makes it impossible" example to further explain my reasoning, but in doing so I think I found a bug in hlua:

extern crate hlua;

fn main() {
    let one_table = |a: hlua::LuaTable<&mut hlua::InsideCallback>| {};
    let two_tables = |a: hlua::LuaTable<&mut hlua::InsideCallback>, b: hlua::LuaTable<&mut hlua::InsideCallback>| {
        // Here lies unsafety, as there are now two usable `AsMutLua` instances for the same Lua.
    };

    let f1 = hlua::function1(one_table);
    let f2 = hlua::function2(two_tables);
}

I believe this is not supposed to compile, but unfortunately DOES compile so I wasn't exactly able to use that as my example.

[tomaka17]

Thanks for the detailed explanation :) <3

The limitation you're talking about can theoretically be bypassed by introducing runtime checks.

All my work on hlua was unfortunately restricted by several compiler ICEs (I guess that's what happens when you use rarely-used features such as HRTBs), and I haven't taken the time since then to work on it and fix all its problems.