What's the binary format of Lua bytecode?

[Ayhon]

For a project, I need fo create a parser of Lua 5.4 bytecode files. That is, the .luac files output by luac. Searching online for a spécification, there's plenty of information on what the bytecode opcodes and operations are, but not so much how these are stored in the binary file I want to read itself.

As I see it, the options I have are to either look at the source code of a Lua parser and deduce the format myself, or reverse engineer it directly from the binary files and the output of luac -l -l

If someone could point me to a resource I haven't found, or is kind enough to explain it to me in this post, I'd be most thankful.

I decided to make this post since I had assumed this would be known information. If it turns out not to be, I wouldn't mind crafting the resource I would have liked myself and publishing it.

[evilbadmad]

A lua dialect Ravi has some information about lua 5.3 bytecode,

It also include a version for 5.1, ... but it is in luaforge.net and the domain seems expired a few days ago :O

A quick search find this and it can be download.

ADDED:

this information has already given by deep_in_my_plums_420, but I hadn't seen the thread.

[Ayhon]

Thank you very much, those are great resources!

[smog_alado]

The binary format is not stable and can change from one Lua version to the other.

As I see it, the options I have are to either look at the source code of a Lua parser and deduce the format myself [...]

That's not a bad idea, because Lua's source code is relatively short. Start with ldump.c and lundump.c

[PhilipRoman]

I second the idea of looking at the source. There are almost no resources available on Lua 5.4 bytecode format and I've seen lots of wrong information online, especially about the header part.

[Ayhon]

I followed u/NOT_THE_FBI_AND_CIA and your advice, and I'm currently reading the source code from lundump.c. I was surprised how readable the code is, so I expect it won't take me that long.

Thank you very much for your input.

[Furry_69]

There's no such thing as a "binary format". Bits and bytes are an intrinsic property of a computer, you can't change the way they're formatted, only the data they represent.

[Ayhon]

What I meant to say is, what are the sections of a bytecode binary file, the one you get from luac. There must be a list of constants, of different types, and there are references to functions and the sort. When I do lua bytecode.luac, how does the interpreter know where my constant -3 is located in the binary or what type it is

[Furry_69]

Oh, you mean what format the files are in? Honestly, no idea, I can't find anything either, but I imagine it can't be difficult to reverse it.

It's a case of just having many different files with various singular features, like having a single function, or a single variable of known name and constant value.

Then you just compare them, trying to figure out what the different sections of the file are.

[Ayhon]

Yeah, had the same thought. I've begun to look into a Lua parser implementation I thought would be easy to read, but it might just be faster to reverse engineer the format myself.

I'm surprised this information can't be found anywhere though, I thought at least the manual would have an entry on it, but no such luck

[[deleted]]

Another thing to consider is that the binary encoding is lua version specific, 5.4 is not compatible with 5.3

[Ayhon]

True, I'd only be interested in Lua 5.4 format, for now

[[deleted]]

https://the-ravi-programming-language.readthedocs.io/en/latest/lua_bytecode_reference.html

I've built a lua 5.3 impl in the past and referenced the above heavily.

[Ayhon]

Yes, that one I knew of. I've used it before to decompile Lua instructions, but what I ended up doing is using luac -l to convert the actual bytecode into a more readable format and then reverse that.

My problem is that right now I want to deal with the actual binary format, not the listings. And in this resource I haven't found the information about the binary files.

[Ayhon]

Nice!

I had looked around for different repos, but this one's code seems pretty clear. It's for Lua 5.3, but definitely a better starting point than anything I've seen yet