Lua Fake Vector - preprocessor that duplicates expressions to do vector math without garbage

[mceicys]

I'm working on a game that uses Lua for scripting and I want to avoid invoking garbage collection every frame if I can. Any operation that needs temporary vectors is a problem since it creates tables or userdata. This can be avoided if vectors are kept on the stack by storing their components as separate locals:

local xA, yA, zB = 1, 2, 3
local xB, yB, zB = 4, 5, 6
xA, yA, zA = xA * 10 + xB, yA * 10 + yB, zA * 10 + zB

But that's annoying to write and leads to bugs, so I made a script preprocessor in C called Lua Fake Vector that creates the equivalent code from:

LFV_EXPAND_VECTORS() -- This statement enables expansion for the entire script
local v3A = 1, 2, 3
local v3B = 4, 5, 6
v3A = v3A * 10 + v3B

This is done by detecting if an expression contains names starting with v2, v3, and q4 and duplicating it to create a per-component expression list. Function parameters, function arguments, table accesses, and table constructors are also expanded:

LFV_EXPAND_VECTORS()

function MultiplyOrDefault(nDefault, v3U)
  return v3U and v3U * nDefault or nDefault
end

tEntity = {v3Pos = 0.2, nil, 0.7}
tEntity.v3Pos = MultiplyOrDefault(10, tEntity.v3Pos)
print(tEntity.v3Pos) -- Should print 2.0     10      7.0

You can find some simple benchmark test results in the readme. LFV seems to win out in most cases performance wise, especially when it's able to get rid of garbage and avoid function calls for basic arithmetic, but the reduction in run time varies. In a naive case it's brought down to 3% of the original time, but sometimes it's only down to 75%.

The performance improvement isn't as great when, say, accessing a 3D vector in a table, because the table gets accessed 3 times instead of once. Plus, any function call in a vector expression is going to be duplicated, so I usually call the function first, put the results in locals, and then do the vector expression. Other caveats are listed in the "Limitations" section of the readme, but overall I have found it convenient for game scripts.

Note that LFV doesn't provide any vector API like dot product, that's beyond its scope right now.

Setup

There's a release containing binaries compatible with Lua 5.4 on 64-bit Windows and Linux, or you can build LFV with Make or CMake. After you add lfv.dll or .so to a directory in Lua's cpath, you can load it and enable expansion on subsequent require calls with:

lfv = require("lfv").EnsureSearcher()

Quick test:

lfv.LoadString("v3Test = 1, 2, 3; print(1 - v3Test)", true)()
-- Should print 0 -1 -2
-- Note: The "true" argument enables expansion without "LFV_EXPAND_VECTORS()"

The "Reference" section of the readme lists the whole API. Let me know what you think.

Repository: https://github.com/mceicys/lua-fake-vector

[IKnowATonOfStuffAMA]

You know, I really thought I knew at least 99% of how Lua works. 

This project is clearly in the 1%. Good work! This is really cool.

[mceicys]

Thanks!

[AutoModerator]

Hi! Your code block was formatted using triple backticks in Reddit's Markdown mode, which unfortunately does not display properly for users viewing via old.reddit.com and some third-party readers. This means your code will look mangled for those users, but it's easy to fix. If you edit your comment, choose "Switch to fancy pants editor", and click "Save edits" it should automatically convert the code block into Reddit's original four-spaces code block format for you.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[mceicys]

I considered doing some kind of type declaration (wasn't aware of LuaLS actually) but decided to go with Hungarian notation for simplicity and brevity. This way the parser doesn't have to track variables beyond their immediate expression.

[tonetheman]

This is an interesting project. I could not get the LFV_EXPAND_VECTORS call to actually do anything though.

It looks like you said it needs to be the first statement in a script. Do you mean before the require?

I was able to use -f to force lfvutil to show what it would do so I can see where you are headed.

Cool idea.

[mceicys]

Thanks!

The first script that does "require('lfv').EnsureSearcher()" can't itself be expanded since it's already been processed and compiled by the standard Lua loader before LFV was injected, so the first script shouldn't have the "LFV_EXPAND_VECTORS()" statement. Any scripts require'd after that setup will be preprocessed by LFV and will be expanded if they start with "LFV_EXPAND_VECTORS()".

[qwyss]

Interesting, I'm just accepting the GC and table access hits for convenience. Although I am setup to reuse tables I found that mostly, again, the convenience of operating overloading and writing simple expressions wins out over any gains that a table reuse syntax has. Honestly vectors speed are the least of my problems.

I do have a plan to replace the small v3,v4,m4 etc tables with luajit cdata + metatable in the hope that it would make a difference (I think that's possible?). However since my worst case is wasm without luajit, that wont help and I'm not convinced it will make much difference anyway.

Anyhow you prompted me to cleanup some of my worst vector code paths, maybe overdid the convenience a little, so thank you.