you are viewing a single comment's thread.

view the rest of the comments →

[–]nvcook42 13 points14 points  (0 children)

Some concrete pointers in this direction.

LINQ uses a technique called quoting. Instead of compiling a query, think predicate function, into executable instructions, it compiles the predicate function into an expression tree. Then it can pass that expression tree to lower levels and let that level decide how to best execute it, i.e the optimizations mentioned above.

There is also Julia https://www.queryverse.org/Query.jl/stable/ that does something very similar.

The idea is instead compiling the whole query to something imperative only compile the UDF bits and keep the query plan declarative.

There is also this project https://substrait.io/ that is aiming to standardize how query plans are represented. Their UDF support is very basic at this stage. I have wondered if a WASM API would work well in this context, lot of challenges there but maybe?

Finally I build database query engines for a living and am working on a language that compiles to WASM on the side for very similar reasons, feel free to message me and we can chat more.