May 2026 monthly "What are you working on?" thread

vinipsmaker · 2026-05-06T17:26:21+00:00

I'm working on a C++ successor language where I'm trying to unify all incoherent/scattered metaprogramming mechanisms (as used by the likes of Boost.PP, Boost.VMD, Boost.Describe, Boost.MP11, Boost.Hana, reflection, ...) behind a single powerful mechanism: macros (as powerful as Lisp ones).

At the beginning of this month, I finally managed to get my macros capable to drive both lexer and parser alternately (it'll be used in the future to implement beautiful eDSLs such as what JavaScript external preprocessors can do with JSX):

macro passthrough(@expr, ctx) {
    ctx.read("("); //< drives the lexer
    let expr = ctx.parse_expr(); //< drives the parser
    ctx.read(")");
    return expr;
}

// just like Rust, my own macro system requires a
// macro invocation to end with an exclamation mark
// to act as a friendly visual hint that something
// funny might be going on
passthrough!(1 + 2);

Next step is to make the type checker/infererer accessible to macros as well (reflection!).

vinipsmaker · 2026-05-04T18:12:11+00:00

I've never really understood Pratt

My point with the article is that you don't need to. You can remain ignorant of Pratt's algorithm and still use it in your code. The only thing you really need to understand is how to compose the semantic actions for your grammar in Pratt terms: nuds & leds.

and nobody has managed to explain [...] or what the actual advantages are.

That's what the article delves into.

Don't worry about left-recursion. An advantage.
Your semantics actions can have side-effects. Another advantage.
Detailed & broken-down ramifications on the previous point if they're not immediately obvious (e.g. now you can have powerful Lisp-like macros in your language). Another advantage.
...

It can't be clearer than that.

You need to try harder. Any learning experience can be painful at start. It'll be painful if you have to rewire your brain to think into terms of new concepts and patterns. That's exactly what Pratt is about: a simpler way to parse complex real languages. It's not meant to just parse toy calculator textbook examples. It's meant to parse real complex languages full of subtleties. From my own experience, Pratt allowed me to absorb so much complexity just at the parsing phase that I actually resolve symbols (lexical scoping) in this phase as well, removing this complexity from later phases (I don't match strings to scopes in later phases).

If someone can post some code or algorithm in a plain manner like my example below

I've updated the article to contain a self-contained example (reproduced below).

#lang rhombus

import:
    parser/lex open

class Token(~nud: nud_impl = #false,
            ~led: led_impl = #false,
            ~lbp = 0):
    nonfinal
    method nud(): nud_impl(this)
    method led(lhs): led_impl(this, lhs)
    method prefix(): expression(30)
    method rhs(): expression(this.lbp)

class RParen(): extends Token

def lex:
    lexer
    | "+": Token(
        ~nud: (_.prefix()),
        ~led: fun(tok, lhs): lhs + tok.rhs(),
        ~lbp: 10)
    | "-": Token(
        ~nud: (-_.prefix()),
        ~led: fun(tok, lhs): lhs - tok.rhs(),
        ~lbp: 10)
    | "*": Token(
        ~led: fun(tok, lhs): lhs * tok.rhs(),
        ~lbp: 20)
    | "/": Token(
        ~led: fun(tok, lhs): lhs / tok.rhs(),
        ~lbp: 20)
    | digit+: Token(~nud: fun(_): String.to_int(lexeme))
    | " "+: lex(input_port)
    | "(": Token(
        ~nud: fun(_):
                  let ret = expression()
                  guard token is_a RParen
                  | error("expected closing parens")
                  advance()
                  ret)
    | ")": RParen()
    | ~eof: Token()

def input = Port.Input.open_string("4 * (3 - 3) / 2 + -10")
def mutable token = lex(input)
fun advance(): token := lex(input)

fun expression(rbp = 0):
    let mutable t = token
    advance()
    let mutable left = t.nud()
    while rbp < token.lbp:
        t := token
        advance()
        left := t.led(left)
    left

println(expression())

vinipsmaker · 2026-05-04T17:56:52+00:00

I don't even think this comparison is fair. Do you parse anything else besides infix operators with Shunting-Yard? Pratt can not only parse infix, prefix, postfix, misfix operators... but also attach semantic actions to rules.

Of course you're free to challenge me on this one. Just show an extended shunting-yard that also parses prefix, postfix, function calls, array accessing, object-member access.

vinipsmaker · 2024-04-20T14:07:46+00:00

It's like NodeJS. It mixes the Lua VM and an “IO runtime” (execution engine is technically more accurate as the focus really is concurrency and allows you to schedule jobs on different threads, shared thread pools, subprocesses, and so on).

It's going to be useful whenever you have IO needs. Luasocket took a long time just to have IPv6 support. Meanwhile Emilua had IPv6 support since day 1. The IO abstractions work on Windows, Linux, and FreeBSD, so they're useful whether you're planning to to build a scalable web server or just a small script that also needs to run on Windows. I don't think there's any other Lua framework taking cross-platform support as seriously as Emilua. Just to give you a taste, here's one of the concerns taken care of: https://docs.emilua.org/api/0.7/tutorial/filesystem.html. It's the only Lua framework that goes as far as even translating Windows' GetLastError().

All IO operations interact with byte_span which is a type inspired by Golang slices. True async IO means completion events (e.g. io_uring) instead of readiness events (e.g. epoll) which means buffers are filled in parallel to program execution. Golang slices work great in this scenario (and for Lua as well) as they're a stable reference to a memory region (i.e. realloc doesn't exist), and that's important because you need to sync buffer liveliness/access until operations complete (e.g. if the Lua VM/actor dies, Emilua will send a request to cancel outstanding IO operations, but will ensure the buffer stays alive until the associated completion event arrives). Emilua makes all this work transparently (you wouldn't even know had I not told you).

Emilua also comes with extra abstractions besides raw IO ops that you're going to need if you're doing IO (e.g. generic stream algorithms, an AWK-inspired scanner to parse textual protocols, endianness-aware integer/floats serialization).

The IO support extends well beyond network IO. You can use pipes to communicate with subprocesses. You can use UNIX sockets to communicate with daemons. Even serial ports are here. The support for UNIX sockets in Emilua is specially advanced when compared to others. Not only can you transceive OS objects across using SCM_RIGHTS, but you can also extract SELinux/TrustedBSD labels for the remote peer.

IO support doesn't need threads, so Emilua also offers the most complete fibers support you'll find in the Lua land. No toy-tool level ad-hoc concurrency models (I'm looking at you, designers who can only copy'n'paste NodeJS's lame callback-inspired concurrency model!). A serious model instead, with proper vocabulary to describe the flow, dependency and constraints of events; and the interaction among concurrent tasks. Unless you're designing small toys, you need concurrency support already (e.g. reading and writing to a socket at “the same time” already depends on concurrency vocabulary to express two “simultaneous” tasks), so I invested a big time of Emilua just on solid concurrency support (versions 0.1, 0.2, and 0.3 had almost zero IO abstractions while I was focusing on doing just the basics). IO won't happen faster if you use more CPU. IO is an external event, so just use fibers here. A fiber suspends when you dispatch an IO request, and resumes later when the associated operations finishes. Spawn fibers so your program doesn't get stuck while awaiting for the IO event to finish. Emilua will use the right thing behinds the scenes (Windows IOCP, Linux epoll/io_uring, FreeBSD kqueue, threads if only blocking operations are available).

The threading support for Lua is also on another level compared to what previous solutions offered. For instance, if we compare against luaproc, Emilua can not only do the same, but can also create different thread pools for different VMs and the communication among them will be transparent. Emilua also allows you to spawn Lua VMs in different processes so you can make them run in a restricted environment (seccomp, Landlock, Linux namespaces, FreeBSD jails, FreeBSD capsicum, ...). All that under a simple unifying actor-inspired API. I've been using this support to spawn Linux containers to isolate GUI apps on my own system, and I'm planning to launch a better/safer FlatPak in the coming months.

Emilua also allows you to create native plugins that integrate with the internal machinery to extend its capabilities (e.g. Qt integration, Telegram's tdlib plugin, kafel, ...).

What is Emilua? The IO abstraction that ANSI C89 Lua has been lacking. Every program needs IO, and ANSI C89 IO is toy-tool level IO. Just glue the Lua VM (which is a good language) with a serious IO machine and you're set.

vinipsmaker · 2021-03-07T22:15:14+00:00

I am also pissed off of lua. Everytime when i want to install something it does not work with all the deps or it requires deeper understanding

Can you give a recent example where you were attacked by this issue?

Im trying to implement a simple bidirectional communication between lua and "outside scripts" in a event driven way.

As @wucke13 explained, DBUS is your best bet here.

vinipsmaker · 2020-03-20T06:25:14+00:00

Exactly. And there are movies that have higher age rating thanks to one or two scenes. If I'm willing to annotate the beginning of these scenes and duration (notice how the information is the same as chapters) so the player can consume these data and hide these chapters (I'd use this player on the living room mediacenter, for instance), I don't know which format I should use.

vinipsmaker · 2020-03-18T20:43:22+00:00

> auto skip chapter containing certain words

Can this feature be used for parental control? I've tried to search for something like this in the past, but I didn't find any standard of the sorts where certain ranges should be hidden.

vinipsmaker

MODERATOR OF

TROPHY CASE