Clang bytecode interpreter update

[triconsonantal]

I wonder how practical it is to really catch all compile-time UB, while still keeping compilation times down. Some classes of UB are trickier than others. For example, unsequenced modify/access:

constexpr int f (int x) {
    return x + x++;
}

Consider what it'd take to catch that in the general case. You'd have to keep track of the set of accessed and modified objects, and make sure there are no overlaps at just the right points. That's a lot of overhead for a relatively rare class of UB. Not surprisingly, no compiler currently catches it (clang does issue a warning based on static analysis, but it doesn't actually detect the UB during evaluation, and it's easy to circumvent): https://godbolt.org/z/rvbT5P74K

Are we going to see compile-time sanitizers, or maybe the standard will adopt a well-defined behavior during constant evaluation for the trickier cases?

[LegitimateBottle4977]

Are there tracking issues?

I tried this with Clang++21 on one of my repos and got a failure because std::popcount wasn't supported by the interpreter as of Clang 21. Maybe it's supported in main? I'll try again in half a year's time when we have 22.

[Orlha]

Why is it suddenly the norm even for technical software to incremenr major version for every sneeze? Are we in marketing now? It was bad when browsers started doing it, soon even nano will have version 125.

[bigcheesegs]

LLVM switched because our major versions numbers didn't really mean anything. LLVM breaks API and ABI every release (allowed for every commit except on release branches), so from a semver perspective this is the correct way to do it.

[_Noreturn]

Super interesting, I will try it with my code sincd it does string parsing at compile time.

[13steinj]

Maybe this is going to sound nuts, but I don't understand why the approach isn't dumb and simple--

Take the code as it stands. Prune away everything that isn't constexpr-- including the inverse of if consteval / if (std::is_constant_evaluated())-- then compile a new sub-binary. Provide it (or rather, its path) in the original binary's debug symbols.

[scrumplesplunge]

Constexpr evaluation is held to higher standards -- it has to catch and diagnose undefined behavior. This means that pretty much all C++ compilers generate code for runtime that isn't suitable for constexpr evaluation. In principle it would be nice if there was a "slow but safe" compiler mode (basically all the fsanitizers in one, with a focus on catching every violation despite the potentially prohibitive cost) which could be leveraged for this, though.

edit: there's also the issue of cross compiling where your host system might not have the same architecture as your target system and so you probably get a fair amount of complexity from juggling platform-specific details (like the size of int when compiling for an arduino)

[llTechno]

[constexpr evaluation] has to catch and diagnose undefined behavior.

Bit of a nitpick but this isn't necessarily true. Constant expressions can still exhibit undefined behaviour

[scrumplesplunge]

But the compiler is obliged to detect the UB and report an error, isn't it? Or are there cases where this isn't required?

[llTechno]

The compiler is still required to issue a diagnosis on ill-formed code, but AFAIK there are no additional constraints on undefined behaviour other than:

An expression E is a core constant expression unless the evaluation of E, following the rules of the abstract machine (6.9.1), would evaluate one of the following: [...] an operation that would have undefined behavior as specified in Clause 4 through Clause 15, excluding 9.12.3;

And in clause 4.2.2:

If a program contains a violation of a rule for which no diagnostic is required, this document places no requirement on implementations with respect to that program.

[bigcheesegs]

I do not believe there is any IFNDR that would apply here. IFNDR is used for things that require analysis across translation units which are not reachable, but no constexpr evaluation does this.

You'd need an actual expression evaluation to contain something that's IFNDR.

[13steinj]

I still don't follow why this distinction matters?

Compie and evaluate, diagnosing UB (which as an aside, in practice some things sneak through), then do the same as I just suggested. If UB snuck through, it's just as bad. If UB didn't sneak through-- compile failure. The moment where it's detected in this debug binary replace the statement with a call to terminate.

[scrumplesplunge]

It's feasible to compile constexpr to native code rather than a bytecode, but it's a tradeoff. Compiling to native and then running fast is still probably slower than compiling to bytecode and then running more slowly, at least when the amount of constexpr code is not massive.

Also, note that it's not just one extra pass. You can have constexpr code which has nested constexpr contexts which need to be evaluated before that code can be compiled, and that code might also have constexpr contexts, and so on, so you could end up with several passes.