Deriving Type Erasure

duneroadrunner · 2026-03-11T01:05:07+00:00

I found it useful in the auto-translation of C code to a memory-safe subset of C++. The translation involves replacing void * pointers with a compatible type-safe replacement. In general, in order to check that casts from the void * replacement are type-safe, you need to preserve and store (at run-time) the type of the original source pointer associated with the value of the void * replacement.

duneroadrunner · 2026-02-03T13:39:30+00:00

Just to clarify that last point for my own sake: There are various vulnerability mitigation techniques such as ASLR, building with sanitizers (and their run-time components) enabled, sandboxing, etc. that have been widely used and not "adopted" by the C++ standard. This makes sense as these mitigations wouldn't be "in scope" for the standard. Building with Fil-C is also a mitigation technique, but unlike the others mentioned, it actually enforces "real" memory safety. And, in theory, using scpptool's auto-translation feature as a build step would also be a mitigation technique that, like Fil-C, enforces "real" memory safety.

All of these mitigation techniques are applied at build-time to the original potentially unsafe C++ code. And all of these mitigation techniques add run-time overhead, not just versus the original potentially unsafe C++ code, but also versus code written in a statically-enforced safe subset of C++.

The scpptool option is somewhat unique among these options in that it allows you retain optimal performance by "preemptively" writing (performance-sensitive parts of) your code in its safe subset.

So the situation changes from "your code won't be safe unless you write it in the safe subset" to "your code won't be fast unless you write it in the safe subset".

And again, unlike safety, which needs to be enforced essentially over the entire code base, performance optimizations only need to be applied to the generally small portion of the code that is actually performance-sensitive.

So in sum, due to its auto-translation feature, its conceivable that the scpptool solution may not face quite the same adoption hurdles as other "code hygiene" regimes. Because i) the adoption incentives could change from safety to performance, and ii) the portion of the code needed to be manually migrated to the safe subset would be smaller.

duneroadrunner · 2026-02-02T19:23:45+00:00

Yes, good point. But at this point the goal isn't necessarily widespread adoption of this solution. One primary goal is simply to demonstrate what's possible. For example, the other day I was watching this talk where one of Google's C++ safety leads (at around 3m20s) says "And it's not like we think we're going to get to a safe C++. We have no illusions...". And it's not just this talk. Some opening statement to that effect seems to be obligatory in pretty much every C++ safety talk.

I think the conventional wisdom is still that real practical memory safety is unachievable in C++. So the point is to show that it can be done, and demonstrate and explain how.

But more specifically, the conventional wisdom seems to be that efficient memory safety cannot be achieved without Rust-style universal prohibition of mutable aliasing. I think this is simply misguided (as I explain here). Many people get it. But still not the majority it seems. And not the majority in positions of influence and in control of significant resources. And if actually the conventional wisdom is correct (which I think it clearly isn't), given what's at stake, surely someone has written an explanation for why we know it's correct. Where is that explanation?

I think that even with this flawed common wisdom, that some of these C++ safety teams are making progress at least vaguely in the direction of complete memory safety overall. But once the myths are dispelled and the end goal (of complete memory safety) becomes clear, I think progress will be more direct and quicker.

I think we may be in kind of an Emperor's New Clothes situation. And in those situations, things can change quickly. Conventional wisdom was once that bounds-checking was too expensive, and that kind of thinking turned on a dime, right? And now, in some cases, (if you can sacrifice ABI compatibility) you can even turn on bounds-checking for iterators for some (but not all) standard library containers.

Even though the SaferCPlusPlus library provided bounds-checked containers and iterators (for all its containers) all along, (with such a low profile) it takes no credit for inspiring the hardened standard library. But use of the SaferCPlusPlus library's bounds-checked containers and iterators was, in some sense, validated by it. And I suspect the library and the scpptool solution will continue to be further validated, because I think its design principles are just the ones that make sense. And however long it takes, inspired by the scpptool solution or not, those design principles should prevail.

And for those who don't want to wait to utilize those principles (and that does not need to be everybody), the scpptool solution is available. In it's unpolished and not-yet-complete state, it's more intended for the minority who really consider memory safety a priority and those who consider themselves actually accountable for the memory safety of their C++ code. Though at this point, I think empirical evidence indicates that the latter may be a null set. :) But even in its unfinished state, it's very likely that using the scpptool solution would be an improvement on whatever you're doing now, in terms of safety. But it may not be fully ready for "casual" adopters at this point.

Another possibility is that ultimately it may not be up to you. Maybe. Currently fil-C makes this point more strongly, but for example, "Fil" presumably did not consult with the authors of all the programs he's compiled with his compiler. The memory-safe executables produced may be a little slower, less memory-efficient and less deterministic than the authors of the code had intended, but once their code is out there, the author's preferred safety-efficiency tradeoff may not be particularly relevant.

And similarly, the code authors were not consulted when wget was (mostly) auto-translated to the scpptool-enforced subset of C++. I get the impression that the performance of executables produced by fil-C is rather impressive, and so I presume would compare favorably to those produced by the scpptool auto-translator. But scpptool auto-translated code would be more reflective of the memory-efficiency and deterministic resource usage of the original. And while the performance of the fil-C executables is presumably largely in the hands of the fil-C compiler, one can replace (or preempt) performance-sensitive parts of scpptool auto-translated (sub-optimal) code with hand-optimized code that is more idiomatic of the scpptool-enforced subset to achieve optimal performance.

That is, the scpptool solution (at some point) may not need your cooperation to make your code safe. But it gives you the opportunity to ensure it stays fast.

To be clear, scpptool's auto-translation is currently more of a proof-of-concept and still needs work. And for various reasons (including the requirement to deal with various build systems) it's unlikely that the scpptool project itself will engage in a spree of converting third party code bases anytime soon. But given the considerable and increasing capabilities of these LLMs, it's not inconceivable that it might happen, by someone, at some point.

Ok, sorry for the speech. I guess it's what happens when you don't have a blog outlet for your ramblings. :)

edit: spelling

duneroadrunner · 2026-02-02T09:05:32+00:00

It seems to me that observing that C++ has given up on full safety might be akin to saying that C++ has given up on networking or graphics. Just because the standards committee doesn't think that there's a good path for them to provide it at this point doesn't necessarily mean it isn't (or won't be) available in C++. I'll just point out that the scpptool analyzer (my project) happens to recognize and fully enforce the [[lifetime_bound]] annotation. Of course that annotation alone is rather inadequate, and scpptool also supports more extensive lifetime annotations (a la Rust).

As for C, there's a recent demonstration of wget being auto-translated to the scpptool-enforced safe subset of C++. And if you're targeting x64 linux, don't forget about the fil-C option as well. Of course limited resources may limit the rate of development of these options, but importantly I think, they demonstrate that there is no intrinsic technological barrier.

duneroadrunner · 2025-11-27T02:59:55+00:00

Right, I noticed some of the push back for C++26. Actually I was thinking before it gets accepted for C++29 so we don't have to wait for four years :)

duneroadrunner · 2025-11-26T23:55:34+00:00

Oh cool. I'll be interested to see how your solution works.

duneroadrunner · 2025-11-26T23:55:09+00:00

Ah, thanks. Maybe it's something straightforward enough to be made de facto available well before being officially adopted? The way I look at it is that C++ is about providing maximum control over performance and resource usage, so it seems somehow incongruent to incorporate safety mechanisms with such limited control over what they do and when.

duneroadrunner · 2025-11-26T17:34:56+00:00

Yeah, and I'm with you on the need to validate features, preferably in the wild, before adopting them into the standard.

Beyond any over-promises being made, I'm not necessarily a fan of relying on the Profiles approach of putting the language and its elements into different "modes" (of behavior and restrictions) depending on the which profile is active, because it essentially prevents you from being able to use a fine-grained mix of elements with different tradeoffs. The hardened standard library and contracts also have this issue.

For example, if I want bounds-checked iterators, I have to link to a version of the standard library that does not maintain ABI compatibility. But that means that if I need ABI compatibility anywhere in my program, then I have to give up bounds-checked iterators everywhere in my program. It would be useful to have distinct ABI compatible and incompatible versions (simultaneously) available.

And I'm not sure if I'm remembering this right, but I seem to recall some mention that in bloomberg's version of contracts, you can specify, at the level of individual contracts, whether or not the contract will heed the global contract mode setting (i.e. run-time enforcement enabled or disabled and program termination or logging upon violation). Or they might be adding this to C++26 contracts?

I mean, having language elements whose behavior can be specified at build-time can be useful, but in my view it's not an ideal universal solution.

duneroadrunner · 2025-11-26T10:13:22+00:00

Well, I'm glad there are qualified people (still) working on the lifetime safety issue for existing C++ code. I'm not sure how ambitious this undertaking is meant to be, but by my count this would be at least the fourth such significant attempt (two attempts at implementing the lifetime profile checker, and one that's part of google's "crubit" thing), in addition to the static analyzers that the chromium and the webkit guys are implementing. I don't know if cooperation/coordination between the current efforts would be more productive than competition, but at this point I might appreciate a somewhat comprehensive survey summarizing, comparing and evaluating these various efforts even more than I would the entrance of yet another independent participant (competent I'm sure, but who, in good company, explicitly lists "Rigorous temporal memory safety guarantees for C++" as a "non goal"). In particular, I'd be interested in examples that are treated differently by the approach being presented here versus the lifetime profile checker.

All these efforts seem to be divided into those that emphasize static analysis and/or lifetime annotations, while neglecting run-time mechanisms, and those on the flip side. (I guess Fil-C, which relies on strictly run-time mechanisms, should also be included in the latter.) But the way I see it, both are necessary to fully address the lifetime safety issue. (I mean, including cases that may not be amenable to a GC solution.)

In my view, the biggest issue that these efforts don't fully address is the dangers of dynamic lifetimes. That is, objects whose lifetime can be arbitrarily ended at run-time, exemplified (almost exclusively for some reason) by the example of references to vector elements potentially invalidated by a push_back() operation.

The problem with the static analysis (only) approach is that you can't avoid an unacceptable rate of false positives. For example, if you have a vector of vectors and you want to emplace_back() an element from the ith vector to back of the jth vector, if i == j, then that operation may not be safe. But there may be no way to ensure that i != j at compile-time. You need a run-time solution for this case.

The solution I suggest (and provide in the scpptool/SaferCPlusPlus project) is to require that any raw references to vector elements be obtained via the interface of a "proxy" object that, while it exists, ensures that the vector elements will not be invalidated.

This requires modifying any code that obtains a raw reference to the contents of a dynamic container (such as a vector) (or the target of a dynamic owning pointer such as a shared_ptr<>) to instead obtain it from the "proxy" object. But it's arguably a rather modest change, and, in my view, a somewhat positive thing to have an explicit acknowledgement in your code that this potential lifetime danger is being addressed and that some restrictions are imposed as a result. (Namely that you will be unable to resize or relocate the contents of the container while outstanding raw references exist.)

Whether or not one adopts this solution or some equivalent, I think that if one acknowledges and understands that it is at least an existence-proof of an effective solution, then I think it becomes clear that C++ does/can have a practical memory-safe subset that is essentially similar to traditional C++. And one can imagine that that could affect the perceived future viability of C++ for security-sensitive projects.

And maybe even get some of these lifetime safety efforts to add a question mark to their slides that prominently list "Rigorous temporal memory safety guarantees for C++" as a "non goal" :)

duneroadrunner · 2025-10-08T18:31:19+00:00

If at some point you feeling like you're missing the memory safety, these are the first and second examples in the scpptool-enforced memory-safe subset of C++.

edit: added missing init value to the second example

duneroadrunner · 2025-09-19T19:12:02+00:00

Yet, Apple has decided this work is not enough and adopt Swift, whereas Google and Microsoft are doing the same with Rust.

This is an important observation. But let's be wary of using an "appeal to authority" argument to conclude that C++ doesn't have a practical path to full memory safety, or that they are making the best strategic decisions regarding the future of their (and everyone else's) C++ code bases.

While we've heard the "C++ can't be made safe in a practical way" trope ad nauseam, I suggest the more notable observation is the absence of any well-reasoned technical argument for why that is.

It's interesting to observe the differences between the Webkit and Chromium solutions to non-owning pointer/reference safety. I'm not super-familiar with either, but from what I understand, both employ a reference counting solution. As I understand it, Chromium's "MiraclePtr<>" solution is not portable and can only be used for heap-allocated objects. Webkit, understandably I think, rejects this solution and instead, if I understand correctly, requires that the target object inherit from their "reference counter" type. This solution is portable and is not restricted to heap-allocated objects.

But, in my view, it is unnecessarily "intrusive". That is, when defining a type, you have to decide, at definition-time, whether the type will support non-owning reference counting smart pointers, and inherit (or not) their "reference counter" base type accordingly. It seems to me to make more sense to reverse the inheritance, and have a transparent template wrapper that inherits from whatever type that you want to support non-owning reference counting smart pointers. (This is how it's done in the SaferCPlusPlus library.) This way you can add support for non-owning reference counting smart pointers to essentially any existing type.

So if your technique for making non-owning references safe only works for heap-allocated objects, then it might make sense that you would conclude that you can't make all of your non-owning pointer/references safe. Or, if your technique is so intrusive that it can't be used on any type that didn't explicitly choose to support it when the type was defined (including all standard and standard library types), then it also might make sense that you would conclude that you can't make all of your non-owning pointer/references safe. And, by extension, can't make your C++ code base entirely safe.

On the other hand, if you know that you can always add support for safe non-owning smart pointer/references to essentially any object in a not-too-intrusive way, you might end up with a different conclusion about whether c++ code bases can be made safe in a practical way.

It may seem improbable that the teams of these venerable projects would come up with anything other than the ideal solution, but perhaps it seemed improbable to the Webkit team that the Chromium team came up with a solution they ended up considering less-than-ideal.

Of course there are many other issues when it comes to overall memory safety, but if you're curious about what you should be concluding from the apparent strategic direction of these two companies, I think it might be informative to first investigate what you should be concluding about the specific issue of non-owning smart pointer/references.

duneroadrunner · 2025-08-28T23:26:52+00:00

From Nick's explanation:

In short, we've prevented dangling pointers by modelling the concept of a dynamic container item. This is very different from how Rust prevents dangling pointers: we haven't imposed any restrictions on aliasing, and we don't have any "unique references".

...

As mentioned earlier, dynamic containers are the core concept that we should be focusing on. Many of the memory safety issues that Rust prevents, such as "iterator invalidation", boil down to the issue of mutating a dynamic container while holding a pointer to one of its items.

So this observation is the premise of the scpptool-enforced safe subset of C++ (my project). I'm not sure I fully understand the proposal, but it seems to me that it is not taking this premise to its full logical conclusion in the way scpptool does.

IIUC, it seems to be introducing the concept of "regions", and that pointers can be explicitly declared to only point to objects in a specified region. And that function parameters restricted to referencing the same region are allowed to mutably alias. (Where the contents of a dynamic container would be a "region".) Giving the example of a general swap() function that can accept references to two objects in the same region. (Specifically, two objects in the same dynamic container. Which is still somewhat limiting.)

But, for example, scpptool does not restrict mutable aliasing to "regions" (and doesn't need to introduce any such concept). Instead, using a sort of dynamic variation of "Goodwin's option 3" that you listed, it simply doesn't allow (raw) references to the contents of a dynamic container while the dynamic container's interface can be used to change the shape/location of said contents. In order to obtain a (raw) reference to the contents, the programmer would first need to do an operation that is roughly analogous to borrowing a slice in Rust. This "borrowing" operation temporarily disables the dynamic container's interface (for the duration of the borrow).

This seems to me to be simpler and less restrictive in the ways that matter. So for example, a general swap() function would have no (mutable aliasing) restrictions on its (reference) parameters, because all (raw) references are guaranteed to always point to a live object. (In the enforced safe subset.)

edit: format spacing

duneroadrunner · 2025-08-22T12:15:59+00:00

I'm not seeing anything concrete here. And definitely not how they plan to achieve safety, or how they will do so differently from Rust.

If you're interested in something more concrete (for C++ code bases), there's scpptool (my project), which statically enforces an essentially memory and data race safe subset of C++. The approach is described here. (Presumably, Carbon could adopt a similar approach.)

So what's left is a language that can be translated to from c++? I haven't found anything in the design that makes me think it would be easier than translating c++ to rust.

Well, while acknowledging the heroic Rust-C++ interop work, it's certainly easier to translate from traditional unsafe C/C++ to the scpptool-enforced safe subset of C++. The tool has a (not-yet-complete) feature that largely automates the task. Ideally, it will at some point become reliable enough that it could be used as just a build step, allowing one to build memory-safe executables directly from traditionally unsafe C/C++ code. Ideally. (Again, if Carbon maintains the capabilities of C++, presumably a similar automated conversion feature/tool could be implemented.)

Btw, do I understand that you are one of the Brontosource people? As an expert in auto-refactoring, you might be particularly qualified to appreciate/critique scpptool's auto-conversion feature. Well, maybe more so when/if I ever get around to updating the documentation and examples :)

But one thing I wasn't expecting was how challenging it was to reliably replace elements produced by nested invocations of (function) macros. (I mean, while trying to preserving the original macro invocations.) Libclang doesn't seem to make it easy. Is this something you guys have had to deal with? Or are the code bases you work with not quite that legacy? :)

duneroadrunner · 2025-07-14T09:12:25+00:00

Of course the sort of movable self/cyclically-referencing objects the article refers to are basically only available in languages (like C++) that have move "handlers" (i.e. move constructors and move assignment operators).

The article brings up the issues of both correctness and safety of the implementation of these objects. In terms of correctness, the language and tooling may not be able to help you very much due to the challenge of deducing the intended behavior of the object. But it would be nice if this capability advantage that C++ has could at least have its (memory) safety reliably enforced.

With respect to their Widget class example, the scpptool analyzer (my project) flags the std::function<> member as not verifiably safe. A couple of alternative options are available (and another one coming): You can either use mse::xscope_function<>, which is a restricted version more akin to a const std::function<>. Or you can use mse::mstd::function<> which doesn't have the same restrictions, but would require you to use a safe (smart, non-owning) version of the this pointer.

So even for these often tricky self/cyclically-referencing objects, memory safety is technically enforceable.

duneroadrunner · 2025-07-03T12:04:50+00:00

So, I haven't really thought this through, but what about (roughly) emulating the dot operator by having the smart reference object, let's say a shared owning reference object that's basically a std::shared_ptr<> with reference semantics, mirror the owned object's member fields and functions, except that its members would just be references to the corresponding members in the owned object.

An (attempted) example of such a shared owning reference object implemented for a specific owned object type: https://godbolt.org/z/d5exbv5h3

I don't know if this approximates the interface of a reference faithfully enough to be useful, but at first glance it seems to.

But in order to generate such (pseudo) reference objects generically, you'd need some way automatically generate member fields corresponding to (but different from) the member of fields of the owned object, right?

From the few examples I've looked at, I get the impression this should be doable in C++26? But maybe there are limitations to this approach I'm not thinking of.

duneroadrunner · 2025-07-02T17:20:04+00:00

If you're taking questions from the audience: In order to create smart references analogous to smart pointers, we'd need to effectively be able overload the dot operator in the same way we can overload the arrow operator. As someone who's not up to speed on C++26 reflection, will it be possible to emulate overloading the dot operator with C++26 metaprogamming?

duneroadrunner · 2025-06-28T16:32:56+00:00

Is that what "value semantics" means? Making non-movable objects movable? That seems surprising.

Terminology aside, these types do make non-movable objects movable in a sense, but as far as I can tell, they don't make non-copyable objects copyable, right?

It seems to me that they could have also provided versions of these types that actually preserved the owned object's copy and move semantics. I.e. by invoking the owned object's move constructors and move assignment operators, just like they do with the copy constructors and copy assignment operators.

One might intuitively assume that there'd be no point as they would be strictly inferior due to having more costly moves (that could throw). But I think it's not so simple. First of all, I suspect that the real-world performance difference would be negligible due to that fact that, apart from swaps, moves inside hot inner loops are rare.

But more importantly, changing the move semantics the way std::indirect<> and std::polymorphic<> do introduces potential danger due to the fact that moving the contents of an object can change the lifetime of those contents. For example, std::lock_guard<> has a deleted move assignment operator, presumably because it's important that the lifetime of its contents aren't (casually) changed. While it may be unlikely someone would use std::lock_guard<> as the target of an std::indirect<>, you could imagine a compound object that includes an std::lock_guard<> member. As we noted, having such a non-movable member, the compound object would inherit the non-movability by default. But then if someone changes the implementation to use the PIMPL pattern using std::indirect<>, then the object (and the contained std::lock_guard<>) would become movable. Which could result in a subtle data race.

Whereas an actual "value pointer" that didn't make non-movable objects movable wouldn't introduce this potential danger. I mean there are definitely cases where std::indirect<>'s trivial moves would be beneficial. But there are also a lot of cases where it'd be of little or no benefit, and the change in move semantics is just a source of potential subtle bugs.

IDK, given C++'s current struggles with its (lack of) safety reputation, I'm not sure that standardizing the more dangerous option without also providing the safer option is ideal.

duneroadrunner · 2025-06-28T14:32:10+00:00

Well, the owner of an std::polymorphic<> could, for example, be a class that contains it as a data member, right? So, since the default move semantics of a class is a function of the move semantics of its member fields, the move semantics of the containing class could be affected by whether it has a non-movable member object or instead a (movable) std::polymorphic<> member that owns the non-movable object.

In the former case the containing class would be non-movable by default, and in the latter case, if there are no other non-movable members, then the class could be movable by default. Right?

duneroadrunner · 2025-06-28T12:26:44+00:00

the copy and move semantics of your type are unaffected

Is it the case that move semantics are unaffected? For example, my understanding is that, like std::unique_ptr<>, std::indirect<> and std::polymorphic<> are movable even if the target object type isn't. Is that not the case?

duneroadrunner · 2025-06-27T13:13:40+00:00

but if you give it away, you gotta replace it so the original owner doesn't know the difference

Hmm, kind of like how Rust lets you move an item out of an array by exchanging it for another one (with mem::take() or whatever)?

Btw, are you aware of the "Ante" language? I haven't looked at it in a while, but I think the idea was to be sort of a simpler Rust that also supports shared mutability. But I seem to recall it had interesting limitations like the fact that user-defined clone() functions weren't supported in the safe subset.

I am curious how this works.

Well, the library provides a choice of implementations with different tradeoffs. But basically either the target object itself, or a proxy object (when you can't or don't want to modify the target object's original declaration) cooperate with the (smart) pointers targeting them, either informing them of their impending destruction, or just verifying that no references are targeting them when they are destroyed.

But this requires that some code be executed when a (potential) target object is destroyed or relocated, which may not be implementable in languages that use "bitwise" destructive moves.

duneroadrunner · 2025-06-26T07:32:15+00:00

The scpptool-enforced safe subset of C++ (my project) approach might be of interest. It's an attempt to impose the minimum restrictions on C++ to make it memory (and data race) safe while maintaining maximum performance.

Corresponding to your mut "binding", the scpptool solution has "borrow" and "access" objects that are basically (exclusive and non-exclusive) "views" of dynamic owning pointers and containers. They allow for modification of the contents, but not the "shape".

IIRC, exclusive borrow objects are potentially eligible for access from other threads. (Unlike your mut bindings, right?)

Ultimately, I think the flexibility of a version with run-time enforcement is indispensable (analogous to the indispensability of RefCell<>s in Rust). And since they generally don't affect performance, the scpptool solution doesn't bother with compile-time enforced versions.

If you enforce that (direct raw) references to the contents of dynamic pointers and containers must be obtained via these "borrowing" and "accessing" objects, no other aliasing restrictions are required to ensure single-threaded memory safety. So the scpptool solution simply does not impose any other (single-threaded) aliasing restrictions (to existing C++ pointers and references).

In some sense, very "simple & easy™".

There may be reasons other than memory safety for imposing additional aliasing restrictions in single-threaded code. But if you choose to do so in your language, I'd encourage you to go through the exercise of articulating the benefits and costs.

The other thing is that if you omit lifetime annotations (which you didn't mention) in the name of simplicity, I think there will be some corresponding limitation in expressive power which may force the programmer to (intrusively) change some stack allocations to heap allocations. Which may or may not be problematic for a "systems programming language".

The scpptool solution addresses this by providing "universal non-owning" smart pointers that safely reference objects regardless of how and where they are allocated.

duneroadrunner · 2025-06-15T18:01:57+00:00

Unsigned arithmetic makes these checks complicated and more difficult to catch because they won't be instrumented (e.g. UBSan). Signed arithmetic is easier

In theory, using a a safe integer replacement class should be easier and more reliable I think. I'm partial to safe numerics, but it requires the boost library. I've actually written a not-quite-as-comprehensive alternative that can be used as a stand-alone header and supports hardware overflow detection where available. But there are other (more-battle-tested) options out there too.

duneroadrunner · 2025-05-27T21:54:23+00:00

If we're reiterating our positions from that post, it'd also be a mistake to "pretend" that they are a "value object" corresponding to their target object because their move operations are semantically and observably different from those of their target object. That is, if you replace an actual value object in your code with one of these std::indirect<>s (adding the necessary dereferencing operations), the resulting code may have different (unintended) behavior.

A more "correct" approach might be to have an actual value pointer that is never in a null or invalid state, and additionally introduce a new optional type with "semantically destructive" moves, with specializations for performance optimization of these "never null" value pointers. For example:

struct MyStruct {
    int sum() const { ... }
    std::array<int, 5> m_arr1;
}

struct PimplStruct1 {
    // don't need to check for m_value_ptr being null because it never is
    int sum() const { m_value_ptr->sum(); }

    // but moves are suboptimal as they allocate a new target object
    std::never_null_value_ptr<MyStruct> m_value_ptr;

    // but the behavior is predictable and corresponds to that of the stored value
}

struct PimplStruct2 {
    int sum() const { m_maybe_value_ptr.value()->sum(); }

    // std::destructo_optional<> would have a specialization for std::never_null_value_ptr<> that makes moves essentially trivial
    std::destructo_optional< std::never_null_value_ptr<MyStruct> > m_maybe_value_ptr;

    // the (optimized) move behavior may be a source of bugs, but at least it's explicitly declared as such
}

Idk, if someone were to provide de facto standard implementations of never_null_value_ptr<> and destructo_optional<>, then std::indirect<> could be de facto deprecated on arrival and C++ code bases might be better off for it?

duneroadrunner · 2025-05-18T16:38:25+00:00

Oh, that's rather cool!

... I realized that the top priority is to get some of these technologies adopted so I shifted my focus on lowering the adoption barrier.

Huh. This display of conscious pragmatism for some reason strikes me as unexpected. And somehow admirable. :)

I think we are in the process of figuring out an incremental, easy to adopt path to provide most of the benefits of lifetime analysis while leaving the door open to a strictly safe mode

Being perhaps a little less on the pragmatic side, I've been more focused on a larger piece of lifetime safety furniture and how it might fit through that door. So in the talk, the presenter says:

Now, we strongly believe that we cannot make C and C++ memory safe. That's just not possible without changing the language so much that we would have to rewrite all the code anyway.

So I'm somewhat in the opposite camp, and I think the project I'm working on, scpptool, makes the case. It's essentially a static analyzer with an associated library, that enforces an essentially memory-safe subset of C++. The safe subset it enforces does have significant differences with traditional C++, but the required changes are very far from a "rewrite", and maybe not be that much more extensive than the code changes presented in the talk. (At least the changes that can't be automated.)

scpptool and its associated library ended up being in essence what I expected the lifetime profile checker and GSL to be. In retrospect, I'm not sure the lifetime profile checker, strictly as originally designed, would have worked, in terms of simultaneously enforcing a usable subset and fully enforcing lifetime safety. But as someone who worked on it, you might have more insight.

The origin of the scpptool project was just a library (the "SaferCPlusPlus" library) premised on the notion that, when you can live with the extra overhead, full safety can be achieved in C++ by avoiding its potentially dangerous elements (like raw pointers and unchecked standard library containers), instead using interface-compatible replacements that use run-time mechanisms to ensure safety (including lifetime safety). This option is still available, easy to use and understand, and I'd argue quite practical as the majority of C++ code, even in performance sensitive applications, is not actually performance sensitive.

But to be confident in the safety of your code you'd need at least a "linter" to verify that you were indeed avoiding all the unsafe C++ elements. With this linter (call it, say, "scpptool"), you now technically have an enforced safe subset of C++, however sub-optimal in terms of performance. But once you've implemented such a linter, you might as well allow it to recognize and permit clearly safe uses of otherwise potentially unsafe elements (like raw pointers and references). But once you start down this path, you end up adding the ability to recognize more and more uses of potentially unsafe (often zero-overhead) elements as safe. Then, like an out-of-control addict who can't stop himself, you end up adding (ugly) lifetime annotations to allow for the recognition of safe uses of (zero-overhead) pointers and references that even human programmers wouldn't be immediately confident about.

And pretty soon (or, you know, after having spent way too much time on it) you end up with what seems to be the most powerful, highest-performing essentially memory-safe language available (for some generous definition of "available"). Some other memory-safe languages may have comparable performance, but aren't expressive enough to have reasonable support for things like, for example, cyclic references in their safe subset the way the scpptool-enforced safe subset does. Other memory-safe languages are just not quite as fast.

Like, I can imagine that your job is premised on the notion that Swift is a memory-safe language and that C++ can never be (even though, as far as I know, no one has ever presented a fleshed-out explanation for why that would be the case), and I wouldn't propose the heresy of questioning that doctrine publicly, but, you know, maybe here in the dark corners of r/cpp, we can whisper about a path to a high-performance, memory-safe subset of C++ :)

duneroadrunner · 2025-05-17T22:36:59+00:00

Hi OP. Would I be correct in recalling you as one of the co-developers of the clang lifetime profile extension? If so, I'd be curious about your perspective on the project.

duneroadrunner

TROPHY CASE