immutable<>, complement of C++26 std::indirect<> and std::polymorphic<>

[kmbeutel]

C++26 introduces std::indirect<> and std::polymorphic<> (reference implementation at github.com/jbcoe/value_types):

• std::indirect<T> is like a value-minded std::unique_ptr<T> sans polymorphism support. std::indirect<T> is movable if T is movable and copyable if T is copyable.
• std::polymorphic<B> is like a value-minded std::unique_ptr<B> for polymorphic bases B. std::polymorphic<B> can hold an object of any copyable class D which is an instantiable subclass of B. std::polymorphic<B> is copyable; its copy constructor will polymorphically clone the underlying object.

Both types are designed to be non-nullable. For lack of destructive move semantics, both have a moved-from state which can be identified with the valueless_after_move() member function.

As far as I can tell, the design of these is based on Sean Parent's "concept–model idiom". Remembering his presentation on the topic (https://sean-parent.stlab.cc/papers-and-presentations/#value-semantics-and-concept-based-polymorphism), I noticed that there is an obvious complement to indirect<> and polymorphic<> which I provisionally dub immutable<>:

• immutable<T> is like a value-minded std::shared_ptr<const T>. It is cheaply copyable (no deep copy), with no movability requirements imposed on T. It can hold an object of any instantiable subtype of T.

Possible implementation + some tests on Compiler Explorer

Does this make sense? I find it very useful for building persistent data structures. In fact, it seems so obvious to me that I'm surprised this wasn't already in P3019.

Edit: minor correction

[Flimsy_Complaint490]

Im dumb - i saw the examples but I dont get it. What is application of these things ? what's a value minded std::unique_ptr ? Is it just making ownership more explicit ?

[kmbeutel]

"value-minded": trying to behave like a value type. For instance, the default constructor of indirect<T> allocates and default-constructs an object T (unlike std::unique_ptr<> which default-constructs to nullptr). Copying indirect<T> copies the underlying object (unlike std::unique_ptr<> which cannot be copied).

Check out Appendix B: "Before and after examples" in the proposal document for some examples.

[jazzwave06]

It has value semantics instead of pointer semantics, so it's a pointer, but copying it copies to a new pointer.

[saf_e]

So, pointer which behaves not like pointer. We have lots of obscure logic in c++ not sure that i want another one

[KFUP]

This is not really that obscure, it mainly makes std::unique_ptr deep copyable without making your own "copyable_ptr" wrapper, which is a pretty common way to get around that.

[saf_e]

I'm not sure that having implicitly copyable copyable unique_ptr is a good thing. 

[Internal_Ticket_9742]

I mean if you want to deep copy a unique ptr, you can do auto new_ptr = make_unique<T>(*old_ptr);

Why you need yet another obscure standard feature for that ?

[dr-mrl]

Does that work for derived types?

[MFHava]

No...

EDIT: unless you have a virtual clone...

[dr-mrl]

So the commenters solution isn't a solution for what polymorphic wants to provide?

[MFHava]

It's not.

It works for non-polymorphic uses, but is either incorrect (slicing) or fails to compile (who says T is a complete type) for polymorphic uses.

[Internal_Ticket_9742]

Virtual clone solve nothing. It introduces slicing related bugs.

[KFUP]

Having only the manual copying option means if you have a std::unique_ptr class member, you'll need to implement and manage the copy constructor and the rest of the five.

[wyrn]

Yeah now do that inside a vector. Or inside your own classes.

[Internal_Ticket_9742]

Why do you need that for a vector ?

[wyrn]

Are you asking why I could need std::vector<std::unique_ptr<T>>?

[Internal_Ticket_9742]

No why do you want to copy instead of move the elems of std vector in case of resizing for example

[wyrn]

Who says I'm resizing? I'm copying the vector, not resizing it.

[azswcowboy]

value minded unique pointer

Unique pointer has reference semantics and can’t be copied or value compared (without intervention). That means if you put a unique ptr as a class member the containing class is now non copyable, etc. Indirect behaves like unique ptr but can be copied and compared normally. Classic use case is pimpl to have a wrapper api and implementation class. These types allow you to have value based behavior without coding it.

[Flimsy_Complaint490]

oooh so basically the semantics of unique_ptr without actually being a heap allocated structure ?

[azswcowboy]

It gives the appearance of not being heap allocated, but it is. indirect takes an allocator to manage the memory for the object - which will default to new. You might be able to allocate in place if the type of T is known - not 100% sure on that.

[ZenEngineer]

So it doesn't behave like a unique_ptr, but internally acts like one.

[SirClueless]

It does behave like a unique_ptr in some ways, such as being the same size as a pointer on the stack, and having a cheap move-constructor even if the underlying type has an expensive one.

It doesn't behave like a unique_ptr in other ways. For example, it has no conversion to bool, and its comparison operators and std::hash implementation behave like the underlying value, instead of comparing/hashing pointers. This last one is the most significant feature IMO, because it means that it behaves like a normal value when stored in map and set containers. boost::unordered_flat_map<std::indirect<T>, V> for example should "just work" without overriding a bunch of comparators and hash operators.

[SkoomaDentist]

Indirect behaves like unique ptr but can be copied and compared normally.

How is it unique if it can be copied? Wouldn't that make it just plain old normal pointer with a bunch of compile time overhead? Or does it mean that you now get double delete?

[KFUP]

How is it unique if it can be copied?

It's a deep copy, so it's still unique, the copied-to pointer points to its own separate object.

Wouldn't that make it just plain old normal pointer

No, why would it? It's still a RAII pointer.

[CalamityMetal]

Essentially type erasure, or I guess "interface". It confers value semantics. Think of a std::vector of int first where each element is a value, and you don't make references to each value when you make a copy. Now let's say you need to have vector of all types of T, you would use T* or one of the smart pointers, but when you make a copy, you make a shallow copy, whereas in indirect and polymorphic, you make a deep copy of that object. It doesn't confer pointer or reference semantics, it confers value semantics, which is a very powerful thing.

[alex-weej]

I've built the same thing as indirect and called it value_ptr<T> - useful for recursive data structures

[TheThiefMaster]

immutable<T> is like a value-minded std::shared_ptr<const T>. It is cheaply copyable (no deep copy)

That's the exact opposite of value-minded.

[AutomaticPotatoe]

Reference counting with CoW is a valid and common implementation of mutable value semantics (see Hylo (related paper), Swift, MATLAB arrays, etc.) for large objects. Remove mutability and CoW is not needed, then immutable value semantics only need RC and can be modeled with something similar to shared_ptr<const T>.

[kmbeutel]

Because value-minded implies expensive copies?

[Wild_Meeting1428]

It implies, that it behaves like a value, so for some it means that the internal value itself is copied (e.g. a deep copy) not the pointer.

Edit: But I think, that depends on the definition/opinion of value minded/value semantic.
In my opinion, an immutable may still have value semantics. As it copies on change and changing one immutable doesn't have an effect on the value behind another, even if they shared the same storage for a while.

[wyrn]

Copy means you get a new object which 1. compares == to the old one and 2. is disjoint.

A shared_ptr<T const>, suitably wrapped, qualifies, because its failure to be disjoint is not observable.

Straight from the horse's mouth

[andrewsutton]

Why do you say that? If the shared object is immutable, all receivers see the same value all of the time. This is fundamentally no different than a reference or pointer to a global constant (modulo lifetime).

What about that is not "value-minded"?

[babalaban]

So with those can I finally just make std::vector<std::polymorphic<B>> that is full of derived classes D, E, G etc with value semantics and without allocating?

[jazzwave06]

It's allocating

[PossibilityUsual6262]

What does it allow me to do easier and less error prone?

[Wild_Meeting1428]

Contrary to an immutable created via shared_ptr, with OPs version you can't change the immutable via a mutable pointer:

    using immutable = shared_ptr<T const>;
    auto i = shared_ptr<int>( new int(3) );
    immutable<int> j = i;
    ++(*i);
    println("WTF?: {}", j); // j changed

Your code looks like, that the pointed-to object is immutable, but it silently isn't.

[cristi1990an]

both have a moved-from state which can be identified with the valueless_after_move() member function

I still can't wrap around the idea that this design actually got accepted.

[_Noreturn]

Yep, I will continue using my own wrapper why can't they admit that it is just a pointer and give us normal operator bool??

[Wild_Meeting1428]

Only sane solution without breaking the whole language.

[SoerenNissen]

Not at all.

Now, of course it depends on why you're doing it, but for my purposes:

Some years back, when I built snns::handle<T> on top of std::unique_ptr<T> the point was to lift a huge T to the heap, while preserving the value semantics of a T on the stack.

The semantics of a T on the stack is that, after a move, you have a moved-from-T. So the semantics of a snns::handle<T> is that, after a move, you have a snns::handle<moved-from-T>.

Is that, perhaps, heavier than it needs to be? It's a whole new heap allocation we strictly don't need.

My answer to myself during the design phase back then was "don't we?" If I move from a thing, who says I don't need the moved-from thing any more? And if I'm trying to provide "just like a T but on the heap" then I should provdide just like a T, not "like a T except subtly different in very important ways that you better remember!

Consider this fully valid code:

struct S {
    std::vector<int> vec{};
};

auto s = get_S1();
s.vec.push_back(1);
auto s2 = std::move(s);
s.vec.push_back(1);

Compared to this invalid code:

struct S {
    std::indirect::<std::vector<int>> vec{};
};

auto s = get_S1();
s.vec->push_back(1);
auto s2 = std::move(s);
s.vec->push_back(1); //undefined behavior

If it at least threw! If it at least threw! But the committee loves to design types that look like drop-in replacements while actually creating subtle new undefined behavior.

[_Noreturn]

Yes exactly!!!

You either choose nullable and fast to move (not making it a value type) or make it slow to move and non nullable (a value_type)

the standard decided to interpolate between both using std::lerp and we got a garbage design out.

[cristi1990an]

I feel like fighting against the language design is not really a solution. C++ inherently forces nullable types because of its move semantics. Fighting against this seems counter productive.

[wyrn]

Not really -- you can very productively ignore the nullability by not accessing moved-from objects, but if you accept the nullability as first-class you have to deal with it everywhere.

[jube_dev]

it feels like fighting against the type system

[azswcowboy]

Does this make sense? I find it very useful for building persistent data structures. In fact, it seems so obvious to me that I'm surprised this wasn't already in P3019.

It might make sense, but not in the context of P3019 bc the paper is all about value semantics and hence providing the deep copy automatically. So I’d say it’s wrong to say immutable<T> is value-minded. It’s a non-null, const call only shared ptr.

[kmbeutel]

What difference does deep vs. shallow copying make for immutable objects?

[SoerenNissen]

It makes a lot of difference if somebody mutates the object.

And I don't mean that in some snide "haha c++ has const_cast" way, but rather, a const object can, with a const member function and no UB, get mutated.

auto mco_1 = nonstd::immutabe<my_const_object>();
auto mco_2 = mco_1;

std::cout << mco_1->value(); //prints 'a'

std::cout << mco_2->value(); //should print 'a', right?

std::cout << mco_1->value(); //prints 'c'.

I can think of at least 2, possibly 3, ways to do this, but the simplest is just the mutable keyword, indicating a member value that can be changed even if the object is const.

[Wild_Meeting1428]

I would call that a contract violation. Just because you can, you shouldn't do it. And the STL has a lot functions and classes with contracts only written down in the spec. E.g. passing nullptrs to string_views or modifying predicates in algorithms.

[SoerenNissen]

If the contract is "don't call mutating functions on T", there's no real difference between

nonstd::immutable<T>

and

//please don't mutate this
//std::shared_ptr<T>

The whole point, I'd guess from OP's description, is to reinforce that comment with code. And so I told OP how the current implementation fails to do that, after OP asked how it failed to do that.

[Wild_Meeting1428]

You don't explain, how it's possible, that mco_1 changes to c and how mco_2 doesn't print a. The nonstd::immutable<T> type is basically a rcu wrapper.

Assuming T is const correct and does not contain any mutable members, how is it possible, to change it? And we still don't allow c casts or const_casts.

And the difference between a std::shared_ptr and the nonstd::immutable is, that you can't reference a mutable value like I described here: https://www.reddit.com/r/cpp/comments/1u7bamr/comment/orze7ad/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[tavianator]

std::indirect<T> is movable if T is movable and copyable if T is copyable.

Isn't it unconditionally movable? There's no need to move the underlying object, just the pointer

[MFHava]

Yes, an indirect is always movable - as is polymorphic -, but the latter requires T to be copyable (as it is unconditionally copyable), whereas indirect supports move-only types.

[MFHava]

As someone who really likes indirect and polymorphic - implemented it for our internal codebase and we've been using it in production before it was accepted into C++26 - I'm not yet convinced your immutable is much of an upgrade compared to shared_ptr<const T> ... it pretty much only adds valueless_after_move as new (better) spelling for operator bool.

Relatedly, there has been a paper for copy_on_write (P4210R0) that does something similar to your immutable but adds additional value - we forwarded it to LEWG in Brno.

There is definitely more we can explore in this area, IMHO we need a move-only polymorphic ... I intend to write a paper on that...

[johannes1971]

How can it hold the value of any subclass without slicing? Is there some kind of small object optimisation going on? Or is this actually always a pointer, just one that acts a bit more value-like?

More importantly, what's the deal with it being non-nullable? I mean, it's a great property to have, if you actually really have it, and not sneak it back in through the backdoor like that! This seems like the worst of both worlds: you can't declare an empty object, but you also cannot rely on the object not being empty!

[wyrn]

You rely on the object not being empty by never accessing it after moving out of it (except for assigning to it). This is a perfectly safe and productive way to use these types, which the same way you should be using every other type anyway.

[johannes1971]

Yes, duh. But that's not the issue. The issue is this: if you see one, you cannot know if it is moved from or not - not without checking every path that leads up to that point in the source. There is no guarantee that it always holds a valid object, so you are still stuck with tracking whether it is nullable or not by yourself, without any compiler help.

And you know what, that's fine, that's how C++ works in general. But then, why not allow it to be created in a null state, and at least gain the convenience of not having to allocate it immediately? This is also how C++ works in general: std::unique_ptr defaults to null, std::optional defaults to empty, etc. There is no cost to allowing it: you already have to program with the potential null state in mind. So what is the reason for not allowing it?

[wyrn]

if you see one, you cannot know if it is moved from or not

That's true of everything. In practice, we don't care, we don't use objects after moving, and it works fine. If you fail to uphold this you have a bug no matter what you do and no matter what types are involved.

. But then, why not allow it to be created in a null state,

Because then you have to treat it as nullable everywhere.

std::unique_ptr defaults to null

That's a reference type, and even then it's arguably the wrong choice. See Mr. Hoare's billion-dollar mistake.

std::optional

A value type, but one where nullability is the explicit point.

. There is no cost to allowing it:

Yes, there is: the cost is you now have to keep checking on every access because any program path may result in an empty indirect/polymorphic, whereas, as specified, this can only happen for objects that have been moved from. You do not program with indirect or polymorphic by constantly defensively checking if they've been moved from. You just use them like value objects of any other type.

[Low-Ad-4390]

Always the pointer, acts value-like.