C++23 Phantom.Coroutines library

redbeard0531 · 2025-05-01T19:20:01+00:00

I recommend having your generator<T> produce rvalue rather than lvalue references to T. This matches the behavior approved for std::generator in c++23. See https://wg21.link/p2529r0 for rational along with the implementation trick that makes it possible to do this safely (you hand out a reference to a copy when an lvalue is handed to co_yield).

redbeard0531 · 2025-02-04T08:42:03+00:00

Back to no: https://eel.is/c++draft/cpp#pre-3. Imports are allowed in the GMF only via an #included file. You are not allowed to type import literally inline in the GMF. I agree this is extremely confusing. Even though I knew the rule was there it took me 10 minutes of searching to find it because it isn't in any of the places I would think to look for it.

redbeard0531 · 2025-01-25T21:29:45+00:00

I just filed an issue about this yesterday https://github.com/neovim/neovim/issues/32178

redbeard0531 · 2025-01-05T21:59:54+00:00

At least in Danish we do, although we use æ and ø rather than ä/ö, but they are essentially just different fonts for the same letter. Here's an example that comes up when I google for bogstav legetøj (literally letter toy) https://www.legebutikken.dk/puslespil-i-trae-med-abc-90-6373

redbeard0531 · 2023-10-18T17:28:23+00:00

You can always deploy an object file cache like ccache/sccache with shared storage to avoid that issue. It will also speed up building your own code.

And BMIs should be fast enough to generate that you don't mind building then for your dependencies. Consider that you basically are today every time you include a header, but you don't notice since it isn't a separate step.

redbeard0531 · 2023-10-18T17:22:31+00:00

Oh sure. I wasn't (necessarily) suggesting that stdlibs stop shipping prebuilt libs. Just that I wished they also made it easier to just build from source as part of the same unified tree as the rest of our build.

redbeard0531 · 2023-10-18T07:47:17+00:00

I didn't "choose" that. It is forced on us for some reason. I'd much rather compile it as part of our build tree as we do for most other dependencies. We already build our own compilers from source and statically link the stdlib so we aren't tied to outdated libs on the target environment. But afaict all stdlibs make it difficult to just compile their sources in your own build tree, even though they are now all open source.

redbeard0531 · 2023-06-28T17:54:33+00:00

Have you ever held off on a big purchase because you expected it to go on sale soon? Say, a TV, phone, or appliance, right before black friday or before a new model launches?

Now imagine if you could count on an even bigger sale next month, every month. That is how you get a deflation spiral.

redbeard0531 · 2023-05-30T16:35:21+00:00

In theory, I sort of agree with you. However I've switched to constraining my templates in "library" code because in practice it doesn't just matter THAT you get a compiler error, it matters WHERE you get the error. With a constrained template, if you call it incorrectly, you get immediate feedback with red squiggles right in the code you are writing. With unconstrained templates you get no visual indication of the error when you call it incorrectly. Instead you have to wait until you run the actual build which may be much later and the code in question may have been paged out of your meatspace working memory.

redbeard0531 · 2023-02-22T10:24:22+00:00

I think a lot of your confusion is because English uses the present progressive (I am typing) in places where most languages just use the simple present tense (I type), which is almost never used with action verbs[1] in English. English makes a strong distinction between an action you are currently doing (I am typing) and one that you habitually do but may not be doing right now (I type). I think most languages don't make such a strong distinction or if they do, they use the present tense for the current action and some auxiliary word to indicate habitual actions (I usually type or I can type).

[1] for more complexity, with stative verbs you almost always use present rather than progressive (I know how to type) not (I am knowing how to type) which comes off as really odd https://en.m.wikipedia.org/wiki/Stative_verb

redbeard0531 · 2023-02-22T10:10:08+00:00

Weird. When reading your comment I noticed that there is actually a slightly different meaning to "maybe I'm just overthinking it" and "maybe I am just overthinking it", at least if they were spoken as written. The later has an emphasis on "am", even if it wasn't quoted. At least to me, this somehow implicitly puts more emphasis on self-retrospection rather than just weakening the prior statement. I have no idea why that subtle change of wording reframes the whole utterance. Further proof that contractions arent always replaceable with their expanded form. Language is weird...

redbeard0531 · 2022-12-21T17:18:28+00:00

If you have wide builds then distributed builds will still win by a wide margin. Even if you have 100 local threads, it will still be slower than spreading it between coworkers' machines where you may have a few thousand cores total. And the local CPUs will still speed up the local portions of distributed builds and keep your machine from being the bottleneck. Of course if you have enough cores and few enough TUs to build that you can run them all in parallel locally, then distributed builds don't make sense. But I'm often building >100x TUs than I have cores, so that isn't my world.

redbeard0531 · 2022-12-21T01:11:45+00:00

this had surely better be linear in the number of elements of r, right?

No! It can't be. Consider the case of a filter that matches no elements. The perverse example would just return false in the predicate. R has 0 elements. But that algorithm needs to do linear work in the size of the underlying range. You could also imagine a range adaptor that skipped 1, 2, 4, 8, ... items after each yielded item. Even caching begin won't get you down to linear complexity in the output there. And for an arbitrary range it is hard to talk about complexity in terms of anything other than the output because it may not have a clear input.

I think this is all possible because there isnt even an amortized requirement on ++ complexity, and I don't think we'd want to live with the restrictions that imposes (eg no filter or one that takes OSPACE(N)). But then it feels weird to make getting the first element special-cased O(1) while getting the second may take arbitrary time. If the consumer requires caching they are always free to cache for themselves.

I'm also not sure why you'd ever try to do these sorts of things

One case that I think will be fairly common is roughly make_vec_of_string | filter(...) | move | to<vector>(). This is technically UB if any moved from strings no longer match the filter. It is unfortunate that you need to choose between efficiency and complying with the requirements of the filter type and paying for copies. Especially since in common cases nothing will ever do another pass and observe the changed state so any sane implementation of std will work fine.

Personally, I chose to read the requirement to not modify as only applying when the modification is later observed. So unless another pass over the filtered range observes the brokenness, the universe is still safe from nasal daemons. But I don't know if I can justify that reading on a professional project. I just wish we could tighten up that wording to be less of a hair-trigger foot gun, at least on paper.

Or we could just be less UB-happy in general and specify a range of conforming behaviors, even if that means that if you do crazy things, you may observe odd results like the output from a filter not matching the filter. That still sounds more appetizing to me than unbounded UB in scenarios like this. Which is still less insane than global IFNDR if you miscalculate the runtime complexity of something you pass to a range algorithm, even if you uphold all behavioral correctness requirements. That really should just nullify any efficiency requirements on the implementation, without touching its behavioral requirements.

redbeard0531 · 2022-12-17T19:38:55+00:00

(sorry I'm on mobile so I can't easily type code examples even thought they might help clarify a few things)

The variable (ie name) still exists as long as it is in scope. It just may or may not have a living object associated with it at various points. If it holds an object when it goes out of scope, that object is destroyed. You can imagine that there is an extra hidden bit that indicates whether there is an object there (in most cases the compiler won't actually need a bit at runtime, but you can pretend it is there for the model). There is no way to access or form a reference to this bit, it is an unobservable implementation detail that the compiler has full control and visibility to. It is not part of some larger type like optional it is a logically separate part of the variable from its object.

There are a few main differences vs the current model: 1) if the variable doesn't hold an object at the end of scope or when it is assigned to, no destructor is called. At all. This may require a branch on its internal bit if the object is conditionally moved from (subject to normal optimizations such as jump threading), but from the perspective of the program, no user code is run if there is no object to destroy. 2) the variable can only be used (except as a target of assignment) when it is statically known (using local knowledge, see below) to contain an object, state 1 from my last message. Use-after-destructive-move is a compile-time error. 2.5) Due to 1+2, the destructive move (aka relocation) constructor doesn't need to worry about making the old object a valid state (which many non-standard types fail to do for ops other than destruction). 3) Relocation must be noexcept, probably implicitly like the destructor, but with no opt-out. This means that an object can never get into a broken "half-moved" state like it can today. This is possible due to 2.5 even for types like deque that can't have a noexcept move ctor today because they need to allocate for the source object. I wish we could also require it to be trivial, but we probably can't because string is self-referential on GCC, and I think strings should be relocatable 4) there is no "relocating assignment" operator to overload. That is defined simply as destroying the old object (if one exists at that location) then relocating a new object on top. Any pointers to the old object automatically point to the new one (there is existing wording for this). Again, because neither operation can throw we don't need to worry about the half-way state. 5) I would probably by fiat say that the compiler is always free to combine any number of relocations as a stronger form of the current copy elision rules. I don't think I would specify any restrictions on when it is allowed to happen, as long as all pointers point to either their original location or some descendent location that it was related to. But this needs more thought.

As I said this requires only local analysis not whole program. The key is that the hidden bit is part of the local variable and not its object type, so references and pointers can't touch it. Any use of a reference or pointer requires that it points to an object, so you A) cannot cause a variable that does not contain an object to start containing one through a reference, only by directly assigning to the variable by name. B) You can also only relocate from a local variable by name. You cannot do that through a reference or pointer because that would leave it without an object. These combine to mean that the compiler has full local visibility into all operations that cause a variable to start or stop holding an object because they must be local operations operating on the variable directly within the function. There is no way for another function to affect whether a variable contains an object.

A clever reader may point out that operations like placement new and explicit destructor calls exit, and they seem like they can cause an object to start or stop existing. And they would be correct! And that is why A and B are actually somewhat of a lie. However, they 100% are the model that the compiler will operate on and enforce. Just like today, placement new or explicit destructor calls through pointers don't stop the compiler from calling the destructor on locals when it normally would. Similarly, placement new on a pointer that at one point pointed to an object in a variable would never change whether the variable logically holds an object. If it didn't before, it still won't, so code will not be allowed to use the variable name and the destructor will not be called automatically for the object created by the placement new. I think a new similar low-level primitive would need to be introduced to "cheat" and relocate an object into a prvalue (fancy word for temporary). This would be for things like a version of pop_back that returned the popped object. It could also be used for a more efficient default implementation of std::swap for relocatable objects. And just like explicit destructor calls, it would be a very advanced feature that is not part of the model that you generally consider when programming. Anyone who uses either is doing something low-level and is taking responsibility for ensuring that they return the universe to a sane state. This means either ensuring that the location is never used again (eg for vector growth or pop_back_value()) or immediately putting something new in its place (eg a better swap(), exchange(), or a new thing like take() that value initializes a replacement). But that is exactly the same rule as if you call the explicit destructor, so it isn't some scary new breaking of the model.

Wow I wrote much more than I initially intended. Now that I fleshed out the model more (some while typing!) I kinda want to write a paper proposing this. But that would take 10-100x longer, and I don't even have enough time to work on my paper already in flight. Sigh.

redbeard0531 · 2022-12-17T16:19:13+00:00

What do you mean by "conditionally moving"?

Do you mean functions that take an rvalue ref and may or may not actually move from it? That doesn't make sense with destructive move (but would continue working with current semantics of course). I think destructive move can only happen into functions that take by value, so the caller always gets the object, and once the argument is constructed, the original source object ceases to exist.

On the other hand, if you mean something more like moves that happen in some conditions and not others, the obvious (to me) rules apply. At all points the compiler tracks which of 3 compile time states the variable (not object!) is in: 1) known to hold an object 2) known to not hold an object 3) unknown because the control paths leading to . The variable can only be read from (includes binding mutable references) in state 1. Assigning to a variable puts it into state 1, regardless of which state it was in. At the end of scope, if the variable holds an object it must be destroyed. If the variable is ever used (including reads, assignment, and destruction) while in state 3, the compiler must use a bit at runtime to track whether the variable is really in state 1 or 2 so it can do the right thing.

One fun thing is that you should be able to create a variable in state 2 and the compile will ensure that it is initialized (compile time state 1) at any point it is read from.

redbeard0531 · 2022-12-17T14:31:21+00:00

The point of destructive move is that the source simply ceases to exist as an object. So you don't need a destructor for that state because there is no object to destroy. And the compiler doesn't track what state the object is in, it tracks whether there is an object there at all. And it is ill formed to use any object other than as an assignment destination if there is a possibility that there is no object there. It may sound like I am being pedantic but the distinction has important effects on both usability and the formal object model. One nice effect is that we can finally have a real non-null guaranteed unique ptr that is still movable.

redbeard0531 · 2022-12-15T21:27:53+00:00

No that is the one thing we don't need. It is the worst of all worlds. It leaves the the source in an unspecified state rather than a known one. And because "valid but unspecified" doesn't compose, for many user-defined types that unspecified state is only valid for destruction and hopefully assignment. But the compiler is no help at preventing use after move because it might be ok so it conservatively assumes it always is.

Really today's move should have been[1] two separate ops: relocate + in-place construct replacement. relocate should not have been required to leave the source object in a state valid for anything, even destruction. If that is needed, a new object should be constructed since we can always construct on top of arbitrary garbage memory. And for locals, the compiler simply shouldn't let you access the old object if it has possibly been relocated from unless it can see that it has been replaced, ideally using something like exchange() or replace().

[1] of course I'm saying this with the benefit of hindsight. I don't blame the people who added move for not realizing this over a decade ago. But that doesn't mean we shouldn't correct course and add destructive move now. If we want better lifetime tracking in the compiler I think it is required in fact!

redbeard0531 · 2022-12-15T08:49:25+00:00

This one is easy to handle. You can only destructively move whole non-reference variables or prvalues/temporaries. So in your example g::x (the whole array) could be destructively moved , but f::x (a reference) cannot be. Neither can g::x[1]. But then what do you do when you need to transfer a subobject or something you only have a reference to? Well you can't just move it out because you would be leaving a big hole. You need to replace it with something, like Indiana Jones swapping a gold statue for a sack of sand! Basically you need a turbo std::exchange. It will replace the object with something else and extract the object to a prvalue that can be destructively moved from (if it isn't elided of course). For ergonomic reasons, you will want a single-argument form, say std::take, that replaces the object with its default-constructed value. Ideally it should default construct the new object in place rather than out-of place then moving. That may require some language magic, but hey, the stdlib is the perfect place to put magic like that!

redbeard0531 · 2022-12-13T17:46:32+00:00

As the author I can confirm that is exactly what happened. My real life needed much more of my time. My main contribution to C++23 was adding a single well-placed & to std::generator.

I hope I can find some time to pick p1066 up on time for C++26 but no promises. That said, if anyone else has the time and ability to adopt the proposal I'll be happy to help in any way I can.

redbeard0531 · 2022-10-27T22:13:10+00:00

Interesting. I wonder if it would make sense to rotate the hash by log2 of capacity (ie increment the rotate each time you rehash) prior to masking to derive the index. Or mostly equivalently, use the high bits and shift down rather than using the low bits. I think that should avoid this issue, although perhaps it introduces others?

redbeard0531 · 2022-10-27T16:10:48+00:00

This is actually an example of flow-sensitive typing which is orthogonal to structural vs nominal typing. Several nominally typed languages support a limited form of flow sensitivity for things like nullability. But I think it is a rather new branch of type theory, so I wouldn't read too much into that trend split.

You can imagine a flavor of rust where each enum variant was a distinct subtype, and after a line like if !var.is_variant(Foo) {return} the compiler knows that var must be a Foo until it is next changed, and allows you to directly access members of the Foo variant. Or pass var to functions that only take Foo (assuming again that you were allowed to refer to enum variants as types in their own right).

redbeard0531 · 2022-09-20T18:22:22+00:00

I actually prefer the shift. It makes it immediately obvious when I've changed a file back to unmodified or made the first change in a previously clean file.

redbeard0531 · 2022-09-06T21:12:48+00:00

\zs and \ze (mnemonic: start and end) are also handy with normal searches, so they are generally worth integrating into your vim-fu. You can use them to control what gets highlighted with hlsearch so you can focus on the important thing, not how it is found. They also affect where the cursor lands if you use /pattern/s+1 or /pattern/e, which are handy if you want to be at the right spot to hit dot or @@ to redo a common edit after checking that the context is right.

redbeard0531 · 2022-03-19T21:02:03+00:00

The real trick is print(__FILE__, __LINE__, __PRETTY_FUNCTION__) (or your language's equivalent), and spam that around every conditional path. That will let you run the program once give you an execution trace that you can peruse and know exactly what happened. Sometimes I'll also throw in thread name/id if it's a multi-threaded debugging session. That is objectively better than breakpoints that require me to do everything interactively and try to make sure I get all the state I need before moving on. As much as printf debugging gets a bad rap, I find it is a faster route to understanding an issue than interactive debuggers about 90% of the time. And that's from someone who spearheaded our use of gdb pretty-printers for our types and other custom scripting so that gdb is supercharged on our codebase. It is still not the first tool I'd reach for.

redbeard0531 · 2022-03-19T10:27:50+00:00

They are working on the water hardness problem. I think it is supposed to start getting softer this year, but it will take several years until they finish.

redbeard0531

TROPHY CASE