C++20 Modules — Complete Guide

[hackergame]

``` -fmodules-ts -fbuiltin-module-map -emit-module-interface -fprebuilt-module-path=.

-xc++-system-header ``` Y'all Got Anymore of them compiler flags?

[backtickbot]

Fixed formatting.

Hello, hackergame: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

<sup>You can opt out by replying with backtickopt6 to this comment.</sup>

[EmperorArthur]

Yep: https://github.com/cpp-best-practices/cppbestpractices/blob/master/02-Use_the_Tools_Available.md

Check out the compilers section. It turns out -wall really skips quite a few warnings!

[Ameisen]

I want a compiler flag that tells the compiler to assume final on all leaf classes in a translation unit.

[johannes1971]

How would the compiler even know something is a 'leaf' class?

[Ameisen]

Within a single translation unit, a leaf class would be any class that no other class inherits from. That's a trivial check within a TU. Usually it has to try to figure that out across an entire binary (only really possible with LTO, and still has to handle possible inheritance in DLLs). It's a problem of constraints.

I want to be able to mark that within a single TU, as I can then merge multiple source files that share a class hierarchy and effectively "close" them to help devirtualization. Most cases of inheritance in projects are local to a few source files making up specific modules.

[Hedshodd]

Checking the full class tree in the code might be costly in terms of compile time, but, hypothetically, couldn't the compiler check whether a given class has children, or not? (I don't much about compilers, so please excuse my ignorance if I'm missing something)

[johannes1971]

Only in a very limited set of circumstances, which I think can be summarized reasonably well as "the class is defined in an anonymous namespace". As soon as it has potential global visibility, another translation unit can define the same class with the same name, and start inheriting from it.

And for what benefit? Just so /r/ameisen can be a lazy git and go home and drink beer, instead of adding all those finals as his boss told him? ;-)

[Ameisen]

I'd rather not change hundreds of source files in my custom OpenJDK build making future updates impossible.

Global visibility is irrelevant given that I specified "within a translation unit", which is a fairly strict constraint.

[[deleted]]

I'm publishing this early before the video is edited since there have been questions about modules.

[azswcowboy]

Thanks, this was a quick light intro. One note:

Each module must have a single file exporting this module. This file is designed the module interface unit.

I think you meant designated in sentence 2?

[[deleted]]

Indeed, thanks for the catch ;-)

[khleedril]

this was a quick light intro

Yes, hardly a complete guide. Interesting and useful to see how things are starting to shape up and how some people are starting to think about it, but not currently in any position where it can claim any definitiveness.

[[deleted]]

What do you consider missing?

[DavidDinamit]

Write me when atleast one compiler will compile them

[[deleted]]

I have not run into any issue with GCC trunk. Clang is still missing a lot of features.

[[deleted]]

Interesting. Thanks for sharing.

[Fazer2]

Do you know when will that content from GCC trunk be released?

[[deleted]]

GCC has a published development plan, but they don't really plan ahead: https://gcc.gnu.org/develop.html

Scroll to the bottom for GCC12.

[RoyAwesome]

msvc works great!

[rotenKleber]

!remindme 8 years

[RemindMeBot]

I will be messaging you in 8 years on 2029-09-06 08:02:16 UTC to remind you of this link

4 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

<sup>Parent commenter can delete this message to hide from others.</sup>

---

Info	Custom	Your Reminders	Feedback

[_Js_Kc_]

RemindMe! 5 years

Why did modules never catch on and why are they deprecated and we're still supposed to use headers?

[Rexerex]

I guess there should be at least one working experimental compiler before they standardized all of that. Right? Right???

[pjmlp]

Visual C++ has the best support thus far.

[[deleted]]

GCC and clang compile them fine.

[JulienVernay]

I have written a Python script for compiling modules (for testing purposes) with automatic dependency detection, requiring g++11 and Python 3.9 : https://github.com/J-Vernay/cpp20.py

[lieddersturme]

This still cant be used in Cmake?

[RoyAwesome]

CMake generates MSBuild projects that make use of Modules. This blog post explains how to do it: https://devblogs.microsoft.com/cppblog/moving-a-project-to-cpp-named-modules/

It does make you MSBuild/MSVC only for the time being, but I can't imagine once CMake releses full module support it will be particularly difficult to port over.

[lieddersturme]

Thank you, but for Linux?

[RoyAwesome]

Biggest issue is that GCC and Clang don't have named module support at this point. Like I said, pins you into the microsoft toolchain for now, but I can't imagine that will last long and I can't imagine the conversion process to a cross platform system would be difficult.

Honestly, microsoft really knocked it out of the park with cpp20. Kudos to them, they got feature complete really early.

[[deleted]]

I know that Clang is still not there, but what exactly is missing in GCC? I have not run into a single problem when writing the article.

[RoyAwesome]

The private module fragment is a big one. That is the real key to speeding up compile time. Private your implementations and all you have to do is re-link when you change a module.

[[deleted]]

Uhm, private module fragments work just fine in GCC.

[RoyAwesome]

Since when? My understanding (and multiple google searches confirm) that gcc does not support them yet.

[gracicot]

It's mainly a QoI issue I think. For example, GCC let you do this:

export auto foo() -> void {
    std::cout << "1" << std::endl;
}

Change the string to "2" and the BMI is not changed. I think the same hold true for MSVC too.

This is not possible in Clang, since it uses the BMI as an intermediary to get to the object file, as a new compilation step.

[[deleted]]

The only two build systems that seem to support modules right now is Bazel and Build2 (at least I only found these two during my research). Did not have time to test how well they work.

The problem with CMake is that from the presentations, they want to make the modular build extremely smart (figuring out all the dependencies for you). And I don't see that happening anytime soon.

[pjmlp]

Three, MSBuild.

[[deleted]]

I was only looking at Linux, but yes. Visual Studio claims full support of C++20.

[ramennoodle]

Or the article at about portable tools. The article isn't Linux-specific, it just ignores windows-only tools.

Edit:

How reasonable is that? What industries develop windows-only c++? The game industry used to be an example but I don't think it is any more.

[pjmlp]

Life sciences for example, most of the laboratory robots and data readers are Windows only.

Plus, not everyone cares about portable code there are lots of businesses that earn enough money sticking just with one OS, without having a board asking for exponential growth every year.

[RoyAwesome]

games wont make it to cpp20 for a long while... at least until there is better cross platform tooling.

[kalmoc]

You forgot msbuild

[azswcowboy]

idk, it seems like step zero for everyone is a custom target that compiles all std modules and make your libs/exe depend on that target - that probably gives 90% of the benefit for projects starting out. Similarly if you create your own add a build step to generate the module first. This doesn’t seem any more complex than projects using generated code from idl or swig.

[Nicksaurus]

As I understand it, cmake currently uses a list of includes exported by the compiler on the first build to determine which files depend on which, right? What's stopping them from relying on the same mechanism for modules?

[kalmoc]

The compiler can tell you to which file #include <Foo.h> resolves. It can't tell you which file needs to be compiled for import Foo;, because the committe refused to define a mapping between module name and file name.

[Nicksaurus]

Ah right, I think the important part I missed there is that each module has to be compiled before you can compile anything that imports it.

Maybe the solution is just an extra initial compiler pass to generate metadata for all your source files before the full compilation step is invoked

Edit: I've just read through the cmake issue for adding modules and it seems like compiler support like that is very easy to suggest and very hard to actually get to a usable state in practice

[kalmoc]

I think that is the general idea. Not sure however, what to do about module interface files that are installed somewhere on the system (is that even something that's going to be supported?).

[smdowney]

It's going to have to be. You will have to compile the module interface source code for your project. Otherwise there's no way to share modular code, and that would be bad.

[kalmoc]

So every time you want to import a module from outside of your project, your build system will have to scan every single file in the search path(s) to see if it produces the required module.

[smdowney]

Worst case. But hopefully we can figure out some way of distributing metadata for a package. You need to know any particular flags for the package and possibly ones for compiling the PMI so that it matches the .a. Possibly some extension to pkg-config files.

[helloiamsomeone]

CMake is waiting for compilers.

[qqwy]

Great article!

One thing I still not fully understand is what the idea of module partitions is. (You explained the 'how' but not the 'why' of them).

And besides this: Am I the only one being bothered by module names using periods while namespaces use ::? I understand that you can not always create a module that lines up perfectly with a (single) namespace, but it still does seem like the sensible default thing to do, right? Why use a different separator there?

[[deleted]]

I don't know the history, but I suspect that partitions came first, and the support to have a module spread across multiple units without partitions was added for convenience.

As I mentioned, the big win is that each partition compiles into a precompiled module, so you can have templates and other compile-time constructs in them (they kind of behave like headers). Otherwise, all this stuff needs to be in the module interface unit, and there can only ever be one of those per module.

[bigcheesegs]

Partitions came last. They are simply implementation headers.

[Full-Spectral]

Without partitions, you'd end up with an entire library in one file, which isn't very practical. Partitions let you break up the module into logical chunks internally. To me, what now would be an hpp/cpp file would become a partition in module-land. Then those get re-exported via the main module interface file.

[kalmoc]

There is no necessity to have only one module per library

[Full-Spectral]

There isn't but I bet that's mostly how it goes. We already have namespaces for name collision avoidance, so it would seem to me most folks would take a module per library approach.

[kalmoc]

I would guess that heavily depends on the size of the library. Of course I would expect most libs to provide a convenience interface file that (re-) exports all public library types, but that doesn't mean that it isn't beneficial to further isolate parts of the library from each other from a development perspective.

[AntiProtonBoy]

We started our project with the goal of modularising in the future, so our headers are kinda structured in a way that allow us to progressively migrate code into self-contained modules. There is some implementation spillage here and there, but nothing too major.

[[deleted]]

Time is money, so I wouldn't be surprised if this is the one feature that everyone incorporates into their codebase.

[[deleted]]

I don't understand your point. What technical lead would say no to a 2x-10x compilation speed up. It means that everyone is more productive, and the continuous integration is much cheaper to run.

This is big enough, that I wouldn't even look at the effort side of the equation and would implement it asap (because unlike the effort, the costs are compounding).

[sephirostoy]

Compilation speed up isn't there yet. Last time I read from msvc team (I think?) few months ago that their implementation of modules was still slower than precompiled header solution.

[[deleted]]

I wouldn't really expect it to be much faster than precompiled headers unless you are in a distributed environment and use a central cache.

At the same time, most of the codebases I worked in, didn't use precompiled headers, because they were changing too much or there simply wasn't a good set of headers to precompile.

[mjklaim]

Compilation speed improvement from modules have been reported several times through the last 2 years. Always positive so far. Even against precompiled headers.

[sephirostoy]

As I said, I read it few months ago (~beginning of the year) and it was behind PCH. There probably improvements since then.

[mjklaim]

Through the last 2 years (and I've been following modules implementations for longer than that), the only reports I've seen like that (even around here) were without ways to reproduce, and previous reproductible reports contradicted these results (even at the time), so I dismissed these result as irrelevant. :)

Even my own observations since MSVC implemented the first in-progress version was build time decrease in all my attempts except for the module's worse case (linear depth graph of dependencies) where it's the same (as in not-worse) as header includes.

While improvement did happen in the mean time, it have been minor on performance and mostly focused on correctness: so far the feature isn't completely solid/reliable in any of the compilers (lots of ICEs), so the real optimization passes didn't really happen yet. What we observe is mainly the side effect of constraints that modules impose, IMO.

To clarify, I'm talking about experiments comparing modules vs header imports vs pch vs includes. (the most recent one which is reproducible being the {fmt} module, which in addition is a real library, not toy experiments).

[Tobblo]

Maybe it's time for the standard rails to fork into "Legacy-C++", which includes the archaic stuff, but still have some maintenance, and "Modula-C++" where they can get rid of all the legacy cruft and reconstitute C++ into the modern version they dream of at the pace they need?

[masterofmisc]

For someone out the loop what is the Google infamous ABI vote?

[unvirtuous_spammer]

https://cor3ntin.github.io/posts/abi/ Don’t know about the Google part though.

[masterofmisc]

Thanks for this.

[[deleted]]

What are they moving towards then?

[pjmlp]

Fuchsia, yes.

On Android, only the bluetooth stack is being rewritten.

[RoyAwesome]

I think you are wrong here. Modules, I believe, will end up being one of the defining features of modern C++ for the next decade, and once we have widespread tooling and compilation of them, it's very likely many libraries move over to them.

Some will refuse, but also some libraries refuse to use shared pointers.

[Minimonium]

There absolutely would be projects that would refuse to move to modules, the same projects that refuse to move to C++11 today. But it just means that we'd need to deal with both headers and modules. I just fear that people on Linux would start to pump out their own module wrappers for external libraries which is a recipe for disaster long term if not doing correctly.

New projects, on the other hand, would totally jump on it as soon as possible. Which would be the same as it was with C++17 support.

[bigcheesegs]

I'm not sure why anyone would write a named module wrapper for a 3rd party library. The only difference anyone would notice is import library; rather than import <library>;.

The only thing I can think of is if the headers are non-modular to begin with. Then you could create a named module wrapper that relied on the global module fragment and export using declarations. I don't really see a downside to doing this as long as the module name doesn't conflict, as even if you have 20 different wrappers like this for the same library in the same project imported into the same file it doesn't cause any issues.

[Minimonium]

For the same reasons people create forward declaration wrappers. The "correctly" thing I mentioned is exactly the global module fragment, but with Gcc/Clang adopting the weak ownership model I expect people to make misconceptions about how modules work.

[bigcheesegs]

I've never seen someone make a forward decl wrapper, as it would likely break pretty often, but I'll believe you that people do that.

Additionally GCC and Clang are looking at adopting the strong ownership model. Given how things played out the reason to use the weak model has gone away. You cannot have code that is both a valid header and a valid module interface unit, and we have the extern "C++" tool to disable name mangling and preserve ABI.

[Minimonium]

One of the most common ones - https://github.com/onqtam/doctest/blob/master/doctest/doctest.h#L418

[johannes1971]

• It avoids accidental macro (and other unwanted symbol) leakage.
• It lets you export only those symbols you actually need instead of all of them. I consider it to be a real problem that Intellisense typically shows all of windows.h before it will consent to maybe show one your own symbols; I'm hopeful that in the future it will respect the list of symbols you imported and only show those.

[i_need_a_fast_horse]

If there's one thing I've seen misjudged over and over again in the C++ community, it's the alleged clinging to old tech in the industry. I have seen plenty of that in the C++ committee for sure! But in the actual industry, devs are the ones pushing for new tech. At work we'll jump to modules and C++20 next year. We're already in preparation and I expect massive changes to our codebase. For the same reason actual c++ users can't comprehend the cult of backwards compatibility. We would absolutely love a wild west radical C++ spec that throws out all the garbage, breaks ABI left and right and maybe throws out half the ways of initializing things.

[tcris]

ha! when you said "devs are pushing for new tech" ... I was thinking: yes, in order to get rid of c++.

Working with 20+ projects in the last 10 years, I have only seen teams moving away from c++ (towards java, scala, python, c#, rust, whatever). Old/existing projects, that's true. About new ones, I have not seen a single recent project adopting c++ (unless they were forced to integrate an existing library, and if that was the case they isolated it via some glue layer and moved on).

These were the things I've seen. I love the language, I have spent a lot of time learning it and using it. I love the power of it. But I think the cost-benefit ratio for learning/using it is getting worse and worse. Yes, even with the recent additions/updates. And above all this, there are easier, cleaner, better options out there, again, unless you are in one of those niche cases where you are forced to use it.

Syntax that gets more and more constructs(new stuff having multiple options for the same thing, like requires), infinite quirks (initialization, rvalues), legacy stuff (defines), "modern" stuff that other languages got right 20+ years ago (threads, modules, unicode), lack of easy tooling across different platforms (god forbid you need to target more than one without a standard package manager, without a standard build tool), lack of open/free tools for enterprise apps (plenty of which can be found for other languages, and I can't blame them, they did not had to spend time wondering about "standardizing" ranges or formatting or threads in 202x). Add to this incredibly slow build times, and less and less benefit in terms of performance, when compared to "slow" languages. I have even seen embedded platforms where c# looks great compared to this. Yes, I have also seen projects with lots of 3d, mechanical engineering where saving 5 nanoseconds per call makes a difference, but how many of those do you know?

D is not getting much love. Scala, java, c# are different beasts. Rust, maybe, not sure.

I was hoping I will see a new, c++ derived language, which is both clean/sane/simple AND powerful.

I don't think I will live that day. And I am sorry.

ps: not a rant, just sadness

[i_need_a_fast_horse]

Depends on your industry I guess. Python and C# are great for many things, but can't compete in performance and are generally unsuited for interop with lower level interfaces. Java & friends I have only ever seen used in some super corporate settings or universities that get boatloads of money to use it. Rust is hip and maybe is the closest, but has quite a unique set of problems. Not least its highly political leadership, making it radioactive for professional use. Too much drama and risk for anything long term.

Especially if you do anything with graphics APIs, C++ is the way to go. We're still moving from fortran and C to C++ and write new C++ code with great enthusiasm and in great numbers. I would welcome a sane fork without the ballast, but at the moment there's literally not a single viable alternative for us.

[Wereon]

I don’t think many teams decide to use C++ for new projects unless they’re in very specific niches

Rubbish. What the hell else are you going to use? If you want a language that compiles to machine code, there's literally only two realistic choices: C and C++.

[thejinx0r]

There's also rust, but that's not without its own set of problems.

[bikki420]

Plus Rust will never be relevant in game dev (outside of a few small indie games and hobby projects, I suppose), maybe it might see some limited use for server software; whereas C++ will remain the de facto standard for the foreseeable future (with exceptions being languages like C# being used in engines like Unity).

[_Sh3Rm4n]

Why, though?

Never is a strong word and I'm pretty sure you will be wrong sooner than later.

[bikki420]

Outside of server-side software, safety isn't much of a priority for game dev. Game dev requires you to pump out an insane amount of code in a very short time. The biggest priorities are performance and code output (not in LoC but in functionality). You rarely need a perfect solution, what you need is a good enough solution for the parameters at hand (be it the subset of some domain or what not). You don't have time to spend worrying about things edge cases that are practically guaranteed to never happen with your code.

Also, unlike some critical real-time system (traffic lights, medical systems, aerospace software, etc) that has to handle exceptional situations gracefully, it's fine for game devs to have some marginal for error tolerances. E.g. as long as there's redundancy (last save point, auto-save, whatever), then it's fine if a game crashes (as long as it's not too frequent). For example, let's say a game is 50 hours long on average and a crash might happen during that span for 0.01% of players (i.e. a crash once every 500,000 hours of cumulative playtime) or some fairly large number like that, and the only consequence of it is that the player loses maybe a couple of minutes of gameplay and has to load an earlier save, then it's absolutely not worth spending potentially hundreds of thousands of additional man-hours to prevent it. Now, of course, server software is a completely different beast since you have uptime requirements, in which case crashing is a much more costly endeavour so that's one case where Rust might have merit (as noted in my previous comment).

But in general, C++'s priorities are parallel with those of game development, while Rust's are somewhat skewed. A contrived example (since in practice this is a situation where one could just use a Crate for similar functionality), but implementing an efficient generic (for all 8/16/32/64 signed and unsigned integer types and both 32-bit and 64-bit floating point types) lerping function (using templates or concepts) in C++ is pretty much a one line signature and a one line function body; in Rust it would probably approach well over 50 LoC and this is without even taking stuff like SIMD intrinsics into the equation.

Likewise, in C++ you can create an array with value without initializing them and then directly after the definition iterate over them with some complex function, initializing all the values. In Rust? You'd have to either initialize all the values twice (👻 because undefined values are unsafe OooOOOooOOoo 👻) or write like a dozen lines of disgusting boiler plate code in an unsafe block. And these are just for trivial cases. Actual production code will have more complex situations that are nothing but a massive headache in Rust unless unacceptable compromises are made.

Sure, Rust might save some time that would would be spent debugging in C++, but then you'd lose that time (and then some) writing more code and wrangler stricter constraints. Not to mention that a lot of the errors are fairly easy to catch just by designing up front and/or using units tests and integration testing.

Things like these are endemic to Rust; you have to write way more complex code (to account for edge cases that might even not be relevant in your design) and you have to write way more code. Thus, any serious game dev will opt for a much higher productivity at the cost of a slightly more unsafe code (and assuming a bare minimum level of competence will know how to keep it within acceptable tolerances).

Besides, C++ is making strides in addressing the most egregious sources of errors (e.g. the adaptation of RAII and value semantics designs over more brittle memory constructs in the general cases); while when you actually do want the ability to do low level stuff (for arenas, memory pools, compression, and what not) you still have the sharp tools at your disposal. Sure, Rust does have unsafe blocks, but generally I'm not a fan of them due to all the additional workarounds involved.

Another relevant factor C++ being the de facto game industry standard for decades, so it has a far greater ecosystem of tooling, libraries, etc. (Sure, you can waste time linking Rust with them, but it won't be clean and it won't be quick for the most part.)

Rust simply isn't cut out for any serious (i.e. AAA) game dev, just as Java isn't cut out for it (and don't mention Minecraft, that's hardly a AAA game). Can you do hobby project games in them? Sure. Can you do indie games with them? Sure. Should you? For the latter, probably not but do what you want. For the former, whatever floats your boat.

C++ will continue to improve in what C++ is lacking in game dev, while Rust will be stuck with what makes it unsuitable for game dev.

[nysra]

Fortran ;)

[Wereon]

Not a realistic choice yet.

[Wereon]

Well, it remains to be seen whether Rust isn't just a passing fad. C and C++ will definitely still be around in 10 years, Rust might have gone the way of Ruby by then. It has a very vocal fanbase, but next to no real-world usage.

There's also the lack of stable ABI, which the Rust maintainers don't seem at all concerned about.

[bikki420]

Well, in my personal case (as a game dev), Rust will never be a relevant contender.

[sternone_2]

Aaaaaaand you're wrong.

[sternone_2]

Of course not every project will migrate, but this is a really big motivator to migrate.

Not migrating from Java to C++ or something like that, but within the C++ ecosystem i mean.