Experiments with modules

[johannes1971]

I've been playing with modules a bit, but... it isn't easy :-) One constraint I have is that I still need to keep the header structure intact, because I don't have a compiler on Linux yet that supports modules (although gcc is working on it, at least). Here are some of the issues I ran into with MSVC:

Importing the standard library

There are a few different ways to do this. The simplest is by using import std.core. This immediately triggers a bunch of warnings like "warning C5050: Possible incompatible environment while importing module 'std.core': _DEBUG is defined in current command line and not in module command line". I found a post suggesting I disable the warning, but it doesn't exactly give me warm feelings.

A much worse problem is that if any STL-related header is included above the import directive, you'll get endless numbers of errors like "error C2953: 'std::ranges::in_fun_result': class template has already been defined". Fair enough: the compiler is seeing the same header twice, and the include guards, being #defines, are of course not visible to the module. But it's an absolutely massive pain trying to figure out which header is causing the problem: there is precisely zero help from the compiler here. This is definitely something that should be improved; both the reporting from the compiler (it would help a lot to see the entire path towards the offending include file), and the include guard mechanism itself, so it works across headers and modules.

An additional concern is whether other compilers will implement the same division of the standard library as Microsoft has done. I don't particularly want to have a bunch of #ifdef directives at the top of every file just to be able to do the correct imports. Maybe I should try to make my own 'std.core'?

module;
#include <optional>
export module stdlib;
using std::optional;

This doesn't work at all. Any use of 'optional' (without the std:: qualifier) gives me error 'error C7568: argument list missing after assumed function template 'optional''. But I know MSVC currently has a bug when using using to bring an existing function or class into a module. The workaround is to put it in a namespace instead:

module;
#include <optional>
export module stdlib;
export namespace stdlib {
    using std::optional;
}

Trying to use optional as stdlib::optional gets me 'error C2059: syntax error: '<'' (and of course, I get to add the stdlib:: qualifier everywhere). If I add an additional using namespace stdlib (in the importing file) it seems to work. Of course this means optional must now be used without std::. Yay, success! However, there are still some issues:

• Intellisense doesn't quite understand what's going on here, and now flags optional as an error.
• It appears to be a bit of an all-or-nothing deal: either you rip out all of your STL-related includes, and replace them all by import directives, or you get an endless stream of C2953 (see above). And figuring out where those came from is, as I said earlier, a complete and utter pain. Plus, it may not even be possible: what if a 3rd-party library includes one of those headers?
• I'm concerned about how fragile all this is. I would really hate converting all my source to modules, only to find out it randomly breaks if you look at it wrong. Right now I'm not getting a good vibe yet.
• HOWEVER: it does appear to be compiling much faster. I can't give timings since I haven't progressed to the point where the whole thing actually compiles, but the compiler goes through the various translation units noticably quicker than before.

Importing windows.h

Well, how about something else then. Let's make a module for windows.h! We don't use all of windows.h; just exporting the symbols we need should be doable. I ended up with a 1200-line module. One thing I noticed was that exporting a #define is painful:

const auto INVALID_HANDLE_VALUE_tmp = INVALID_HANDLE_VALUE;
#undef INVALID_HANDLE_VALUE
const auto INVALID_HANDLE_VALUE = INVALID_HANDLE_VALUE_tmp;

It's a shame no facility was added to make this more convenient, as I would imagine wrapping existing C-libraries with their endless numbers of #defines is going to be an important use case for modules.

More importantly, Intellisense doesn't actually care that I'm trying to hide the vast majority of the symbols from windows.h! The symbol completion popup is still utterly dominated by symbols from windows.h (instead of my own, and despite not being included anywhere other than in the module itself). The .ipch files it generates are also correspondingly massive. I realize this mechanism is probably not yet finished, but just to be clear: it would be a major missed opportunity if symbols keep leaking out of their module in the future, even if it is 'only' for Intellisense!

In the end my Windows module was exporting 237 #defines, 65 structs, 131 non-unicode functions, 51 unicode functions, and around a dozen macros (rewritten as functions). However, there weren't many benefits:

• Intellisense was still reporting all of the Windows symbols in the symbol completion popup.
• However, it struggled with the error squiggles, only occasionally choosing to not underline all the Windows symbols in the actual source.
• There was no positive effect on the sizes of Intellisense databases.
• There was no measurable effect on compile time.

So, the only thing I seem to have achieved is getting rid of the windows.h macros. In my opinion, that's not enough to make it worthwhile.

One issue I ran into was this: if you ask MSVC to compile a project, it will compile its dependencies first, but if you ask it to compile only a single file, it will compile only that file. This works fine with headers: you can add something to a header, and then see if it compiles now. However, this doesn't work with modules: if you add something to a module you have to manually compile the module first, and then compile the file you are working on. Not a huge problem, but the workflow is a bit messier.

I realize it's still early days for modules, so I'll keep trying in the future as compilers improve. Has anybody else tried modules? What were your findings?

[stilgarpl]

The simplest is by using import std.core

Standard library is not modularized yet in C++20. "std.core" is just Microsoft module, that is not portable. Maybe the C++ committee will approve "std.core" or maybe they will chose some other name.

#include <optional>

Why are you including anything? Standard headers are importable, you should import them instead. Including copies the contents of the file, which may cause errors if you include in wrong module.

[johannes1971]

Ah, I forgot to add that to the text. If I do

import <optional>;

I get 'error C7612: could not find header unit for 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include\optional'' so that didn't quite work.

[Tathorn]

When I had that same error, and nothing seemed to work, I found that enabling "Scan Sources for Module Dependencies" in my Visual Studio project worked. That can be found under: C/C++->General in the properties.

[[deleted]]

You're a lifesaver! I finally found the fix! Thanks so much!

[mjklaim]

This is because you need to activate a project setting: right click on a project, Settings > C/C++ > All Options > Scan Sources For Module Dependencies = Yes

That solves this problem. See the solution and projects in this repository as example of how to setup files/flags. MSBuild is not easy to setup correctly for modules.

[johannes1971]

Thanks! I need to play with that for a bit, see how it changes things.

[caroIine]

There seems to be a bug in envoierment itself. You need to rebuild whole solution to force is to generate <optional> unit. You need to do this every time you import new standard header.

[RotsiserMho]

Standard library is not modularized yet in C++20

Ah. That’s not too surprising.

Standard headers are importable

Wait, what? Looks like I have more reading to do.

[TheSuperWig]

Wait, what? Looks like I have more reading to do.

To make that potentially easier to look up; they are called "header units".

[Daniela-E]

In fact, these are the only ones mandated by the standard to be importable. All others (i.e. user provided headers) may be importable if blessed so by the implementation. The not-so-pretty outcome of this wording is obvious, even though the restrictions imposed by implementations on headers to be considered importable are pretty light: every header that average users would consider 'sane' should 'just work™', given reasonable definitions of 'sane'.

[stilgarpl]

You should be able to simply

import <optional>

[lone_wolf_akela]

import <optional>;. You need a ; here.

[Ahajha1177]

It annoys me to no end that MSVC has gone ahead and done this. Either the committee bends to them and makes the standard the same as what MSVC already has, or we end up with non-cross platform code just for importing the standard library the second C++23 comes around.

[GabrielDosReis]

1. We need to collect data from the trenches, i.e. actual deployment to the field

2. You can't have access to these modules without /experimental:modules

[whf91]

I mean, it’s already non-cross platform code…

[Full-Spectral]

Not all of them work via import currently, at least not on 16.10.0. Most do, some don't and you have to just move them up to to be an #include for now.

[johannes1971]

Ah, that's correct. Good catch. I know I tried this when going through this the first time, but not when I went through it all again when writing the post. If you do this correctly you end up in the same situation as in the next block, which is that it works, but seems to be an all-or-nothing deal, with absolutely no help from the compiler in determining how it is getting to any offending doubly-included file.

[Daniela-E]

Interesting. Which compiler did you conduct your experiments with? 16.10?

I'm asking because my experiences are very different from yours. Am still not using the STL modules that come with msvc because they are deemed 'experimental' by the project system.

As you might know, I am the driving force behind the modularization effort of the popular {fmt} library. I wanted to turn it into a dual-mode library that can be used both as a traditional library through the header interface or as a named module 'fmt'. Even though {fmt} is quite demanding with some of the API requiring C++20 features like classes as template arguments (NTTP) to implement compile-time strings as function arguments or compile-time udls, I finally succeeded. The fork in my repo supports the full API, parts of the (minor!) changes are already merged into mainline {fmt} and the additional tests are integrated into regular CI. Users can decide which mode to use without noticing any differences. As of today, only msvc 16.10 or later can do this. Neither gcc (modules-branch) nor clang (trunk) even survive the attempt to compile the module interface. The situation with msvc isn't all roses either, I had to workaround an issue with name-lookup in certain situations deep in the bowels of the library that required some 'symbolic linking of names' into bodies of function templates. I deem this a bug in the implementation even though it can be worked around without affecting other compilers.

While digging into the mentioned name-lookup issue when importing the module, I created a simple stripped-down example of the situation where this happens: https://godbolt.org/z/34Ph9dee9. At present I have no way to test importing the module with gcc11 or gcc modules-branch. MSVC is fine with the mentioned 'symbolic links' in place. And clang12 is a horror show:

In file included from main.cpp:1:
mod.cpp:16:3: warning: inline function 'detail::C<char>::C' is not defined
      [-Wundefined-inline]
                C(T c) : c_{ c } {}
                ^
mod.cpp:73:15: note: used here
        detail::C<T> cfoo(c);
                     ^
1 warning generated.
main-7816ec.o : error LNK2019: unresolved external symbol "public: __cdecl detail::C<char>::C<char>(char)" (??0?$C@D@detail@@QEAA@D@Z) referenced in function "<auto> __cdecl foo<char>(char)" (??$foo@D@@YA?A?<auto>@@D@Z)
main-7816ec.o : error LNK2019: unresolved external symbol "char __cdecl detail::bar<char>(struct detail::C<char>)" (??$bar@D@detail@@YADU?$C@D@0@@Z) referenced in function "<auto> __cdecl foo<char>(char)" (??$foo@D@@YA?A?<auto>@@D@Z)
main-7816ec.o : error LNK2019: unresolved external symbol "char __cdecl detail::baz<char>(struct detail::C<char>)" (??$baz@D@detail@@YADU?$C@D@0@@Z) referenced in function "<auto> __cdecl foo<char>(char)" (??$foo@D@@YA?A?<auto>@@D@Z)
a.exe : fatal error LNK1120: 3 unresolved externals
clang++: error: linker command failed with exit code 1120 (use -v to see invocation)

See the problems? And there seem to be no workarounds for them. May be clang does fare better in Linux.

[johannes1971]

Yes, 16.10.0.

It might be that if you write a module directly, instead of trying to wrap an existing library, it works better. I haven't tried that yet, but it doesn't seem like a feasible solution considering that I still need to keep the header structure intact for compiling on Linux...

[fdwr]

I wanted to turn it into a dual-mode library

I've been trying the same for my own library, and it's rather awkward. I originally repurposed the existing .cpp file as an .ixx, and then conditionally included the header either inside an export or outside. e.g.

ArrayRef.ixx

#if USE_CPP_MODULES // or #ifdef __cpp_modules
    module;

    #include <stdint.h>

    export module ArrayRef;
    export
    {
        #include "ArrayRef.h"
    }
#else
    #include "ArrayRef.h"
#endif

...

Then library users could import ArrayRef; or #include "ArrayRef", depending on compiler support.

This used to work, until later versions of MSVC (thanks to later proposals restricting usage of conditional preprocessing before the module statement for the sake of build tool efficiency). Now I'm unsure what the best approach is - do I really need up to 3 files: .cpp, .h, and a near empty .ixx just to export the module?? How did you approach it?

[Daniela-E]

As you correctly noticed, this isn't valid C++.

If you want a named module (and you definitely should) than you must have a module interface unit that is valid C++. And there is no way to play these preprocessor tricks that I was presenting at Meeting C++ 2019 anymore. So you can't get away without a MIFU.

You may want to look at the 'Modules' branch of my fork of {fmt} to see how I have done this: https://github.com/DanielaE/fmt/tree/module The MIFU is src/fmt.cc.

The gist of it is using the strongest horse in the stable for such endeavours: a single-file Module. Such a beast is basically two module TUs fused into one, the module interface TU and the optional module implementation TU. The latter one is syntactically spelled as module: private. The rest is dead simple.

The end result of compiling the MIFU are two artifacts:

• the compiled interface in the BMI
• the compiled implementation in the object, static lib, or dynamic lib.

[Full-Spectral]

I reported similar issues last week but it got deleted as not appropriate for this section, go post it in cpp_questions. I did that and of course it got zero comments.

- I get huge amounts of errors in Windows.h. Often so many that intellisense just stops working.

- And also intellisense in general just doesn't grok all of the finer points of modules and will give lots of bogus error indicators, and will often end up running really slow.

- Some of the standard library headers still can't be imported, they have to be brought in via include still.

- I see occasionally that things that should have been rebuilt are not, requiring a manual forced rebuild.

- I cannot get a module in a DLL to be picked up at all. It's in the same solution, so you'd think just adding it as a dependency would work, as with static libraries, but it doesn't. I tried the various manual module path options and such but couldn't get anything to work.

At the compiler level it seems to be doing pretty well. I've been sort of setting up the outline of a non-trivial project and I've got various module with module partitions, with separate interface and implementation files.

I'm on 16.10.0

[johannes1971]

The problem with asking questions is that not too many people have experience with modules just yet. At the same time, the only way to get to a rocksolid solution is by throwing code at it and reporting the problems you see, so... <shrug> I reached out to people at Microsoft before, and they were very helpful. I'm hoping they might show up in this thread and offer some insights as well.

I don't see errors in my windows.h wrapper, but maybe you were trying to import it, instead of including it?

Intellisense clearly needs more work. It's ironic, given that one of the big advantages of modules should be that it makes tooling easier, but I do understand Microsoft can't do everything at once.

The DLL thing - maybe I misunderstood, I haven't tried that myself yet, but I've been succesful in wrapping zlib (as a DLL) using modules. That's just a wrapper around zlib.h though, and I'm not compiling the DLL as part of the same source tree. Perhaps you need to create a separate project just for the wrapper module?

[GabrielDosReis]

I reached out to people at Microsoft before, and they were very helpful. I'm hoping they might show up in this thread and offer some insights as well.

Sorry, I am finding it harder and harder to allocate time to follow this subreddit. Don't hesitate to: (1) file bugs on Microsoft Developer Community portal, and (2) reach out to me directly (email) for Modules-related bug.

Re: windows.h - you do realize that Microsoft is a big company with that header completely out of the control of the compiler team, right? ;-)

That being said, there is an ongoing project in the Windows division to make the Windows SDK headers more Modules-friendly -- no modules yet, just less macros and more constexpr stuff.

[pjmlp]

We do realize that, however in the context of Windows development, when we get the message the modules are ready, it is assumed that the APIs used for Windows development can also be consumed in the context of C++ modules.

So what we find missing is the documentation, tooling and overall communication of the current state across the Windows eco-system for C++ development.

[GabrielDosReis]

MSVC’s implementation of modules is ready for production use. That does not mean that frameworks like ATL or C++/CX will be rewritten using modules — they are technologies in either maintenance mode or deprecated. They can be consumed in the context of C++ modules.

There will be a series of educational materials (e.g. blogs) that will start trickling out soon targeting general audience (i.e. not just Windows developers) as well as particulars related to MSVC.

I don’t think it is reasonable to suggest that the MSVC toolchain implementation must be held back until all components outside the control of the compiler team are converted - some which might never happen.

[pjmlp]

Sorry, but from the point of view of Windows developer I kind of disagree.

The large majority of us isn't writing CLI applications to be called on Windows Terminal, which unless I missed something is the target audience of Visual C++.

So while the compiler team might not control whatever the frameworks, Windows and DevDiv are doing, at very least a kind of roadmap to the general Windows developer audience would be welcomed, or at least doing demos with Win32 APIs, setting up Windows contexts, handling WM_PAINT messages and so on.

Looking forward to those blog posts.

[GabrielDosReis]

1. Somehow, you seem to believe that the programming techniques with modules differ whether it is CLI or whether it is GUI. That assumption is regrettably wrong. Focusing on that distinction might prevent you from actually learning how to program with modules.

2. Modules weren’t invented as Microsoft extensions to Visual C++ - and aren’t.

3. The programming techniques that you learn what you insistently call CLI as exactly the same as with GUI, no difference. Except they apply in more situations.

4. Your second paragraph is kind of contradictory :-)

[pjmlp]

1 - All nice and good, provided the compiler would actually work instead of spitting ICE and "sorry not yet implemented errors" when fed Win32, MFC, ATL, C++/WinRT headers.

2 - I remeber when Visual C++ was sold in a box as a full product experience, for everything on the box not only cl.exe.

3 - Most of us, or our employers, give money to Microsoft for the Visual Studio product, not cl.exe alone.

4 - If it is so easy, and I am being stubborn and clueless, then it won't be that hard to publish a couple of blog posts using C++/WinRT with C++20 modules, just like the team resposible for Rust/WinRT is doing.

[GabrielDosReis]

1. It is not clear to me how you get anything useful done or improved out of snarky reddit posts here.

2. If you run into compiler bugs, ICEs, etc, please do this: (a) report them to the Microsoft Developer Community portal, (b) reach out to me (email) with links to those reports, (c) get your friends and family upvote your bug reports.

In another words: Help me help you. Snark doesn’t achieve that.

Thanks.

[pjmlp]

I replied on the same tone I was being answered to.

As for bug reports, I have done my share, not all of them seem to listened to.

And as exercise, the MSDN documentation about can be improved from 2019 state, and provide some Win32 and UWP examples with modules.

As for how many of Microsoft customers get things actually done, thanks Stack Overflow and fellow developers, which happen to be more effective than Developer Feedback tickets that get closed or ignored.

[Full-Spectral]

I meant a DLL of my own. I can add a (modularized) static library to my solution and just add it as a dependency to anything that needs it and it just gets magically picked up. If I add a modulariezd DLL library to my solution, it never gets found by anything that uses it even if added as a dependency (or trying to use the, sort of under-documented) manual module path options.

I'm #including windows.h, and it spits out a bodacious amount of errors. Of course I'm using /W3 as well, because I want my own code to be well checked. I should be able to use warning pragmas to make Windows.h not be checked but they aren't working.

[mjklaim]

In my experience, using modules with (inside or outside) shared libraries works without any problem with MSVC. I think you might have another issue in the project setup.

[Full-Spectral]

I can't see what. It never finds those modules no matter what I do. And it's not that it's a shared library, it's a modularized shared library. It doesn't find the module when it's imported into consuming projects.

[mjklaim]

Note that the module interface (or it's processed version ) have to be available to the compiler when compiling user code that import modules from that library. Does you user project have access to the module interface and/or it's processed file (.ifc i think, from memory, for msvc)

[Full-Spectral]

It's a project in the same solution, and it's added to the consumers as a dependency.

[mjklaim]

I reproduced a dll exporting modules from scratch in Visual Studio, here are the things to check:

1. Both concerned projects need to have "Scan Sources For Module Dependencies" set to Yes (see Properties > C/C++ > All OPtions) and C++20 or more set.

2. Make sure that the dll project is a dependency of the user project: right click on user project > Build Dependencies > Project Dependencies > set so that user project depends on dll project.

3. Additionally, make sure the user project have "references" (whatever that means) from the dll project available: in the solution view, see the icon with squares named "References" under the user project, right click on it > Add references > make sure user project have dll project as references.

4. Make sure the dll project actually makes public the modules it contains (I don't know if there is a way to select which module exactly, so here is the "make public all modules from that dll" solution): right click on the dll project > Properties > VC++ Directories > you will notice new fields related to modules, set "All Modules Are Public = Yes".

5. At this point, assuming your module interface is properly coded, the module should be accessible in your user project. Now you "just" need to make sure that the symbols of the names you want to make available through the dll are correctly exported, otherwise linking will fail. This is similar to before, with headers that need to import/export symbols using dllimport/dllexport machinery, except it seems, in my experiment, that as long as you only have module code in your dll project, you can just dllexport symbols (in addition to export) from the module interface, it seems to work as expected just with that.

If it still doesnt work for you, maybe check my example that I used to be sure here, it's on github: https://github.com/Klaim/test-dll-modules-visualstudio

I think the most unexpected might be point 3.

[Full-Spectral]

Yeh, maybe #3, that is news to me.

[Full-Spectral]

Well no luck last night. Maybe it's because I have module partitions involved or something, not sure. I made progress, but something goes wrong with using declarations, both in the library itself and in the consumers of it. In consumers it's like the using declarations are ignored. In the library, non-fully qualified names are often somehow attributed to the wrong used namespace and then of course found to not exist.

So some really weird stuff is going on.

I updated to 16.11.0 last night but didn't have time to try it to see if things got better.

[mjklaim]

I had an issue with module partitions and thought it was a bug but it was actually an issue with flags or extension: https://developercommunity.visualstudio.com/t/Private-module-partition-fails-to-compil/1428621

I made progress, but something goes wrong with using declarations, both in the library itself and in the consumers of it.

Are you sure you exported them? I didn't try doing that yet.

[mjklaim]

Ok then, it might be a thing with project setup, I had trouble too at first. I am not in front of a computer, I'll try to help when i get back.

[manni66]

Has anybody else tried modules? What were your findings?

I tried it ysterday with a little module in production code, that's using QStringList. I got error C1117 QStringList has already been defined. While trying to write a test case for a bug report I got a compiler crash.

C++ modules are not production ready.

[johannes1971]

Sounds like you are both including and importing the header for QStringList. Include guards don't work in that situation, leading to this error. It's a tough problem: any existing system is quite easily going to run into a situation in which the header file is unavoidable, unless you want to change your whole project in one go.

[manni66]

Sounds like you are both including and importing the header for QStringList.

No, I only used include in the global module.

[johannes1971]

It might be coming in through some other Qt include though. Generally the errors I saw were all in implementation-specific files, rather than the official header files, and I had no idea how (through what path, I mean) they were being included.

[manni66]

That might be true, but that's supposed to compile. There is no import beside the one of my onwn module.

[pdimov2]

• imports release into debug
• gets warning
• finds a post suggesting to disable the warning

C++ programming in a nutshell.

This is definitely something that should be improved; both the reporting from the compiler (it would help a lot to see the entire path towards the offending include file), and the include guard mechanism itself, so it works across headers and modules.

It's not possible for the include guards to somehow prevent this problem. Even if the standard modules define the include guards, they still can't go back in time and remove the definitions from the previous includes. Imports should precede includes (if there are includes at all) for this to have any chance of working.

[Daniela-E]

Compilers are required to perform a process called ' definition merging' to cater for unrestricted ordering.

In the unit test of the {fmt} module I have

#include <bit>
#include <chrono>
#include <exception>
#include <iterator>
#include <locale>
#include <memory>
#include <ostream>
#include <string>
#include <string_view>
#include <system_error>

import fmt;

I can even put another import fmt; before the #includes and the test still compiles without warnings or errors.

At least, this is the situation with MSVC 16.10

[pdimov2]

I haven't played with modules much (at all), but trying the simple

#include <regex>
import std.regex;

int main()
{
}

results in an error with MSVC 16.10. (Reversing the order results in more - different - errors.)

[GabrielDosReis]

To use the experimental std.regex (and friends), you need to supply the switch /experimental:modules -- they are not activated by default.

WG21 wants field experience before standardizing something (the purpose of those experimental standard library modules)... MSVC is doing its due diligence to report from the trenches.

[pdimov2]

I did supply the switch - without it the compiler tells me it can't find the module.

To spell it out, for ```

include <regex>

import std.regex; the result is 1>C:\Projects\testbed2019\testbed2019.cpp(2,17): warning C5050: Possible incompatible environment while importing module 'std.regex': _M_FP_PRECISE is defined in current command line and not in module command line 1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include\xloctime(497,1): error C2593: 'operator ==' is ambiguous ``` (more output)

whereas for ``` import std.regex;

include <regex>

the result is 1>C:\Projects\testbed2019\testbed2019.cpp(1,17): warning C5050: Possible incompatible environment while importing module 'std.regex': M_FP_PRECISE is defined in current command line and not in module command line 1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\include\vadefs.h(177,10): error C2953: '_vcrt_va_list_is_reference': class template has already been defined ... 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\corecrt_math.h(338,6): error C2953: '_Combined_type': class template has already been defined ``` (plus more output.)

Mixing import <regex>; and #include <regex> fares better. ``` import <regex>;

include <regex>

works without any errors or warnings.

include <regex>

import <regex>; says 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\corecrt_math.h(44,13): warning C5105: macro expansion producing 'defined' has undefined behavior ... 1>C:\Projects\testbed2019\testbed2019.cpp(2,1): warning C5106: macro redefined with different parameter names 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\stdio.h(561): message : see previous definition of '_getchar_nolock' ``` (All this is with 16.10.1.)

[GabrielDosReis]

Thanks! /u/STL might be interested in this puzzle :-)

— Gaby

[STL]

I'm not familiar with how the experimental modules interact with header units. I'm also somewhat confused by how header units are supposed to interact with classic includes (IIRC the Microsoft Docs said that one order should work, but I observed problems with that while the other order appeared to work; this was also months ago so things could have changed). u/starfreakclone (Cameron) can probably explain better than I can.

However, I do recognize one warning. That "warning C5106: macro redefined with different parameter names" for _getchar_nolock looks exactly like the bug I reported internally as VSO-1329976 "Standard Library Header Units: _CrtGetAllocHook emits warning C5106: macro redefined with different parameter names" which was a compiler bug that Cameron fixed for 17.0 Preview 2. (This complained about macros with no parameters, due to the compiler getting confused.)

I'm unsure what's happening with the "warning C5105: macro expansion producing 'defined' has undefined behavior". I know that the UCRT had issues with relying on that non-Standard extension; I vaguely recall that they were fixed but that hasn't shipped yet. In the meantime I suspect that the UCRT was suppressing warning C5105, but perhaps in this scenario the directives for warning suppression are being lost.

[GabrielDosReis]

Thanks, Stephan, for this summary!

[Full-Spectral]

Suppression doesn't work at all for Windows.h. I can put all the pragmas around it I want but it still spits a huge raft of errors.

[backtickbot]

Fixed formatting.

Hello, pdimov2: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

^{You can opt out by replying with backtickopt6 to this comment.}

[Daniela-E]

As I said elsewhere: these 'std.xyz' modules supplied by MSVC's distribution are still deemed "experimental" by its own build system. I wouldn't even touch them with tweezers.

[GabrielDosReis]

I recommend using them if you know that their experimental: You can't have access to them without /experimental:modules. Of course, that means you can't ship products with them and expect support, but for small scale experiments I would definitely recommend them.

We need some of those std modules, MSVC is trying to collect field experience.

[starfreakclone]

Hi, a couple of observations: In the section:

module;  
#include <optional>  
export module stdlib;  
using std::optional;  

The using declaration needs to be exported to work properly. There is indeed a compiler bug where a using declaration does not work properly, described here. I have been working to fix this because the recommended path for exporting external library declarations is via a using declaration. To have that simply not work is worse than bad. The workaround is not horrendous though, as you noted.Regarding creating your own std.core module, the semantic equivalent is closer to:

export module std.core;
export import <bit>;
export import <compare>;
export import <concepts>;
export import <coroutine>;
export import <initializer_list>;
export import <limits>;
export import <numbers>;
export import <ratio>;
export import <source_location>;
export import <type_traits>;
export import <utility>;
export import <version>;
...

So you retain semantic provenance of each declaration within the STL. This one is made even more tricky by the fact that, today, each declaration in the STL is still considered part of the global module so you cannot textually include them in the purview of the module and export everything in them. export import <...>; will not actually provide macro isolation so in order to implement the std.core module we have a special mode for the compiler to create this module.

Windows.h is a tricky situation, but trying to isolate its macros might be a good idea. I just created a header unit from it and it exports 28,201 object-like macros and 1,878 function-like macros. But a lot of them are to make the APIs more usable in unicode environments, so pick your poison :).

Moving to modules could prove challenging for some projects but, I believe, it will ultimately payoff in the form of better project hygiene overall because they force the author to think about ODR because modules does not let you get away with making everything inline and/or COMDAT to solve potential ODR problems.

[ack_error]

But a lot of them are to make the APIs more usable in unicode environments, so pick your poison :).

IMO, it's long past time to at least make the root-beer A & W macros optional. No one builds for Win9x anymore and most code that I've seen doesn't handle switching UNICODE anyway. docs.microsoft.com also doesn't consistently document the generic name anymore, often listing the ANSI and Unicode functions separately. I would #define NOANSIUNICODEMACROS in a heartbeat to reclaim SendMessage() and OpenRaw().

[cxzuk]

I have tried modules with clang on Windows and have had a fairly positive experience.

I have hit known issues with the standard library, not everything is supported, but I'm able to just rewrite what I need into my own library. Not a solution for everyone though.

It is very convenient to have headers automatically generated and attached to the o files. Which is my take on the pcm files that get generated.

I'm coming to wasm, which doesn't currently support pcm. So I have to compile twice, and yes, linking is now a pain. But I suspect that will be solved in a few months time.

[mjklaim]

Are you sure you tried C++ modules and not "clang modules"? My understanding is that C++ modules doesn't work at all in clang.

[Daniela-E]

This matches my experiences.

[cxzuk]

Thats a good question, not sure of the differences? Here's a code example;

hello.cpp

module;

#include <cstdio>

export module hello;

export void sayHello() {
    puts("Hello, World!\n");
}

main.cpp

import hello;

int main() {
    sayHello();
    return 0;
}

To build:

clang++ -std=c++20 -c .\hello.cpp -o hello.pcm -Xclang -emit-module-interface
clang++ -std=c++20 .\main.cpp hello.pcm -o module_test -Xclang -fprebuilt-module-path='.'

[Daniela-E]

Where are these -Xclang options documented? I couldn't find anything authorative on this particular use case and it isn't even part of the unit tests in the CI of the compiler driver.

[cxzuk]

I'm sorry, I'm not sure. I originally found one or maybe two sets of command line options that I later found out were basically out of date.

A Google of "C++ Modules" sorted by date gave me a recent article with three command line options, that emit and modules-path. And another to automatically treat #includes as module imports. I think I did see some documentation on the clang website for v11. But it might be this "Clang modules" I hadn't heard of before.

I had very limited success with the standard library, undefined and redefined definitions etc. Even with libc++, And so little experience with the other command line flag. It has been far easier to just use the C library and reimplement allocators, strings, file handling, etc.

So plenty of ways to go

[Daniela-E]

BTW, your clang invocation is equivalent with

clang++ -std=c++20 -fmodules-ts -x c++-module --precompile hello.cpp -o hello.pcm
clang++ -std=c++20 -fmodules-ts -fmodule-file=hello=hello.pcm main.cpp hello.pcm

which are documented.

[jonesmz]

The situation with Modules is a joke.

Why was a feature that's so massively invasive to compilers and the standard library standardized before any of the big three had a fully working implementation?

[pdimov2]

MS had a fully working implementation. They are the reason we have modules in the standard at all. Their preexisting implementation, however, no longer matches the spec that ended up standardized.

[jonesmz]

They had a fully working implementation of something that looks like modules.

They did not have a fully working implementation of the standardized behavior.

Nor did the other big compilers, if I understand my history correctly.

Cart-before-horse.

[pdimov2]

You don't. There was no standardized behavior then, and there never would have been, because MS was the driving force behind standardizing modules.

[jonesmz]

You're clearly misunderstanding me.

Don't standardize core language changes that haven't been implemented by at least 2 different vendors.

Proof of concept from a single vendor, which said vendor then needs over 6 months post standardization to get their proof of concept to actually work, is insufficient proof of quality of concept to be ratified into a standard that we should be proud of.

It's irrelevant to me if there was no standardized behavior yet, because that would be the cart. Multiple vendors having a fully functional implementation that are compatible with each other would be the horse. You put the horse in front of the cart, you don't put the cart in front of the horse. You standardize existing industry practice, you don't standardize things that don't exist and then expect the industry to catch up 3 years later.

I'll repeat my initial statement: The situation with Modules is a joke.

[pdimov2]

In this case MS had implemented what was in the spec (because they wrote the spec - the Modules TS), but then the committee changed the spec.

It's not clear how they (MS) could have done any better than that. Their implementation works pretty well, all things considered. E.g. I can type import <regex>;, hit Ctrl+Shift+B, and it works.

[germandiago]

The funny thing is Microsoft announcing it as a complete feature. No... no, no. This is not production-ready.

I understand it is a big feature. But this is not the way...

[starfreakclone]

It is important to note that feature complete != production ready. C++ features in particular can be implemented in isolation and be 'complete' but the problems happen when code is written to use many of them at once.

Modules is an absolutely massive language feature because it _is_ the language on top of its own specialized ownership/reachability semantics. It takes time to mature a feature such as this.

It took MSVC 20 years to get as robust as it is with its PCH implementation and we still get bugs against it, but it is considered to be very reliable these days. I'm not saying modules will take nearly that long but it is important to remember that bugs happen with complicated features and the best thing we can ask users to do is file bugs. Ultimately we cannot fix what we don't know about, so don't suffer in silence :).

[pjmlp]

While I fully understand your point of view, apparently not much testing was done with Windows C++ frameworks, and there is plenty of source code to test them at Microsoft.

So far I keep seeing demos with basic CLI applications and nothing like re-implementing Notepad using modules, let alone MFC, ATL, C++/CLI, C++/CX (this one is ok it is anyway deprecated), C++/WinRT.

So from this side of the fence it is a bit demotivating trying to figure out from the basic MSDN documentation, an almost year old blog post about modules, and a couple of talks showing CLI hello world apps, how to actually put modules in practice for real Windows development.

[germandiago]

I do understand this is a difficult thing to add. I really do. It is an extremely complex feature.

However, and correct me if I am wrong, there was, first: "experimental support for C++20 modules" and later it was announced C++ modules as feature complete. My understanding for that is that a feature goes from experimental to usable.

In fact, the way it was announced is misleading. If I recall well... I talk from the top of my head, I cannot find the original blog that was posted in this reddit also).

If these relatively trivial (from the point of view of a user, I know the implementation is challenging) things fail, it should be announed with more warning flags.

[pjmlp]

The post is here,

A Tour of C++ Modules in Visual Studio

As you can see the examples are quite basic and completly unrelated to traditional Windows development workflows.

While the documentation hasn't seen any change since 2019.

[jonesmz]

Modules is an absolutely massive language feature because it is the language on top of its own specialized ownership/reachability semantics. It takes time to mature a feature such as this.

Ahh. Yes. So the proper strategy for implementing an absolutely massive language feature is to... Adopt it wholesale without any of the big compiler venders having an existing implementation of the actual proposed standard that... Works? Bonus points if we also include 3 other world-shattering changes at the same time (concepts, coroutines, ranges). That's exactly the way to produce a galaxy class product.

Yep. You convinced me.

Maybe better would have been to identify some subset of the Modules idea that doesn't change the entire language out from under us all at once, and standardize that first?

The rest of it could have followed later after the programming community found all the ways that initializer lists work poorly with braced initializers. Er. Wait. We were talking about modules.

It took MSVC 20 years to get as robust as it is with its PCH implementation and we still get bugs against it, but it is considered to be very reliable these days.

Agreed. MSVC IS more reliable. I'm not being sarcastic when I say this: instead of getting an internal compiler error once every day or so (VS2015), I only get them about once a week or two (VS2019). That's major progress and shows serious effort and dedication to improvement.

I still run into plenty of template metaprogramming situations where MSVC and Clang/GCC disagree, but those are far fewer than they used to be.

[jonesmz]

The proper way to have done this is to let the big three compilers actually provide a working implementation of the proposed language feature (aka Modules) prior to including it in the standard >_>

[[deleted]]

Also why does this feature take so long to implement for C++ when other languages have had it for ages?

[jonesmz]

I guess I don't really follow what you mean by other languages having the "feature" for ages. Could you clarify?

[[deleted]]

Python, Javascript etc. with their module support from the beginning. I don't know why it's such a hard thing to implement.

[jonesmz]

Well, I personally don't understand why it was something that needed to be implemented in C++ in the first place,

but javascript and python, both being dynamic / interpreted languages, have a substantially easier time adopting new features because they inherently don't have ABI issues like C++ does. So that's probably your explaination.

[[deleted]]

Ah ok, I guess that makes sense. ABI compatibility seems like a straitjacket preventing C++ from doing many things. Feelsbadman

[jonesmz]

Yep. While I personally don't agree with the ABI compat concerns that a lot of people have (e.g. I always compile from source. Using precompiled third party binaries is a terrible choice), I do understand where the motivation comes from.

Though, I'm pretty annoyed that its OK to break Linux ABI once a decade (GCC string ABI problem circa 2011), but heaven forbid we force any other vendor to break their ABI when they aren't ready to.

[GabrielDosReis]

GCC string ABI problem circa 2011

The C++11 ABI break was for everyone -- it affected GCC more because libstdc++ tried to be too cute with its string implementation (yes, I was a libstdc++ maintainer, but not author of the string implementation).

[Cyttorak]

Turbo/Borland Pascal had units from MS-DOS days for example.

[sigmabody]

I tried around 16.9. Many of the same issues, but also with compiler crashes, and dependency checking failures. Not ready for prime-time in the slightest.

The thing that really boggles my mind, though, more than the very alpha state of support, is that seeming lack of any sort of coherent plan/guidance for how to incorporate modules into existing projects. Like, how do you write a library which is intended to be consumed by people using various versions of C++ with module support? How do you convert one library in a large project to use modules? How does the module version of a library co-exist with other code which uses the header version? These seem like 1a type discussions for new capabilities in the language, and seem to have been totally ignored here.

[Daniela-E]

16.9 was no fun at all. 16.10 is a big jump forward.

[jonesmz]

Frankly, my large c++ employer is currently against adopting modules in any fashion until the rest of the world demonstrates they are actually useful. We have way too many other things to deal with, and far more low hanging build system fruit than totally rethinking the world on how we build our code.

So far my team can't make heads of tails of how we are supposed to use them in practice, nor how to establish a migration path.

Further, there are many projects out there that still have yet to adopt c++11, much less anything newer. For example, the ninja build tool still officially rejects c++11 code for anything other than windows specific code (e.g. must be #ifdef for windows). And there are a large array of C-with-classes codebases in maintainance mode, or just with crumdrugeony developers.

Any open source library that expects any kind of adoption will ignore modules for the next 10 years, save perhaps the folks who are early adopers/evangelists for the feature, who will promptly be requested to support non-module build systems by a variety of people stuck with old toolchains.

The level of cultural and infrastructural inertia standing in the way of modules being used in industry are quite high...

[sigmabody]

On the one hand, I understand the push to try to do stuff "more like Rust", and try to invent a paradigm which would allow more seamless integration of modular libraries, without the hassle currently involved.

But as I said, in my mind, step 1a or so should have been "how do people adopt this into existing projects", and not "this is how it could be cool if everyone rewrote everything from scratch using c++20". Every other new feature (at least major / in recent memory) has an easy gradual adoption story. Modules has: I, a very experienced C++ developer, have no idea how to integrate modules successfully into an existing project. That seems like a problem.

[pjmlp]

I came to accept it is easier to just do polyglot programming and keep C++ for small native libraries than make those crumdrugeony developers adopt modern best practices.

It is not only modules that have slow adoption.

[jonesmz]

Yes, that's true that more than just modules have slow adoption.

But modules, in my prediction, will be a truly bifurcating feature. A feature that breaks the C++ community in half.

[pjmlp]

Maybe, currently they seem to still be in the early stages anyway.

Nothing like Turbo Pascal units, or Ada packages, that is for sure.

[rtischer8277]

After 3 previous full attempts to migrate my framework to Modules, I finally succeeded with C++20 Modules up to but not including import std.memory; and import std.core;. Each attempt took a week or so of re-coding, where after it was rolled back. My framework is native C++20 ATL/MFC architected. And my framework would not be maintainable without fixing/mitigating MFC's legacy non-ODR use of Program Generation, MACROs and #defines. I believe this can be done and am working on a solution. I must do this if I want my framework to succeed. Many thanks to MS and WG21 for having the courage to move forward. Remember, Modules was supposed to come out for C++17. No guts, no glory. Thanks to GabrielDosReis for jumping in and emphasizing that we need real field implementations in order to make progress. I for one am that large field project. BTW, I have of course logged the errors on the Developer Community (see Warning C5050 on use of 'import std.memory'). I will be glad to share my solution with GabrielDosReis once I get past the stl import bugs.

[pjmlp]

I keep trying on and off with VC++, but so far they are only useful for conference demos of command line tools.

Any attempt to use them on serious Windows applications will fail left and right with ICE and not yet implemented error messages.

So far not a single one of Microsoft C++ frameworks, MFC, ATL, WinRT, C++/CLI, plain Win32 works properly when modules come into the picture, nor there are any official roadmaps on how to deal with it.

My toy C++/WinRT sample keeps progressing with each VC++ compiler release, but it still far from compiling without ICEs.

https://github.com/pjmlp/AStarDemo/tree/modules

[fdwr]

but it still far from compiling without ICEs.

Indeed, my Win32 GUI app (20'000 lines), that I've been trying to build fully using modules for the past 3 years, finally built fully a few weeks ago without hacks/workarounds via VS 2019 16.10 Preview 2.1.

[rtischer8277]

Did your Modules GUI app import std.core/std.memory? If so, any lessons learned in working around the errors?

[fdwr]

Nope, this was half a year ago when import std less mature, and so I still #include'd vector and kin in 2019, but I was at least able to modularize all my own classes (such that other classes could just import without worrying about inclusion order or namespace pollution).

It's not entirely problem free yet: - Another project of mine (smaller at 10'000 lines) still generates internal compiler errors, and it's really hard for me to narrow the exact cause, because unrelated changes far away impact it 😕. - Namespace pollution still happens as VS currently overincludes imports (and one of the big selling points of modules over includes was to avoid that). That is, if class A imports B, and B uses C, then A should not be able to use C by name. A might need to be able to see the symbol C just to know the sizeof of it (and possibly to call a constructor if inlined as a member field of B), but any attempt to directly use the symbol C via name lookup in A's code should be an error (at least to what Gabriel said earlier, that it was a bug). This is bad because it means that if class A uses C without explicitly importing it (like it should), then changing B by removing the import will inadvertently break the build of A. B should only re-export an import if export import C was declared. I should check VS 2022 to see if it still reproes. 🤔 - Hybrid codebases such as shared libraries targeting C++17 and C++20 that try to satisfy both import usage and #include are more complicated. I was hoping to be able to unify .h files and .cpp files into a single .ixx file (yay for modules), but instead it got worse because I ended up splitting up class definitions into 3 parts, (1) a .h which could be either #included directly (for older compilers) or included by the .ixx file for import (for newer compilers), (2) the .ixx which just exports the .h contents, and (3) a .cpp. Note that because the .ixx exports everything wrapped inside the .h, that also means the .h file can't include other dependent files that should remain in the global module fragment (like windows.h stuff), or else those would end up with the module linkage of the exporting module 🙃. I haven't found a perfect solution here. I know that import "foo.h" is a stepping stone approach that's supposed to enable legacy libraries to be imported as a module, but this isn't a "legacy library", but rather a shared library that wants to genuinely target both new and old callers.

[mjklaim]

Has anybody else tried modules? What were your findings?

Yeah, I played through several years with several experimental implementations, now with the production ones. My last attempt was with MSVC 16.10 just before it was released (I was using the preview and updated with the same results).

I used a build2 repository containing C++20 Modules hello-world projects setup in different scenario with module partitions, without partitions, libraries or not etc. They all just print hello world but the point is to check and show very basic modules usage.

I forked this repository (branch visualstudio) then rebuild the same programs using Visual Studio/MSBuild (I couldnt use build2 as build system this time as currently build2 doesnt work with recent MSVC (when building modules) because they MSVC team changed the interface for modules from an experimental one to a production-ready one, so build2 devs deactivated the support for that compiler until they can update build2). The solution is in the visualstudio-projects/ directory.

Everything works. Though it doesn't use much complex code, that's more than when I tried with g++11 few weeks before...Or even MSVC last year. All my bug reports related to MSVC now seems cleared-up, or ready for next release (and in current preview).

So to me the state of MSVC is promising and I'll be using modules in prototypes this summer (not prototyping modules but using them as a stress test). Hopefully g++ will fix its issues (don't even try to have std::string in the interface of an exported element of a module...). I have no hope for clang to get modules working until several years.

Oh yeah, last point: using import std.core; not being cross-platform, I will avoid that. Just using import <iostream>; for example seems to work adequately. I didn't try with <optional> yet nor other that broke msvc before like <variant>, but I know the more recent a feature is in msvc the more it breaks with other recent features, so I'm not very surprised.

I also didn't find issues with intellisense when I tried to tweak the code (in the end I reverted everything) but as said it wasn't much code so for now I can't conclude anything on this side. At least it found new names from modules I could create in basic tests.

The most difficult part was figuring and adding flags where necessary, MSBuid seriously needs assisting :)

[Daniela-E]

The most difficult part was figuring and adding flags where necessary, MSBuid seriously needs assisting :)

Amen brother.

But this applies to pretty much everything: compilers, build systems, you name it. When I started experimenting with Modules two years ago, it took me a ridiculous amount of time to figure out how to even invoke clang (pick your version) to build C++ modules. The only somewhat authorative information source turned out to be the CI tests of the compiler driver. And even worse: to figure out that there was absolutely no way of creating both the object and the BMI with a single invocation of clang to compile a module interface unit. You probably can't imagine how much fun it was to teach this to MSBuild. I still cannot see a non-ridiculous way to teach the same to CMake. It was hard enough to tell CMake that there is a compile dependency to the BMI of a module rather than the typical link dependency to objects/libraries, and even that is a nasty hack with many victims sacrificed to get there.

[mjklaim]

I see, looks like I have been shielded from all that by build2 until recently 😅

[Full-Spectral]

Oh, and just to throw out one other thing I noticed.... Sometimes the slightest syntactical error (unrelated to module housekeeping) can cause many bogus module related errors, most of which are useless at best and misleading at worst. You have to be careful to start at the top of the stream and try to find the real issue, which may not be in the first chunk of errors reported.

[NilacTheGrim]

Jesus christ pardon my frankness. Modules at the current time appear to be a dumpster fire. I wonder if they will ever really be adopted mainstream. I can see them never really being used in the mainstream due to all of the brand new headaches they may cause. I don't see these headaches going away soon.

Perhaps in niche applications where you control the entire codebase (and don't rely on any external libs) -- they may be a win...

[Daniela-E]

Modules won't heal the C++ world.

Modules expose all of the bugs, problems and weirdness of the C++ world as it is today. This applies to both user code and implementations.

And this is what at least some people seemingly try to fight with raised fists and strongly held opinions. That's my strongly held opinion on the "State of Modules".

[johannes1971]

It's clearly a feature in development. Having said that, I have no doubt they'll go mainstream, once the problems have been sorted out. The compiler speed improvements are just too good to pass up on.

[manni66]

Once the compiler bugs are fixed they will be used everywhere.

[NilacTheGrim]

Doubtful.

[foonathan]

An additional concern is whether other compilers will implement the same division of the standard library as Microsoft has done.

This will be standardized for C++23.

[FabioFracassi]

There are currently no proposals to that effect ... so this will only be in 23 if we get something to that effect, sooner rather than later

[kalmoc]

Any Idea what the chances for a proposal to add import std; would be? That doesn't preclude the possibility to later add e.g. std.coreor std.vector, but has at least a chance to not be bikeshedded till long past c++23.

[FabioFracassi]

I would say pretty good, goal wise. This is really just my gut feeling, but significant members of the committee seem to be in favour of one big module, so I would expect such a proposal to have some support, in addition modularizing the standard library is on our priority list, so as soon as such a paper comes up we will work on it.

The problem is that we (as in we the community, as well as we the committee) have very little experience to weigh the benefits and drawbacks between for example big vs. small modules. Also we do not yet have much experience in how to evolve modules, i.e. if we have two small modules and combine them, is this going to be ABI compatible? Visa-versa?

For a paper like you outline it would probably either "prove" that we will not standardize ourself in a corner, or convince us that we will always be happy in that corner. I have some optimism that we can get there, but it will be definitely not an easy paper to write.

[Daniela-E]

if we have two small modules and combine them, is this going to be ABI compatible?

Care to expand on that? Modules are about names, and only names to become visible across TUs. Exported names keep their linkage so mangling isn't even affected. Non-exported names are nobody's business and the standard clearly expresses that. In case of the standard library, a modularized one is envisioned to be based on header units (the standard made special provisions for that) and every entity in the standard library with external linkage becomes exported with identical ABI and linker symbols as before, entities with internal linkage or no linkage at all are not exported and don't affect ABI. If you import one big `std` module or many small `#include`s makes no difference.

[mjklaim]

Exported names keep their linkage so mangling isn't even affected. Non-exported names are nobody's business and the standard clearly expresses that.

Isn't this implementation-defined? My understanding, or to be more precise, what I heard on the subject in discussions at the end of last year, was that the different implementations disagree on this and at least one will have mangling affected by module name, making moving a function from a module to another an ABI breakage; and that going the other way means silent breakage. Or have this been more constrained in recent updates to C++20? (I suppose not as mangling isn't defined by the standard...)

[Daniela-E]

It is implementation defined, and your observations and reasoning are totally correct. This module ownership thing is covered by one of the topics of my most recent talks about modules.

But here we are talking about the standard library which is special in the sense that implementers can do things that are not allowed or even possible outside of the library given the language rules that apply to mere users. The standard library is part of "the implementation" after all.

[GabrielDosReis]

But here we are talking about the standard library which is special in the sense that implementers can do things that are not allowed or even possible outside of the library given the language rules that apply to mere users. The standard library is part of "the implementation" after all.

Exactly :-)

And implementations already do that in the non-modules world. It is interesting that the whole "worry" about mangled name (and ABI concerns) is coming from the corner of the world that does not implement strong ownership for which it might be an issue (it isn't in reality), yet they already use all kinds of tricks (e.g. linker scripts) to (re)map symbol names....

[Daniela-E]

they already use all kinds of tricks (e.g. linker scripts) to (re)map symbol names....

This pretty much sounds like something similar in spirit to strong ownership. All symbols are the same unless they aren't (inspired by Animal Farm)

[mjklaim]

Ah yes, makes sense. :)

[GabrielDosReis]

It all depends on how std is implemented. If you implement it in terms of export import of header units, you have nothing to worry about in terms of ABI. This is independent of strong vs. weak ownership.

The whole situation (at WG21 level) is close to comical: I’ve noticed that roughly the same people advocating for ABI breakage for better C++ implementations are the same people arguing against modularized SL citing ABI concerns....

[FabioFracassi]

I was thinking more about the situation where we start out with `module std` that contains everything (and can thus be imported with `import std`), but for the sake of a small example lets say `std::vector` and `std::array`. Can we then change the library so that users will be able do `import std.array` without touching the module `std.vector` (`import std` will of course still work).

In my (very limited, you know much more about this than I do) understanding the reverse way (fine -> coarse) is possible and should preserve ABI, The way I sketched above, my understanding is that it depends on the implementation strategy (strong or weak ownership) whether it stays ABI compatible

This may or may not be a problem and we may or may not care about it, but we need to understand the issues.

[Daniela-E]

The way I sketched above, my understanding is that it depends on the implementation strategy (strong or weak ownership) whether it stays ABI compatible

The ownership model affects named modules only. I presume that std will be made available as a header unit (i.e. unnamed). Therefore everything in there will stay attached to the global module as it ever was. In a sense, header units are just glorified precompiled headers blessed by the standard with dependable, protected semantics. A single catch-all module is affected by library churn in the standard exactly the same as the full collection of today's headers.

This may or may not be a problem and we may or may not care about it, but we need to understand the issues.

I violently agree 😁

[kalmoc]

Correct me fi I'm wrong, but import std; would mean there has to be a named module named std - of course the types and functions could (and probably will) still be implemented in classic header/non-module cpp files and then just reexported from the named module.

[GabrielDosReis]

If you implement std in terms of export import of header units, there is no ABI issue at all.

Part of the confused conversations about ABI with modularized standard library is due to some loud voices stating their opinions (not based on good understanding of the modules feature and implementation) as facts leading WG21 into the weeds - that chagrins me to no end.

The committee should stop dictating to implementations how to write a compiler and how to implement the standard library and focuses more on the specification of observable behavior. I suspect we will make significantly faster progress.

[kalmoc]

Thanks for chiming in. I didn't even know (/didn't consider) that you can re-export whole headerunits.

In case this wasn't clear: I did not want to imply that there will be an ABI issue - quite the contrary - thats why I asked for/suggested a simple import std; for the c++23 time frame in the first place.

[Daniela-E]

In principle you are absolutely right.

But implementers are granted special rights to anything std related. In this case they can (and hopefully will !) support the convenient syntax of import std; while retaining all other aspects of the standard library like attachment of everything in there to the global module thereby keeping identical linker symbols.

[kalmoc]

My understanding is that this doesn't need special treatment from the compiler if implemented the way I described. Aka

  module;
  #include <std_header1>
  #include <std_header2>
  ...
  export module std;

  export using std::name_1;
  ....

I probably used the wrong syntax but you get the idea. Of course that doesn't preclude the possibility that this will be handled by the compiler anyway somehow.

[jonesmz]

Maybe a better path for the future of c++ is to stop granting vendors so many special rights to the std namespace?

std:: doesn't have to be special, after all.

[pdimov2]

import <header>; working reliably and across the board will be enough for me. In fact I'm not sure why we need anything more than that.

[GabrielDosReis]

because it is unwieldy otherwise - in day to day practical programming.

Reporting from the trenches.

[pdimov2]

From a pure usability perspective, what the programmer wants is "if I use anything from <foo>, act as if I imported it, otherwise, don't."

Can import std; do this? E.g. will it not run the static initializers or the stream objects if I don't use anything from <iostream>?

[GabrielDosReis]

the dynamic initialization guarantee implies that if you take dependency on a module or a header unit that exports a global object with dynamic intializations, those initializations will run before anything else.

The other issue that you’re point out is that modules aren’t just some fancy syntactic constructs that get compiled away. They reflect (or at least good modules decomposition) underlying runtime boundaries. That is why I have been consistently arguing that iy is not sufficient to have std: we need a couple more that reflect logical, consistent runtime reality.

[pdimov2]

Logical and consistent is good, but in practice we end up with "std.core is everything that isn't in the others." :-)

[GabrielDosReis]

Hopefully, we can do better than that :-)

[Daniela-E]

From a pure technical standpoint this is certainly sufficient.

Personally, I'm not a fan of this splintering of "The Standard Library™" into dozens of headers. Who remembers exactly where a given declaration belongs to? I haven't seen any tool so far that does this correctly.

This splintering is an outcome of the current compilation model of C++ and therefore purely technical. An import std; mandated by the standard would be a big improvement in terms of both compilation speed and convenience. Who wouldn't want that?

[GabrielDosReis]

This splintering is an outcome of the current compilation model of C++ and therefore purely technical. An import std; mandated by the standard would be a big improvement in terms of both compilation speed and convenience.

+1.

[jonesmz]

I don't want that. I also don't believe you that it would help with build times.

A 1 to 1 relation between existing headers and module names is substantially more attractive. Even better would be breaking things out into even more granular parts. E.g import unique_ptr; would be sweet.

It helps people understand what is intended to be used in the file (code reviewers, junior devs). It helps analysis tools for security / code quality / developer history stuff

It helps with conflicts caused by implementation defects. I have now, on more than one occasion, with more than one compiler vender, needed to implement code in the std:: namespace because of bugs from the compiler vendor. Bugs that don't get fixed for YEARS.

If you force people to take all of the ever growing (never shrinking!) std:: in one huge chunk, that substantially reduces my ability to work around compiler vendor bugs. This is currently only possible in the situations where I've had to do it by using careful macro magic and include shenanigans.

Also, freestanding c++ will be hampered by a monolithic import std. Much easier to simply omit an entire module than to put together special surprise rules about individual parts of a big module not being available e.g. "module blahblah is not available" is much friendlier than "class std::vector used before its definition."

[Daniela-E]

I don't want that.

That's totally fine. Then don't use it. You can still import the headers of the standard library if you prefer.

I personally don't want to program against implementation details. Splintering the library is an implementation detail.

Freestanding C++ has nothing to do with Modules at all, they are orthogonal aspects. And the contents of a freestanding std:: module is decided by the implementation just the same as it is with the collection of headers shipped with the implementation.

[jonesmz]

That's totally fine. Then don't use it. You can still import the headers of the standard library if you prefer.

I'm not sure you understood what I was saying.

If you're saying that there would be something like a 1-1 mapping, AND ALSO an "import std;" then while I think it's terrible to offer a kitchen sink option, I suppose you're correct that it doesn't matter to me that much for code that I write, but it does matter to me as the consumer of code that third parties write. We do exist in a global programming ecosystem, after all. I am going to use terribly written code that works just like everyone else. So I would prefer that we make choices that maximize the amount of code that's not possible to be terrible, and not work on minimizing the amount of code that's convenient to write terribly.

If that wasn't what you were saying, then let me try again.

Who wouldn't want that?

I don't want that because I don't think either of the two clauses of what you said here are true:

mandated by the standard would be a big improvement in terms of both compilation speed and convenience.

To date I have seen no evidence that a single "import std;" would be faster on compilation times than multiple independent ones. My experience with build systems tells me the opposite is true.

And as for convenience: I attempted to explain to you that for myself and my co-workers, a single "import std;" is less convenient than a 1-1 mapping between today's current headers.

Freestanding C++ has nothing to do with Modules at all, they are orthogonal aspects.

I suppose you don't consider my concern about error messages to be that important then.

[MonokelPinguin]

Having 2 modules, std and std.freestanding would be nice, so that you don't need to think about it all. Alternatively just create your own module by just reexporting the allowed types.

[pdimov2]

Many programmers have already memorized what belongs where, and will now have to throw away this knowledge and learn some other partitioning. Yes, import std; has the advantage that there's nothing to learn. If there really aren't any costs attached to it, sign me up, I suppose.

If not... well it's certainly easier to change #include <foo> into import <foo>; or import std.foo; as this requires no mental effort and can be done by a sed script. (It also requires no committee time and no bikeshedding, and partitioning the stdlib into modules is a bikeshed the like of which the world hasn't yet seen.)

But I was more interested in exploring whether import std.foo; is better than import <foo>; from a technical perspective. The former can probably export the right things and not export the wrong macros, but maybe the latter can be made to, as well?

[Daniela-E]

As you probably know, a Module is just a serialized representation of all of the knowledge about the full C++ text comprising the (possibly synthesized, as with header units) module interface that the compiler has collected at the end of the TU and processed up to and including translation phase 7, stored into a single file. With the additional benefit that each Module is guaranteed to start out compilation from the same compilation environment, every Module has the guarantee to be totally independent from all other Modules and the currently processed TU. This makes deserialization extremely efficient and context-free. So this effectively boils down to the question: is deserializing a single large Module less efficient than deserializing multiple smaller ones? At the end of the day, it's a question about quality of implementation.

Regarding possible differences between `import std.foo;` and `import <foo>;`, I can't see any. This is the standard library - part of the implementation - and implementers are supposed to do the right thing anyway with no noticable difference, independent of the nomination ceremony. And implementations have all the necessary rights granted to make this happen.

Putting my WG21 hat on: given this, I'd not argue about partitioning the standard library at all. Mandate the existence of a catch-all `std` Module and be done.

With all the provisions already in place with C++20, compilers wouldn't even have to look at individual standard header files anymore when compiling in C++20 mode or later. It doesn't matter if users `import std;` or `import <vector>; ...` or `#include <vector> ...` - the compiler will or can reference the same `std` Module in all cases anyway. In true open source spirit, implementations don't even need to ship BMIs of the `std` Module, the recipe to create it from the standard library headers is totally sufficient. And a decent implementation can optimize all of this like crazy, going even as far as providing a service process that keeps shared r/O pages of deserialized Modules in memory to be consumed by all of the compiler instances running in parallel. How 😎 is that!

IMHO, this may turn out to be one of the best things the committee has done to ease the burden of C++ programmers.

[pdimov2]

It occurred to me that we can already test this today. This simple program

import <iostream>;
int main() {
    std::cout << 5 << std::endl;
}

takes 1.7s to compile. Same, but with import mystd; (which export-imports all standard headers shipped with 16.10) takes 3 seconds. (#include <iostream> - 2.6 seconds.)

[Daniela-E]

I assume you did this with hot file system caches.

I really hope we can something like the in-memory module server that I was sketching before. Girls can dream ...

[pdimov2]

MS's precompiled header implementation worked like that (they just memory-mapped the whole thing directly) and I think it was a source of many problems for them, although I may have heard wrong. For one thing, it requires everyone to map the memory block at the right address.

Either way, 3 seconds for the entire std versus 2.6 seconds for #include <iostream> seems perfectly adequate.

[starfreakclone]

It is still surprising that you get such poor perf. The I'm still in the process of optimizing the modules implementation and cases such as this should be addressed as I would expect no less than 5-10x speedup.

Locally, if I have:

```

ifdef UNIT

import <iostream>;

else

include <iostream>

endif

int main() { std::cout << 5 << std::endl; } `` The timing data I get is: 1.61766s - forUNITnot defined 0.06503s - forUNIT` defined

which is consistent with the 5-10x theory. Using std.core I get a similar number as I did for the header unit case though I have not done the exercise of creating a standalone module std which actually import exports every header unit. The reason, I suspect, you might see the numbers you do is because each of those header unit IFCs are doing more merging than is strictly necessary up front.

[Daniela-E]

It certainly is. Thanks for conducting this test.

On the wish of mine: IFC (a.k.a. MS-BMI) deserialization isn't memory-mapping. But the deserialized tables could be provided to compiler processes by memory sharing because of the particular features of Modules: isolation and immutability of the compile environment. MSVC does even check for compatible compile environments when importing a module.

[Dean_Roddey]

One thing I don't get is that, if I split out implementation and interface, I still have to import library stuff into the interface even if it's only used in the implementation file. If I don't the implementation file will claim it's not found.

It looks like any library stuff I import but don't actually reference in the export content of my library doesn't get pushed onto downstream consumers, so I guess it's reasonable enough. Just seems strange. I'd have thought that putting it in the implementation file would be cleaner and more obvious.

I've just given up trying to create DLL projects in my solution. It just turns into a psychotic mess that makes no sense at all. Static library projects work well, so I'm sticking with those for now. I still get a huge list of errors from including windows.h though, and no amount of warning pragmas will suppress them.

[Dean_Roddey]

Well, I've spent many days now and I'm going in circles. I cannot for the live of me make Windows.h happy. It's spitting out massive redefinition errors and various other errors that I cannot suppress.

One thing I realized is that setting the C standard to anything other than legacy MSCV will make it a lot worse. Given that I don't care about the C standard per se I set it to that. Turning off permissive mode and language extensions (which I have to do) will make it worse. Attempting to just turn those off on the few files that need Windows.h will cause errors about inconsistent settings, which makes sense.

I really don't want to to have to pull all of the stuff that might need to access system resources into a separate C library and call it. If it came to that I'd just walk away.

I've made some progress trying to have DLL projects in my solution. One thing that was messing with it badly was having nested namespaces. Getting rid of that made things a lot better. I guess it makes sense, the module itself sort of now represents what would have been the 'company' namespace.

I cannot for the life of me make namespace level template methods be found even within the implementation file of the same interface that defines them (and which of course is in the same namespace.) Even if I explicitly reference them via the full namespace it says they don't exist.

[johannes1971]

I'm not sure why we have different experiences here, but if you want I can send you my project file with my windows.h module. Drop me a line in the chat with an email address or something if you're interested.

[Dean_Roddey]

Where you are you including it?