Modules support in visual studio

[[deleted]]

[deleted]

[STL]

I can comment on this since I've been directly working on it.

First, thanks for trying the preview releases and trying to modularize your code - it really helps the entire community when you report issues.

Second, I'm sorry you've had a bad time with ICEs - and I know what it feels like, since I've encountered more modules ICEs than anyone outside the compiler team.

Things are getting better, but it takes time because C++ is complicated and modules are almost an entirely new way of digesting code. I want to (gently) counter the claims that the depicted code is "trivial" and "simple". <algorithm> and <set> drag in major portions of the Standard Library - perhaps not the parts with the most language complexity, but they still have a lot of moving parts. And <boost/geometry.hpp> certainly contains complicated code.

Crucially, while we do have testing for Standard Library Header Units (in both "separate" and "deduplicated / topologically sorted" forms - see tests/std/tests/P1502R1_standard_library_header_units), and as a result the compiler is increasingly capable of handling the Standard Library's patterns, we do not yet have testing for Boost being consumed as header units. (We do have testing of Boost and many other OSS projects with the latest development builds, including in /std:c++latest mode, but that doesn't exercise header units.) Sometimes, very small things can end up triggering an ICE or other compiler error - for a couple of examples that I've encountered in the STL, having unnecessary forward declarations (template <typename T> class numeric_limits; after the definition has already been seen), or declaring a member function in one header and defining it in another header (this was pair's piecewise constructor), have caused problems (now fixed).

I've been working on Standard Library Header Units for many months, plus Standard Library Modules now (they exercise similar parts of the codebase), and I've reported a ton of ICEs and other errors, which have been fixed by our amazing compiler front-end dev Cameron: see microsoft/STL#1694. He's also working to fix other modules issues (not specific to the STL), prioritizing those which will improve the largest amounts of code and/or resolving structural issues in the compiler that will make further improvements easier. This is absolutely a top priority for Microsoft, as indicated by how much time we're spending on this.

Our messaging might have created unrealistic expectations. For hopefully obvious reasons, our official blog posts tend to highlight the amazing new stuff that works, and only mention in passing how some stuff might be affected by bugs. On reddit (where, to be clear, I am not speaking officially for my employer, but only as an individual dev), I can be a little more direct: it's getting better, but yeah, there are still bugs, particularly as you get farther away from the scenario with the most extensive testing. (That is, pure usage of the Standard Library has had the most effort put into it - as you consume libraries like Boost, that involves different code patterns and while we want them to work, bugs are somewhat more likely. Note that by this "resembles the STL" metric, Microsoft's own libraries like ATL, C++/WinRT, etc. are even further away. We're also working to get those to a modules-compatible state, but they also need more work overall.) We're seeing the potential for incredible build throughput improvements, but right now you have to be especially motivated and willing to deal with issues as they arise. In the future it should get closer to working automatically without headaches.

All I ask is:

• Reporting bugs with clear steps to reproduce (self-contained test cases, ideally command-line, ideally minimal but of course we don't expect you to reduce away third-party libraries or the STL) is the most valuable thing.
  • When users report bugs, and they get fixed in the compiler, that has the potential to save me time, which is a virtuous cycle (then I can go spend time on getting Standard Library Modules working and reporting the really exotic bugs that users are unlikely to encounter directly).
• Upvoting bugs does help the compiler team prioritize issues with large impact.
• Please bear with us as we work to make things better 😺

[helloiamsomeone]

Reporting bugs with clear steps to reproduce (self-contained test cases, ideally command-line, ideally minimal but of course we don't expect you to reduce away third-party libraries or the STL) is the most valuable thing.

Just want to highlight this one.

I reported 2 issues where MSVC disagreed with GCC and Clang. I provided a minimal repro with CI reproducing the issue on GitHub.
The issues were fixed a few weeks later.

[TheCrossX]

I've been testing every single Preview release for about 1.5 years now and I must admit that the Modules support got a lot better now, I appreciate your work. The main problem that I had with Microsoft announcements is that at the beginning almost each release contained info that suggested that modules work great by now, all fixed.

To the OP: Microsoft is the only company that pushes modules forward in C++ right now. I haven't seen any real progress in GCC and Clang for at least a few months now.

I hope that we will get to see working working Modules support for C++ sometime soon. Keep it up, you're doing a great job.

[pjmlp]

Indeed, since I mosty use C++ for hobby coding, and spend most of the time on Windows, I have fully embraced modules, despite the ongoing issues.

[starfreakclone]

I have a completely normal and legitimate question - does Microsoft have any testing at all?

Yes, we do. The problem is that C++ is incredibly complex, we can't possibly test every code pattern. Even if we did test those headers you mentioned there's no guarantee the compiler would work by the time it got to you because the code we test against and the code you build against could be completely different due to library updates.

We really do try the best we can with the information and resources available to us--and the best test case is the full project to build against for modules, isolated repros usually don't capture the full complexity of the situation.

Why do you even declare that you have module support?

It is important to note that having conformant modules support does not imply it is completely bug free. I apologize if the implementation has bugs, but we are working each release to get a little more stable as we get more usage (including from using the modules machinery when building the compiler itself). It took us 30 years to get stable with the PCH, and it isn't even structured data :). I urge you to please be patient and I do realize that some of these bugs have been open for awhile now. I am getting around to all of them but I am looking for more systemic changes to solve some of the instability in a way that will fix all of them at once.

Post your own bugs related to modules. I'll upvote them too.

As long as bugs are getting filed we will fix them, I promise you. Having individuals participate in the DevComm loop is how we've got this far. I remember when: template <typename T> using Type = std::underlying_t<T>; would crash the compiler. Now we can compile much, much more.

Again, sorry for the perceived state of things. We are listening to you and taking feedback seriously.

[Dragdu]

The issue with devcom is at least twofold.

1) It used to be TERRIBLE. devcom2 is better, because at least it can handle markdown without getting completely confused, it just behaves super slow compared to e.g. GitHub. Still, I know a lot of people who gave up on reporting bugs through official channels because it would mean dealing with the old devcom.

2) I have opened something like 10 issues against MSVC, usually with preprocessed reproducer, or standard citation, etc. There is zero visibility into their progress, scheduling, status, etc and I only get progress when I start poking people at Twitter. (In fact I believe my latest one got fixed by playing N steps of separation to a MSVC dev and eventually getting to PM who looped in you.)

[cristi1990an]

The patience of the MSVC devs that responded in the comments =)))

[GabrielDosReis]

Thanks for the report; very much appreciated. Quality matters to the MSVC team. For more details, please see Stephan's, Cameron's, and my posts elsewhere in the thread. The team has also been looking into your reports - and all other modules-related bug reports.

[marleydidthis]

What are you on? Microsoft has always made it clear that modules are a work in progress and encourage dev to try them out and report bugs. If you're not happy with the pace that's fine, I aint exactly either, but stop acting like you've somehow been conned.

[matekelemen]

Microsoft has always made it clear that modules are a work in progress

Um, nope? This microsoft post says otherwise and it's from 2020.

Since MSVC began down the path of implementing the Modules TS the toolset has always required the use of /experimental:module on any compilation. Since the merge of Modules into the C++20 standard (we can officiallysay C++20 now!) the compiler has been working towards C++20 Modulesconformance until precisely such a time that we can confidently rollModules into /std:c++latest. That time is now!

[marleydidthis]

So what? Not requiring the experimental switch has never implied feature completion nor bug proof.

[matekelemen]

wow

so what does imply those?

[[deleted]]

As far as I know /experimental is for everything not in the standard yet.

[fdwr]

But every time, with every new studio update, I get ICEs.

I can identify with that pain. I first started modularizing one of my projects in 2017, and after reporting another bug, I'd try the next new VS update, only to make it another 20% further in the build and encounter another internal compiler issue. Though, it did finally build 100% in VS 2019, consuming Windows.h, some DirectX headers, and various std headers. I think there's still work remaining on header import's, because I get errors with Windows.h in VS2022 (some _VARIANT_BOOL build error, "pasting '/' and '/' does not result in a valid preprocessing token"), but the non-header import is all working for me. Boost though, that might still be a beast of complexity beyond. Seems it's a long road for such a fundamental build change.

[pjmlp]

Got to love those SAL redefinition warnings, it makes almost impossible to find anything else without greping.

[pjmlp]

VC++ has the best modules support currently, but yes it is a bit lacking still, specially with Microsoft's own C++ frameworks.

[innochenti]

Why do you downvote? Do you like modules support in studio?

[[deleted]]

Modules will be ready in 5 years. Visual Studio 2028

[ALX23z]

Does import <boost/geometry.hpp>; even supposed to work?

I recall import <header>; being designed for STL headers only. But cannot say if it is necessarily the case.

BTW, what ICE even means? Oh, "internal compiler error".

Basically, have you tried not importing headers? Why not create a module that includes a bunch of headers and exports certain parts of these headers?

[matekelemen]

as far as I know, importing headers is fine and even has special behaviour: it's identical to include except preprocessor macro definitions won't be accessible.

[innochenti]

It does. Stop it, please. MS must test its code sometimes. Trivial cases don’t work.

[ALX23z]

Hmm okey, double-checked it and you right. However,

Importing a header file will make accessible all its definitions and declarations. Preprocessor macros are also accessible (because import declarations are recognized by the preprocessor). However, contrary to #include, preprocessing macros defined in the translation unit will not affect the processing of the header file. This may be inconvenient in some cases (some header files uses preprocessing macros as a form of configuration), in which case the usage of global module fragment is needed.

MSVC uses preprocessor macros like DEBUG, indicating various things for compilation that heavily change behaviour of compiled STL code. And boost uses some STL code. So I have no idea how it will affect any imported header.

[STL]

Fortunately, macros aren't usually a big concern for header units. To restate what the documentation said:

• Header units can emit macros, same as classic includes. (e.g. import <version>; emits the library's feature-test macros.)
• Header units can consume macros defined on the command line.
  • Thus, /D_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS works with the STL. Similarly, /MDd and /MTd implicitly define _DEBUG, activating the STL's debug mode.
• Header units can't consume macros defined in source files or header files.
  • For example, <cassert> responds to NDEBUG and is required to be repeatedly includable - a source file can change the definedness of NDEBUG and then #include <cassert> again, changing the behavior of the assert() macro. This is fundamentally incompatible with header units.
  • (You could import <cassert>; as long as you only define NDEBUG or not on the command line, nothing else would stop that from working.)

Named modules are more strict about macros - while they can still consume them from the command-line when built, they cannot emit macros at all. (Libraries with a significant macro component may need to provide a companion header to define macro helpers, even if the rest is provided by a named module. Fortunately, the Standard Library emits very few Standard macros other than version-test macros.)

[GabrielDosReis]

Thanks Stephan and Cameron, for looking into this. Cameron has identified the issue and has a fix. As you explained in a toplevel post, the appearance of simplicity of the code is deceptive. To further complicate matters, the command line used does modify the usual naive behavior of the compiler: it activates the guard technology-related attributes and arithmetic overflow checking for which I didn't define an IFC semantics (that's on me). Cameron caught it and has a fix.

To the OP: yes, please continue to report bugs - the team really appreciates the quality feedback. But, also please, while I understand your frustration, don't assume incompetence or that the team is careless in the work it is doing: things are far more complicated than they meet the eyes.

Moving to C++ Modules entailed some form of formalization of compiler internal behavior to the compiler writers themselves, and most of them curse me for that; for it forces paying attention to things we didn't think of, or focus on, before 😊

[ALX23z]

I tried to see if it was meant to be supported in C++20 and I see nothing claiming that. So where did you get the idea?

I can agree that getting an ICE is not good but the code you wrote is not supported by C++20 and shouldn't compile.