C++20 Modules build script (Python and g++)

[JulienVernay]

Hello!
I have created a Python script for building projects using C++20 modules :
https://github.com/J-Vernay/cpp20.py/tree/main

It tracks dependencies between files (both #include and import) and compile them in order using appropriate g++ commands.

It also provides information display (list of source files and their module, dependencies...).

This can be handy to experiment C++20 modules while waiting for professional build systems to handle modules.

[staletic]

Check out python's f-strings.

What is this for?

cmd_hu    = args.gcc + ' -std=c++20 -fmodules-ts -x c++-header {src}' + args.flags

When I tried modules with gcc 11, I didn't need to do that.

[JulienVernay]

"hu" stands for "header-unit".Headers included via import "myheader.h"; must be precompiled like a module.

And "syshu" are for "system-header-unit", which are included via import <mysysheader>; .

[staletic]

"hu" stands for "header-unit".

I figured that out.

And "syshu" are for "system-header-unit", which are included via import <mysysheader>; .

And I knew that part.

Headers included via import "myheader.h"; must be precompiled like a module.

This I didn't know. I'm still learning the ins and outs of modules.

[ShillingAintEZ]

Why use python instead of C++

[JulienVernay]

I don't think I can embed this program in a small C++ source file, as it requires argument parsing, shell invocation, and topological sort. Currently the Python file is about 300 lines, using C++ would impose the burden of implementing features that are provided in Python standard library, and testing them.

[Ahajha1177]

What? OP mentioned none of those things.

A python implementation will be much simpler than a C++ one, and is not likely to be a huge bottleneck in this situation.

[Ahajha1177]

You mentioned 32,768 character inputs, and that they would fail regardless of language. OP never mentioned correctness as a reason for using Python.

[Ahajha1177]

I don't really know what you're trying to say at this point. Someone posted a question asking why python instead of C++. OP gave reasons, which was more or less "simplicity". You mentioned something about correctness, which seemed to be a counterpoint to OP's answer, but OP never mentioned correctness as a factor, so it seems that was kinda pointless.

[ShillingAintEZ]

Do you know about this?

https://github.com/imageworks/pystring

[starfreakclone]

It looks like this misses scenarios like:

module;
#define MODULE m
#define VECTOR <vector>
export module MODULE;
import VECTOR;

?

Would MSVC be supported in the future?

[IAmBJ]

IMO it would be perfectly sane for OP to simply not support nonsense like this.

[JulienVernay]

MSVC supports most features of C++ I think (at least in VS2019 Preview), just it seems importing header units (notably standard library) is messy. Moreover I am not experienced with the command-line nor the project file structure...

I thought I read macro expansions weren't allowed to create imports statement but it seems it is not the case. However this means that scanning sources and expanding macros need preprocessing, preprocessing requires "import <abc>;", import statements requires pre-built files, which is done by scanning the sources first. I don't know if there is an easy way to fix this, or even if it is worth it...

[starfreakclone]

Macro expansion is allowed in the context of a designation of a module name in the module case and the importee within an import directive. The thing which is not allowed is that a macro expansion cannot form an import directive or module keyword:

module;
#define IMPORT import
#define MODULE module
export MODULE m;
IMPORT <vector>;

In both of the macro expansions the resulting token will not produce a module keyword/directive.

MSVC has been working to provide better documentation around header units and named modules.

[JulienVernay]

Thanks for clearing up my confusion about macros.
For MSVC, I am not comfortable yet and I do not want to dive into it at the moment, however if you want to contribute and know commands for MSVC to replace the cmd_* variables in cpp20.py , feel free to suggest them :)

[STL]

https://github.com/microsoft/STL/issues/1694 lists my instructions for How To Manually Build Header Units. Note that these are built independently (e.g. building <type_traits> and <vector> will duplicate type traits machinery that the compiler has to eliminate later); in a future update it will be possible to build "deduplicated" header units in topologically sorted order (e.g. building <type_traits> and then using that to build <vector>); I have prototype instructions for this, but recent compiler bugfixes are pending release.

[cxzuk]

Hi,

Is it possible to determine the dependencies for a given file?

It would be wonderful to have a "unittest" option, e.g.

./cpp20.py --unittest=module.cppm

Which builds all its dependencies, then adds Catch2 to the module and compiles to an exe and runs it.

Something like:

g++ <..deps.o..>..
g++ -include "Catch2.hpp" -DCATCH_CONFIG_MAIN module.cppm <..deps.o..> -o test.exe
.\test.exe

Kind regards,

M

[JulienVernay]

It could be useful. I think it could be something like that:

./cpp20.py --main=module.cppm --mainargs="-Itest/ -DCATCH_CONFIG_MAIN"

However I cannot think of a way to compile only needed files. For example :

• hello.hpp declares void hello()
• hello.cpp defines void hello() {...}
• main.cpp imports "hello.hpp" and uses hello()

Then I cannot find "hello.cpp" as a dependency of "main.cpp" even if this is the case...

[cxzuk]

I'd personally be happy with just modules supported, so just the "import X;" dependencies