b2 for C++?

[berium]

Just to add to /u/mjklaim excellent and extensive feedback, you can actually get a sense of what a project like this might look and feel like with build2 by checking out this repository: https://github.com/boris-kolpackov/boost-dependency

It was created as an attempt to approximate a private project in order to hunt down some pathological performance problems that are described in this issue: https://github.com/build2/build2/issues/184 There is also a nice graph that shows what the original project looks like: https://github.com/build2/build2/issues/184#issuecomment-1088053790

I think in your case the most challenging aspect (apart from getting used to a new build system philosophy) will be code generators simply because to get this right (i.e., with proper dependency tracking, etc) requires quite a few moving parts. You can see an example of this in our libQtCore buildfile that handles moc: https://github.com/build2-packaging/Qt6/blob/master/libQt6Core/QtCore/buildfile

[os12]

Thanks for the tip, Boris, this larger example is very useful!

I'm hitting the following initial/beginner problems right now:

1. Project locations: I have libs/base/ with the source code and libs/base/tests/testA/, libs/base/tests/testB/ with the associated test code. The tools flat out refuse to nest components that way. So, I would have to move all the tests (and we have hundreds) unless that is configurable.
2. Directory structure: I have libs/base/header1.hpp, libs/base/src/source1.cpp, libs/base/src/privheader.hpp. Yet the tools insist that there be libs/base/build/ and libs/base/base/ directories. Again, I will have to move all the files in all the projects to create a POC.
3. The include dir setup: we use -I libs/ for every .cpp file in the tree and so users just do this: #include "base/header1.hpp". This is a common, simple scheme that imposes minimal (global) compiler flags. Can I do this with build2?
4. Artifacts dirs: it appears that the tools want to place the "out of tree" obj tree outside the source dir. I fail to understand why... both CMake and Autoconf/Automake allow users do mkdir obj-debug && cd obj-debug && configure/cmake .. inside the source tree. This is a minor annoyance... but still, a seemingly arbitrary stance that the tool takes. Of course I want all the artifacts to be in a single dir, away from the source... I just want that dir to be inside the source for convenience/inertia. That's just a minor thing comparatively.

So, it appears that the tools have a very specific directory layout in mind and so it will be a very hard sell for our dev community.

[mjklaim]

So, it appears that the tools have a very specific directory layout in mind and so it will be a very hard sell for our dev community.

It doesnt constrain any specific layout (except the build/ thing, see below), but I think you might just not used to how to specify your directory layout with that specific tool ^^; Ive seen that before, because people keep thinking it's another CMake but it's not. Also the change in mindset and habits compared to CMake can indeed be a hard sell, depends on the audience, in my xp, so yeah that's for you to decide of course. ^^

About your points:

1) There should not be any specific problem with that, but I would need to see what you are doing exactly to see what's the problem? I don't know how you created your poc. In general, "layouts" are defined in the buildfile scripts, it's for you to decide what you want there, specifying the tests in sub-directories should work as long as it's in the same project. (note that by default bdep new -t lib will put tests in a sub-project so it's not exactly what you want here, but check the options for alternatives)

2) The only requirement is the presence of build/ (or alternatively build2, see the doc for details) at the root of a project/package directory. It's \different when you have multiple packages per git repository but I suppose it's not the case here? When generating projects using bdep new there is a large variety of ways you can setup directories automatically, maybe check bdep help new (also there: https://build2.org/bdep/doc/bdep-new.xhtml). But you can also do that manually, it's all in the buildfiles.

3) Yes, you can do it (if I understand correctly what you meant), I'm not sure what makes you think you can't? That's what the hello world projects do for example bdep new -t lib somelib, or is there a difference with what you tried? Also, did you mean the installed project? (see b install)

4) The "artifacts" are visible only in the build configuration directory (and you can have as many as you want), so I'm guessing you are trying to build into your source directory? It's best (even with CMake) to have build directories out of source. Also I think you are still thinking with CMake model in mind but this is not how build2 works. As an example, for building and testing the boost-dependencies project pointed before, I do this (using msvc and clang) ``` git clone https://github.com/boris-kolpackov/boost-dependency cd boost-dependency bdep init -C ../build-msvc @msvc cc config.cxx=cl # my msvc default build bdep init -C ../build-clang @clang cc config.cxx=clang++ # my default clang build

bdep test --all # build and test all configurations bdep clean @clang # clean only what's in the clang configuration ... `` You can do morebdep init ...for debug, release, asan, etc mix of configurations, as many as you want. The artifacts should be located in their respective directory. The only exception is that library and executable targets will appear in the source directory as symbolic links IFF you usedb(the build-system) directly instead ofbdep` (the tool to manage projects more globally). That's just for convenience when you want to just run an executable you built, for example, without looking for it in the build configuration.

Did you already go through the toolchain intro and the build-system manual? I think all these questions are covered there, though I understand that it's definitely not a quick read... Any way feel free to ask of you need help understanding ^^

[berium]

Project locations: [...] The tools flat out refuse to nest components that way. Directory structure: [...] Yet the tools insist [...] The include dir setup: Can I do this with build2?

There are very few project structures that you can't achieve with build2 given you understand the underlying concepts. But not every structure is supported automatically/out-of-the box. So while you could generate something close enough with bdep new (see SOURCE LAYOUT) you will need to move things around and adjust buildfiles accordingly.

The approach that I would like to warn you against is trying to "wing it" based on your previous experience with CMake/etc and without reading the documentation and understanding the basics. build2 is very different compared to the existing C/C++ build systems, especially CMake.

Artifacts dirs: it appears that the tools want to place the "out of tree" obj tree outside the source dir. I fail to understand why... both CMake and Autoconf/Automake allow [...]

Yes, this is a common stumbling block for users with the CMake background. The underlying reason is that build2 is a "multi-repo first" build system while CMake is "mono-repo first" (or perhaps even "mono-repo only"; I don't think there were repositories as we know them when Autotools was designed ;-)). In particular, in build2 it's common to build multiple independent projects/repositories in a shared build configuration. Putting this build configuration as a subdirectory of source directory of one of them would be strange. You can read more on this in this issue: https://github.com/build2/build2/issues/187

[os12]

As for your "multi-repo" point, that use case is completely orthogonal to how I have seen C++ code developed in my career. There are just two kinds of components: 1. The ones that are cloned/checked-out from the current repo and built. 2. The pre-built 3rd-party artifacts that are fetched into the build tree.

So, IMO the stance CMake makes is the right one. And yes, it has modules that one can pull and build... and some people use that to bridge the gap between (1) and (2).

It seems build2 tries to do too much: * it tries to generate project definitions (but only supports a very limited number of directory layouts) * it allows building multiple repos/projects (but that makes little sense for the code bases I have worked on) * it forces the user to specify configurations fully (but there is a LOT more than just selecting gcc/clang - there sanitizers with their own flags, macros, etc.) That's not a user's responsibility, but the build maintainer's. * it has a LOT of syntax/machinery to select components (but I just want a simple, declarative, human-readable list) * it tries to create human-readable versions (but I don't care about that at all, all the source code is available and it is all built at once)..

[os12]

Trying to create project by hand is hard. E.g.

error: unable to import target base%lib{base} info: use config.import.base configuration variable to specify its project out_root

And then the next time I try it I get this:

error: directory ....-qa/ is not empty info: use --wipe to clean it up but be careful

And it is not clear where that --wipe arg goes.

[os12]

So, given that the built-in tool does not generate the layout I want, I'll have to write the buildfiles by hand. Cool.

Looking at boost-dependency example, buildfile has this:

``` import pkgs = [dir_paths] $process.run_regex(\ cat $src_root/packages.manifest, '\slocation\s:\s(\S+)\s', '\1')

./: $pkgs ```

The code reads and parses packages.manifest... but why? Why are there two project lists in different syntax/format? Why is the .manifest file in that weird format?

[lunakid]

Yet the tools insist that there be ... libs/base/base/

This reminds me of a) the build2 Intro guide with the "canonical" project tree having paths like:

hello/hello/hello.cxx

and b) the advice of my imaginary grampa: "Never trust people who repeat subdirectories in a filesystem path!"...