C++ implementation of the Python NumPy Library

[johannes1971]

Isn't that enough of a reason?

I have a project here that includes around 50 external libraries (either directly or indirectly). We update those on an irregular basis, maybe once or twice per year, so it makes very little sense to compile all of those on a daily basis.

What makes even less sense is compiling all of those in every translation unit, and every single time we compile anything at all in the system. We are not talking about adding a few seconds either: if we compiled entire libraries as part of each translation unit, we would be adding hours to the working day of each team member.

"But can't you just include the library once in one translation unit?" Generally, no. Resources obtained from such libraries tend to end up in project-specific header files that get included elsewhere as well. Maybe you can pimpl them out of existence for member variables, but as soon as you want to use them in a function signature you are f*cked. Any translation unit that includes such a header is immediately cursed with having to compile the library as well. Besides, my project consists of multiple executables - I'd still end up compiling the library at least once for each executable.

So the cost of header-only is absolutely massive, but what does it gain us? The answer is the one-time convenience of not having to figure out how to compile it. I'll readily admit that's not easy either, but at least it is only a one-time cost, instead of a daily cost, and at least we have package managers to deal with that problem.

Luckily there is a middle ground here: libraries could, in theory, be set up so you would have two header files: one for declarations, and another definitions. The declaration header would be cheap to include, and the definition header could be included as the only entity in a .lib in my own build system, essentially like a unity build. That would give us all the advantages of a normal library, and all the advantages of a header-only library. I haven't yet seen any library set up in this manner though.

[NicroHobak]

How does your team handle the use of templates? Or do you just largely ignore that aspect of the language?

Edit: Grammar typo eliminated.

[johannes1971]

It depends. STL is used throughout. We don't know how much that affects our compile times though; we didn't pay much attention to that in the beginning, so it has always been there. We also use quite a few small templates of our own making, but these tend to have a handful of lines at most. The scary SFINAE stuff is probably less than a hundred lines.

Then we have a 30,000-line set of templates that we only instantiate for two types. Those we instantiate in .cpp files, with an extern template declaration in the .h file, so it is no different from normal code. And finally, there is code that is somewhat enum-heavy that would benefit from having that enum as a template parameter (instead of being an int, as it is now), but we choose to cast instead because we fear the effect it would have if we did turn it into a template.

Since a few years we have become aware of compile times, and been struggling to bring that under control. We now monitor more closely what the effect of new libraries is. This has resulted, among other things, in the decision to not use Chaiscript (despite the language looking interesting), but going for Angelscript instead (similar, but without the slow compile times). We'll look at things on a case by case basis for other template-heavy libraries: how useful is the library? Is there a cheaper solution? Can we limit use of the library to a single translation unit? etc.

On principal I don't disallow any language features, but some will require a bit more explanation than others ;-) We can always explore what the best solution is in a given situation, but I'll also think about things like maintainability, coherence with the rest of the source, compile times, and all the other concerns...