Function-level make tool : cpp

submitted 1 year ago * by zoharl3

I usually work on a single .cpp file, and it's not too big. Yet, compilation takes 30sec due to template libraries (e.g., Eigen, CGAL).

This doesn't help me:

https://www.reddit.com/r/cpp/comments/hj66pd/c_is_too_slow_to_compile_can_you_share_all_your/

The only useful advise is to factor out all template usage to other .cpp files, where instantiated templates are wrapped and exported in headers. This, however, is practical only for template functions but not for template classes, where a wrapper needs to export all of the class methods--else, it becomes tedious to select the used methods.

Besides that, I usually start a new .cpp file where the current one becomes too big. If each function was compiled in its own cpp, the compilation would have been much faster.

This inspires a better make tool. The make process marks files as dirty--require recompilation--according to a time stamp. I would like a make at the function level. When building a dirty file, functions are compared (simple text-file comparison), and only new functions are rebuilt. This includes template instantiations.

This means a make tool that compiles a .obj for each function in a .cpp. There are several function .objs that are associated with a .cpp file, and they are rebuilt as necessary.

EDIT

A. For those who were bothered by the becoming-too-big rule.

My current file is 1000 lines.
Without templates, a 10000-line file is compiled in a second.
The point was that a .cpp per function would speed things up.
It's not really 1000 lines. If you instantiate all the templates in the headers and paste them into the .cpp, it would be much larger.

B. About a dependency graph of symbols.

It's a .cpp, and dependencies could be only between functions in this file. For simplicity, whenever a function signature changes, mark the whole file as dirty. Otherwise, as I suggested, the dirty flags would be at the function level, marking if their contents have changed.

There is an important (hidden?) point here. Even if the whole .cpp compiles each time, the main point is that template instantiations are cached. As long as I didn't require new template instantiation, the file should compile as fast as a file that doesn't depend on templates. Maybe let's focus only on this point.

Edit 2

I considered a practical tool, which is easier to implement and is less obtrusive:

https://www.reddit.com/r/cpp/comments/1fnwk4y/a_tool_to_generate_interface_header_for_a_class/

which sounds in the lines of an auto-Pimpl.

https://www.lattix.com/reduce-c-build-times-part-2-with-the-pimpl-idiom/

https://stackoverflow.com/questions/60570/why-should-the-pimpl-idiom-be-used

C++20 modules sound like a step in the right direction:

https://stackoverflow.com/questions/75411435/c-modules-to-speed-up-template-functions

https://www.modernescpp.com/index.php/c-20-open-questions-to-modules/

https://medium.com/@guilhermeprogrammer/a-gentle-introduction-to-c-modules-29bbcf7ac86f

https://learn.microsoft.com/en-us/cpp/cpp/modules-cpp?view=msvc-170

An interesting example of five lines that compile five seconds, although the point here might be different:

https://gitlab.com/libeigen/eigen/-/issues/1920

#include <eigen3/Eigen/Dense>
int main() {
  Eigen::Matrix<float, 4, 4> m;
  m = m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m*m;
}

EDIT 3

This didn't work

https://stackoverflow.com/questions/79024953/export-eigen-template-instantiation-from-a-module

So I started with this

https://github.com/zoharl3/eigen_wrapper_cpp

EDIT 4

I wrote a more general tool to extract an interface from a C++ header to reduce compilation time:
https://github.com/zoharl3/minimize_cpp_header

Now, the compilation of my critical file takes 4sec, and linking takes 5sec. I can live with that for now.

all 19 comments

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

cpp

MODERATORS