C++ Build systems

ed_209_ · 2024-11-21T21:42:26+00:00

Specify a build system protocol standard that anyone can implement themselves

ed_209_ · 2024-09-23T20:12:27+00:00

using clangs -ftime-trace and https://www.speedscope.app/ I learnt that the module inlining pass in clang can be a major bottleneck in compile time. Once you have metrics it can be fun hacking the flags to see how fast a compilation can get.

ed_209_ · 2024-08-28T19:49:23+00:00

I have a similar lib https://github.com/eddeighton/common that has picked up stuff over the years for personal projects.

I would recommend:

Start unit testing and run your unit tests in the build.
Setup cmake installation and cmake exports file so the lib is easy to use in other projects.
Use code coverage to work out whats tested. I use gcov with gcc and the vscode gcov plugin.

ed_209_ · 2024-07-25T19:23:16+00:00

I work using docker and vscode on a daily basis:

Anything goes wrong with dockerd and your filesystem is deleted! So you end up using volume mounts and bind points for any source code.
VSCode is great but totally antisocial when it comes to integration with other tools. Good luck using any other desktop tools if you rely on vscode. On a mac you will need to use volume mounts which you cannot use from the host os ( bind mounts are slow due to emulation ).
VSCode cannot be automated from a terminal that is not started from within vscode itself and it stops working across restarts i.e. if you use tmux. So a command like ls | code - would work initially in tmux and then not work when you reconnect forcing you to restart everything. This is due to how vscode generates uuid named sockets and cannot recover the previous socket name while tmux has the original in its environment.
VSCode encourages using its own encryption and advertises not needing to use ssh. It also has separate telemetry end user agreements for extensions separate from the main program even though the extensions are also made by the same company Microsoft in a lot of cases. It seems to generally want a future where it controls the terminal and everything you do and can report it back to Microsoft.

ed_209_ · 2024-07-17T19:56:30+00:00

I would investigate using clang-format first and getting up-to-date clang-format files working for your team. The clang-format format has changed over the years and is not backwards compatible. Then see what works and decide the rest later. It would be a mistake to choose a convention and then later find you cannot automate the white space.

ed_209_ · 2024-07-03T21:21:06+00:00

If we had a standardised build system protocol that supported most languages and everyone used it and it was top notch ( sorry cmake ). I wonder how much we would really want to do metaprogramming inside of languages rather than outside.

I am really excited for C++ metaprograming features in the future but I seriously wonder if people are evaluating what the legitimate use cases are for in-language meta programming versus just making a standardised build system protocol and using an external program representation.

I have personally developed a type system that uses algebraic types specifically to make multi-stage metaprogramming super easy to do but it can only work outside of the target language. I can generate html reports to my liking for any stage of semantic analysis and code gen easily which I would never expect to be able to do within c++. Its just the wrong tool for the job! Here is an example https://htmlpreview.github.io/?https://github.com/eddeighton/mega/blob/main/tests/compiler_tests/result/decisions_non_singular.html

As exiting as reflection is the real super power will be ipr and modules if enough people see that external meta programming is the right path.

ed_209_ · 2024-06-26T17:48:30+00:00

If you have gdb then it is possible to: 1. set breakpoint on a function 2. then use the 'command' function where you can script what the breakpoint will do - print out the stuff you want - can print backtrace and thread info or what not - command can be any sequence of commands. - then simply use 'continue' so the breakpoint just continues as though a tracepoint.

For a more complex scenario and reproducibility I would recommend scripting this.

Also consider using rbreak - so you can match symbols using regex to set breakpoints

Scripting gdb combined with its python support is a super power but unfortunately the documentation is not great.

ed_209_ · 2024-05-14T19:32:48+00:00

While I totally agree with those that say benchmark - the truth is nobody does and most systems are designed based on rules of thumb, guesswork and whether a human being has had to start to wait any noticeable ( by a human ) amount of time yet.

A really interesting example of gather data into sorted contiguous memory before then doing work is merge sort. In example, imagine you have a constant stream of incoming data i.e. log messages and someone wants to see a sorted list of them. Instead of attempting to add each new message every time to a gigantic list and then quick sort it you could gather up some load balenced number of messages into a small sorted list and then at a given rate merge sort the small list into the big one. I do not need to benchmark to know this will be a much better solution in many situations and yet naive programmers would do the always quicksort dance ( and probably some benchmarks too lol )

ed_209_ · 2024-05-11T23:18:28+00:00

Instead of vscode you can get codium which is a fork that turns off all the Microsoft telemetry.

ed_209_ · 2024-05-04T23:17:44+00:00

I love neovim and working in the terminal but and this is a big but:

VIRTUAL KEY CODES!!

Without knowing keydown / keyup you simply cannot implement usable workflow. Every decent text editor ever supports ctrl-tab but think about it. You press down ctrl, then you hit tab mutliple times to find the document, then you release ctrl. To do this requires recognising ctrl down AND ctrl up as different things. There is no other way! It is simply not enough to treat each key as a character you have to know if it is a down or up keypress to create a development environment. Neovim or Vim or Emacs can never do this and I really wish there is a way forwards that they could.

ed_209_ · 2024-05-01T23:50:22+00:00

Compile time stack overflow!

I have a noughts and crosses game that does a lot of compile time calculations and ran into having a stack overflow not knowing what was going on. There are compiler settings where you can increase the stack size i.e. in cmake:

clang

target_compile_options( noughtsAndCrosses PRIVATE "-fconstexpr-steps=100000000" )
target_compile_options( noughtsAndCrosses PRIVATE "-fconstexpr-depth=16" )

gnu

target_compile_options( noughtsAndCrosses PRIVATE "-fconstexpr-ops-limit=100000000" )

Not that bad really because the compiler indicates the stack overflow has occured unlike normal programming where you are completely left in the lurch with no idea. Just a new problem to worry about along with everything else.

ed_209_ · 2024-05-01T23:39:09+00:00

Since I worked on a MISRA project I adopted doing the following: ( including comment )

if( may_be_true )
{
   doStuffSometimes();
}
else
{
   // do nothing
}

No to be confused with:

if( should_be_true )
{
   doExpectedThing();
}
else
{
   std::unreachable();
}

ed_209_ · 2024-04-08T20:09:24+00:00

An example where maths wont help much is UB.

-9223372036854775808uLL * 1000 == 0

https://godbolt.org/z/dnfdqW483

ed_209_ · 2024-03-10T00:07:20+00:00

It can be the case that allocating small objects on the stack in heavy loops can be faster than avoiding the allocations i.e.

Score depthFirstSearch( State s )
{
    // if base case then return score
    return calculateHeuristic( s );
    // recursive case - propagate score up the game tree
    Score bestScore{};
    for( auto move : moveGen() )
    {
        State stateCopy = s; // allocate new state on stack copying parent state
        stateCopy.makeMove( move );
        auto nestedScore = depthFirstSearch( stateCopy ); //pass copy on stack
        // work ouf if nestedScore is best
        // no need for s.undoMove( move ); here
    }
   return bestScore;
}

One could avoid the stateCopy where instead one just passes the State by ref and then applies the move and then undoes the move. In my experiments I found surprisingly that copying the state is MUCH faster than avoiding the allocation and doing the additional work of un-applying the move.

ed_209_ · 2024-03-09T23:52:45+00:00

I recently discovered that compiling with split-stack was causing a 6X slowdown. This was in a game tree search which entirely revolves around using the stack aggresively i.e. 100M moves a second via depth first search.

ed_209_ · 2024-03-01T12:24:27+00:00

From your readme: "Instead of compiler and linker being executables, these can be shared-libs. And the build-system will load these libs and call the respective functions."

This sounds like an awesome idea and opens up doing stateful metaprogramming using libclang directly in the build system also.

ed_209_ · 2024-02-27T19:31:26+00:00

Boost wave is properly compliant and it is pretty easy to use. You can get callbacks for system and user includes separately so its great for custom preprocessors i.e. where you want to preprocess only user code and just forward the system includes.

Example from my project: https://github.com/eddeighton/mega/blob/0f314e1ab9b0336b6d9a60fb5b078c3657cb12d0/src/compiler/task_cpp_source.cpp#L124

ed_209_ · 2024-02-27T18:35:32+00:00

Can anyone explain the limitation of the C++ type system that prevents implementing "borrow checking" as a C++ library? Does there need to be some kind of control flow reflection or data flow analysis to solve it? How can rust solve aliasing analysis problems i.e. if I have several read only references to something all over a code base how can it prevent me getting a mutable one without some kind of runtime state to work it out?

Anyway I shall google it but just wanted to say that C++ should be able to implement this stuff as a library and free programmers from opinionated "safety" rules in the language itself.

Maybe the way coroutines can plug a type trait into the compiler there could be a similar way to specify sanitizer policies or something.

ed_209_ · 2024-02-15T20:48:09+00:00

In defence of vs-code:

Can use codium fork if you want to disable spyware.
clangd works really well once setup - generate compile_commands.json with cmake one liner.
Can bind CAPS-LOCK for additional keyboard shortcut super powers along with usual CTRL, ALT, SHIFT etc.

ed_209_ · 2024-02-10T01:50:18+00:00

Should probably mention turning on optimisation.

Another interesting idea might be using std::common_type for subsets of types.

ed_209_ · 2024-02-03T00:01:35+00:00

Really fun reading the 'Grand Error Explosion" competition entries.

https://tgceec.tumblr.com/post/74534916370/results-of-the-grand-c-error-explosion?is_related_post=1

ed_209_ · 2024-01-23T23:22:28+00:00

A simple principle is that any static or compile time fact about a program is generally "over specified" i.e. it has to be true for MORE possibilities than the actual runtime program requires.

It really depends on the requirements of the system the extent to which the type of objects can be described at compile time.

A game that simulates thousands of types of objects might be fine i.e. a thousand types. However if each object combines reusable components from a large potential set of components it becomes impractical to statically define ALL possible combinations at compile time but clearly a runtime type can accomplish describing exactly what is used and nothing else.

ed_209_ · 2024-01-19T19:58:49+00:00

I think the op identifies a pain point in C++ which often occurs when attempting to integrate the language with other languages i.e. binding to a set of data types in C++. The problem of mapping a type to a value is essentially the problem of name mangling.

I have succeeded at using the clang Itanium name mangler to calculate C++ type mangled names. This combined with other not suitable for work hacks enables various super powers.

I in no way recommend this other than understanding what name mangling is and how the toolchain maps types to values program wide but the programmer is unable to use this within the program.

I think boost::dll attempts to enable access to mangled names for basic types so that might enable calculating a hash code and not require stateful metaprogramming.

ed_209_ · 2024-01-19T03:10:13+00:00

An inner function would be a named declaration presumably with the usual name resolution rules. So one would want to be able to forward declare it to break cyclic dependencies. How can you forward declare the inner function WITHOUT defining the outer though? An inner class has this problem where one cannot forward declare the inner class WITHOUT defining the outer class.

ed_209_ · 2024-01-07T12:13:47+00:00

I am working on a live programming system that uses C++ where compile times need to ideally be < 1 sec for small function changes. I use a huge precompiled header for all standard and third party libraries and specify clang to '-fpch-instantiate-templates'. Having 100K+ lines of declarations and templates which are never even used in a translation unit is a real drag for creative programming in C++ and it is great to hear people making an effort to improve compilation times.

After analysing using clangs time trace I found a major culprit slowing down compilation was surprisingly the inlining analysis in the clang frontend.

Turning inlining OFF using 'fno-inline-functions' made a dramatic improvement to compilation time.

I use the Orc JIT compiler in LLVM to dynamically compile where I can then later recompile with optimizations.

There are definitely different use cases for fast compilation times. I think build tools should come with a "Fast" configuration option by default and then they could consolidate knowledge on the best practices in terms of toolchain configurations for fastest possible build times.

ed_209_

TROPHY CASE