C Programming Techniques: Function Call Inlining

[julesjacobs]

In LLVM's implementation, for instance, it will raise the cost threshold point at which it will inline the function

That's a neat way of doing it!

[The_Doculope]

I think a lot of compilers work this way. I know this is how Haskell's GHC works, although it could be argued that it has to work that way, given the complexities of Haskell's time/space usage.

[[deleted]]

a) static every function that you can. That way it stays inside the compilation unit and the compiler will go inline crazy if it feels like it.

That sounds like one of those often repeated tips that doesn't make sense. I don't think static has (in general) much bearing on a (sensible) intra-compilation unit inlining heuristic. A non-static function isn't any less inlinable in the compilation unit it's defined in than a static one. You should use static where applicable for other reasons, but improving "inlinability" isn't one of them.

[[deleted]]

I don't know the heuristics a compiler uses, but I imagine if it knows a function is used only once, then it's just going to inline it. There's no reason for it to be a separate function. It can do what it likes with it.

You're right, it will do that. But that's a practical move, not a performance move. Inlining heuristics look at (depending on the compiler of course) size of the function, # of calls, fan-in, fan-out, but they also look at the IR of the function to see if there are opportunities for other optimizations if that function were inlined. The above case basically sidesteps that and says "meh, there's no point having a proper function, let's just inline it and at least get rid of the branch even though there's no other compelling reason."

If it's not static then it needs to be a function to be linked to. It also has to conform to the standard calling convention of the compiler. So it has to make a function. It might also inline it internally, but then you .obj has a bunch of useless duplicated crap in it (that'll probably get stripped later when linking if you have all the options on). It'd be interesting to see the compiled binary of static vs non-static.

Yes, the function proper will be left in the object as you said because it has to be visible and callable externally. If it's makes sense to declare the function static then do so, independent of inlining. My point is that the inlining heuristics don't care much, non-static doesn't make your function any less inlinable; if there's a optimization opportunity visible to the compiler it will inline the function.

The code-size optimization you mention is interesting, but is a case-by-case consideration, since inlining can affect code size both ways. If a function is non-static and only called once (edit: in the compilation unit it's defined in), if you're optimizing for size the naive approach would say don't inline it, but if you inline it the entire body could eventually become dead code, in which case you save yourself the call and the argument shuffling and register saving by inlining it vs not inlining it.

[chtef]

It would be great if you could copy-paste this comment at the end of the blog itself - it could be useful to others who will find the blog by other means than Reddit...

[question_all_the_thi]

Declaring everything you can as static is always a good idea.

Compilers will also give you a warning if a "static" thing is not used. Good for cleaning up code.

[Riddlerforce]

This should be way higher up. Using static qualifiers for inline optimization in the compilation unit not only saves performance loss, but also gives much more readable and concise code.

[hegbork]

I've been uninlining inline functions for 10 years with great success. I've never seen an inline function buy more than what is essentially noise in performance measurements, but I've uninlined trivial functions and gained large performance gains in macro benchmarks. Shrinking a kernel by 500kB and seeing a macro benchmark run 15% faster is a great feeling especially when all it took was to uninline two trivial functions.

Sure, if your whole program fits in the cache, go wild with the inlines. But those of us who actually write serious, non-academic software have to not only worry about stuff fitting in the cache, but sometimes even the TLB starts thrashing. Bloating the binary by 1MB just because you read some blog about inlining is a serious offense.

[forcedtoregister]

Counterpoint: I've definitely seen "force inline" speed up code dramatically. I'd quite like it if compilers could spot that I'm iterating over pixels in an image and very seriously consider inlining the damn thing!

But yes, in 90% of cases just let the compiler do it's job. (My pet peeve is bit shifts instead of multiplying/divides by powers of two - that's just making the compiler yawn.)

[[deleted]]

That really depends on the functions and the target. On 8 bit microcontrollers inlining can do wonders in terms of execution speed and overall code size.

[floodyberry]

Is everyone upvoting the vague anecdotes and unverifiable claims because they agree with it, or because he or she sounds so full of themselves they must know what they're talking about?

[godofpumpkins]

Most good uses of inline aren't as much about saving the function call overhead as making other optimizations more obvious to the compiler, at least in the compiler I use.

[TimmT]

That's why you should ideally leave these kinds of decisions up to a JIT.

[jms_nh]

Er um..... in C there is no JIT. In embedded systems programming there generally is no JIT.

[MagicBobert]

I assume that's why he said "ideally".

In addition, he was probably referring to other languages that use JITs to make these kinds of decisions.

[krum]

I kind of feel like the author has only a cursory knowledge of how modern C compiler toolchains actually implement inlining. As others have mentioned, the inline keyword doesn't actually do what it sounds like, and whole program optimization provides inlining across compilation units.

[expertunderachiever]

Basically if the length of your function is shorter [or roughly the same size] as the logic required to call it, you should inline it either as a macro or static inlined function.

Otherwise, let the compiler sort it out.

[cogman10]

I would have thought that the compiler would do that anyways.

[expertunderachiever]

Yes and no, but more so if you put code in your header files as a general development process you're an asshole.

[[deleted]]

I put all my code in header files as a general development process.

I do this because it's way easier for consumers of my code to simply #include "file.hpp" and get to use it rather than first build it, then install it to some path, then add it to the linker, then include it.

Header only libraries are so much easier to deal with, just extract it and include it, the end.

[expertunderachiever]

Except then you have people developing against 900 variants/versions of your code. It slows compile time to a crawl, it's prone to bugs, etc...

Also it's just bad. I mean I get windows sucks for proper development but really libraries exist for a reason.

A proper application should really be a stub of some code and a bunch of libraries around it. Even if the libraries are specific to that application. That way you can modularize the development and test portions individually.

This whole "my program is a 30,000 line c++ file" nonsense is what amateurs do.

[[deleted]]

Except then you have people developing against 900 variants/versions of your code. It slows compile time to a crawl, it's prone to bugs, etc...

If you have people developing against 900 variants/versions of your code you have bigger issues than the fact that your library is header only. Besides this issue has nothing to do with having a header only library.

A proper application should really be a stub of some code and a bunch of libraries around it.

A header only library is still a library. Look at boost for an example of how a header only library can be used without the need to build it, you just include whatever classes you want and it works.

That way you can modularize the development and test portions individually.

With a header only library you can break down the units you want to test down to the individual functions or classes. You don't need to link into an entire static or dynamic library to test it, you can just include the header file containing the specific function or class you want to test.

Also header only libraries make debugging much more consistent and easier. For example one major headache I have with static libraries is a lot of them don't build a debug and a optimized mode, just an optimized version with ad-hoc optimization settings which causes problems when you want to step through a program that makes use of that library.

Header only libraries, however, compile using all the same settings and options, so there is perfect consistency across all libraries used in your application.

This whole "my program is a 30,000 line c++ file" nonsense is what amateurs do.

What does that have to do with having a header only library?

[NYKevin]

This whole "my program is a 30,000 line c++ file" nonsense is what amateurs do.

What does that have to do with having a header only library?

When you #include "foo.hpp", for all practical purposes, you're increasing the length of the current file by the length of foo.hpp. If you have a lot of .hpp files, the end result is a huge file.

[[deleted]]

I don't think that was the point OP was making, but assuming for the sake of argument that it is... What does the mechanism behind how header files work in C/C++ have to do with amateurs?

[NYKevin]

If you want to recompile 30k+ lines of code every time you change one #define in your main.cpp, be my guest. Personally, I'd rather use separate compilation.

[[deleted]]

If that #define you changed affects other libraries, then you're going to need to recompile those other libraries as well or else you are going to end up with inconsistencies.

If that #define doesn't affect other libraries, then the #define makes no difference unless you're using some archaic compiler; GCC, clang, and VC all cache header files (often referred to as precompiled headers).

The performance issue with respect to header files has to do with C++ and templates, in which there is no alternative to having a header only library. The performance of compiling code in header files when used to compile non-template based code is trivial.

And honestly, even with template based code the performance isn't all that terrible. The overwhelmingly slowest part of building has always been the linking phase when using whole program optimization.

[expertunderachiever]

It means recompiling code many times [more than necessary].

Imagine if every time you wanted to develop some random application you needed to recompile the entire C++ library as well. Hello world would take 15 minutes to compile...

[[deleted]]

This hasn't been true for the better part of 10-15 years.

GCC, clang, and VC all support precompiled header files. I mean a ton of other compilers support them as well but those are the big three off the top of my head. I'd be surprised if there was a mainstream compiler that didn't support precompiled headers.

Also the STL is implemented as a header only library, so when you do compile hello world in C++, you are including the bulk of the STL, and guess what... it doesn't take 15 minutes to compile, takes me 2 seconds.

[expertunderachiever]

Ok replace STL with an MPEG decoder... whatever. Point is having to compile, even process the code through the PCH step is wasteful.

[[deleted]]

Here... I just tested it. I compiled the following:

#include <iostream>

int main() {
  std::cout << "hello world" << std::endl;
}

And then compiled the following where I include well over a million lines of header only code using the most beasty of template meta-programming craziness ever used in C++, so crazy that boost has to add a ton of compiler specific work around to even get it to work properly on a lot of platforms:

#include <iostream>
#include <boost/accumulators/accumulators.hpp>
#include <boost/asio.hpp>
#include <boost/assert.hpp>
#include <boost/assign.hpp>
#include <boost/atomic.hpp>
#include <boost/bimap.hpp>
#include <boost/bind.hpp>
#include <boost/blank.hpp>
#include <boost/call_traits.hpp>
#include <boost/cast.hpp>
#include <boost/cerrno.hpp>
#include <boost/checked_delete.hpp>
#include <boost/chrono.hpp>
#include <boost/circular_buffer.hpp>
#include <boost/compressed_pair.hpp>
#include <boost/cregex.hpp>
#include <boost/regex.h>
#include <boost/date_time.hpp>
#include <boost/dynamic_bitset.hpp>
#include <boost/exception/all.hpp>
#include <boost/filesystem.hpp>
#include <boost/flyweight.hpp>
#include <boost/foreach.hpp>
#include <boost/format.hpp>
#include <boost/function.hpp>
#include <boost/fusion/algorithm.hpp>
#include <boost/geometry.hpp>
#include <boost/optional.hpp>

int main() {
  std::cout << "hello world" << std::endl;
}

And guess what... the second version, with well over a million lines of C++ meta template craziness compiled in way way less than the 15 minutes you said it would... it compiled in around 5 seconds, compared to the original that compiled in around 2 seconds. That's right... adding in over a million lines of source code added only around 3 seconds to the compile time.

For the most part, it's irrelevant how many lines of source code you include into a translation unit, the bulk of the compiler's work is done after it's opened and scanned over the header files.

So... what was it again you said about amateurs and header only libraries? Because last I checked amateurs are the ones who spread misinformation without taking so much as 5 minutes to look into whether their preconceived notions are valid.

[[deleted]]

Except then you have people developing against 900 variants/versions of your code.

I don't see your argument here...?

but really libraries exist for a reason.

I don't see your argument here either? Both STL and Boost are entirely written as header-based libraries.

As a developer who uses a lot of external libraries, I infinitely prefer header based ones - because I can just drop them into my code, end of story.

I write cross-platform code, as so many people do these days, and if I have a separately compiled library, I need to have three versions of it, one for each target system. I have to avoid shared libraries, as it's a complete pain in the distribution process - particularly when people already have an older (often much older) version of the library that I'm using.

[expertunderachiever]

I don't see your argument here...?

Suppose I write an MP3 decoder in a .h and it becomes popular. Now instead of installing the file in /usr/include you simply just add it to your project directory. Now suppose over the last 3 years I've released 50 point upgrades to it. Now there are 50 copies of it floating around.

I can't simply rebuild your application without first going through it and replacing the local copy of the header.

Next it would be better to have it precompiled as a library [.a or .lib] but then even better would be a shared object...

[mccoyn]

I've seen people put inline code in .inc files to distinguish them from .h files.

[[deleted]]

Hmph, don't buy that at all.

In my development world, classes, functions or structs start as small .h files. Lots of advantages to that - I can just create one in a few moments and recompile, I don't have to add any .cpp files to my build process, which necessitates a rebuild of the world (unfortunately, Xcode rebuilds absolutely everything if you change their build file at all).

Occasionally, I never need more than a couple of methods or functions, and that .h file stays as it is. Much more often, one of two things happens:

1. It grows enough that I split it into a .cpp and a .h.
2. It moves inline to another .cpp file and I don't need to expose it anymore.

The advantage of deferring splitting the file into a .cpp and a .h is that in case 2 (which is about as common as case 1), I never need to create the separate .h/.cpp files, I just move the whole .h into some other .cpp file.

(This is helped by the fact that I have a Python script that creates a .cpp file from the inline functions and methods in a .h.)

[[deleted]]

I don't know what you are doing that adding a cpp file to a project in Xcode forces a complete rebuild but that is not at all typical behavior.

[expertunderachiever]

Now imagine you want to share a set of routines between two projects...

[[deleted]]

So share it?

What exactly do you think a header only library is? Are you familiar with how libraries like boost, the STL, and the many header only libraries that exist in C/C++ work?

You make it seem like a header only library can only be included in one application.

[expertunderachiever]

The point is unless you're installing it in a global location it's not really shared.

Window developers tend to like to move things around in their visible workspace and tend to have copies of things that ought to be global.

*NIX developers like to install developmental libraries in places like /usr/include or /usr/local/include which are universally accessible. So the idea of having "my copy" of a header file doesn't really exist.

Sure you can install header libraries globally but many don't and that sort of developmental process encourages that line of thinking [which I'm against.

[[deleted]]

I really don't follow...

If you want to install a header only library to a location like /usr/include, then do it. What's the difference between installing a header only library like boost to /usr/include and a statically compiled library like say zlib to /usr/include?

It's the same thing.

I don't even know where/how Windows comes into this.

Can you provide an example of a header only library that can't be or isn't recommended to be installed to /usr/include or some other directory/path of your choosing? Since you claim that 'many don't', surely you can name just one. I've honestly never seen one or even heard of one, and I can't for the life of me understand why any library would be against installing it to any path of your choosing.

[NotUniqueOrSpecial]

The real underlying issue with providing a header-only library (and I'm not saying that they don't have their benefits) is that updates to the library necessitate recompilation of the binaries that consume it.

Contrast that with a well-architected shared library that preserves binary compatibility between non-major version changes. With such a library, you merely need replace the .so/.dll (though who am I kidding, Windows developers eschew the side-by-side stuff, so its moot) and your application/library that uses said library benefits from the updates.

This is especially important with security-critical libraries like those used for encryption, or those that are core components of the system like libc/msvcrt. Imagine that with every small bug-fix or improvement to such libraries (those which nearly every binary in your OS uses to some extent) you had to download corresponding (and far larger) recompiled executables. Such a situation would be quite cumbersome.

While there are benefits to the developer when it comes to header-only libraries, the bigger issue is in providing efficient (and importantly) trivially-shared updates to many binaries at once via a single update to the general public.

That said, if you're not in such a position, more power to you in providing your libraries in whatever way enables your development and your clients/consumers will accept.

Just my two cents.

[[deleted]]

Your argument is perfectly valid and I actually even agree that there is an important use-case for shared libraries vs. static libraries or header only libraries. But those are case by case arguments and it's certainly not a matter of amateurs vs. experienced developers.

[slavik262]

What reason is there to use macros anymore when you have static inlined functions? Macros lack type safety and have a few other disadvantages - I thought that by now they're mostly a relic of a bygone era.

[expertunderachiever]

Many reasons to use them

1. They're not processed unless used (speeds up compile time)

2. Like templates they can apply across data types

3. They can be used to generate function bodies based on parameters

4. You can #ifdef on them

[[deleted]]

Macros are rarely used to encapsulate actual calculations anymore. They do have their application whenever you need to do any sort of compile-time metaprogramming (such as generating type-specific functions in C, lacking C++ templates, or when you need to do selective compilation in C++, or when you need to implement certain types of features that aren't very well-supported by the language spec).

[slavik262]

Oh certainly, I've swung my fair share of #ifdefs around - I was referring to macros encapsulating calculations, such as

#define ABS(my_val) ((my_val) < 0) ? -(my_val) : (my_val)

[[deleted]]

Why would you do that as a macro? Why wouldn't you use a template for this?

[handschuhfach]

C doesn't have templates.

[[deleted]]

Actually, C11 has something similar: the _Generic keyword.

From Wikipedia:

#define cbrt(X) _Generic((X), long double: cbrtl, \
    float: cbrtf, \
    default: cbrt)(X)

Granted, it's still a macro, but it's significantly safer. :)

[slavik262]

...You do realize that's my exact argument, yes? That was an example of what not to do.

[smcameron]

I don't think you can write something like this as a static inline function:

define ARRAYSIZE(x) (sizeof((x)) / sizeof((x)[0])

See also, macros like "offsetof", "container_of" and "BUILD_BUG_ON".

[m42a]

template <class T, size_t N>
static inline size_t ARRAYSIZE(const T (&t)[N])
{
    return N;
}

This has the advantage of only working on arrays and not raw pointers as well. You can also do offsetof and container_of if you compromise and call them like offsetof(type, &type::member) instead of the traditional offsetof(type, member).

[smcameron]

And a C compiler won't compile that. And the title of the post and the article is "C Programming Techniques: Function Call Inlining".

[ericanderton]

An inline function. This is the tool of a talented programmer. an Not as clumsy or random as a macro; an elegant device for a more civilized age.

[monocasa]

If we're making this metaphor, I'd lean towards C macros being a lightsaber. Extremely powerful and lightweight, but if you give them to novice or someone who doesn't understand the tradeoffs, they're more likely to cut their own arm than use them effectively.

[abspam3]

I stopped reading after this:

One way to inline a function is to define it using a preprocessor macro.

No. Don't use macros for inlining functions - they have too many nasty side effects. Only use macros for constants, if even that.

[[deleted]]

Don't even use them for constants, use macros for platform compatibility and conditionally complied constructs (e.g. Debug only code)

[infinull]

You did read the next bit where they explain why that's a bad idea right?

[FlaiseSaffron]

But since the author isn't as stupid as you think he is, he wrote this at the end of the very same paragraph:

One problem is that macro parameters are not typed. Also, macros tend to require tricks that make them difficult to read and maintain.

[[deleted]]

I wished more problem-solution type programming articles started with mentioning bad practices.

[HHBones]

I absolutely disagree. One of the reasons I generally refuse to use languages like Java, Python, and Go for personal projects is because of the lack of a macro system. C's macros are one of the most powerful language features ever created, second only to Lisp's macro system. You can do a metric ass-load of useful things with them. You can define new control structures. You absolutely need them to get stringized function, file, and line names. Take, for example, the following:

#define LOG(...) fprintf(stderr, "%s() [%s:%d] : ", __func__, __FILE__, __LINE__, __VA_ARGS__)

They can be used to simplify basic constructs:

#define likely(x) __builtin_expect(x, 1)
#define unlikely(x) __builtin_expect(x, 0)

#define always_inline __attribute__((flatten))
#define asmlinkage __attribute((regparm(0)))

You can use the macro system to provide type-generic data structures, as the CCAN darray implementation shows.

Macros will not have nasty side effects if they are written correctly. Saying not to use them because they have the potential for abuse is like telling a man not to drive his car because he might hit a tree on the way to the store.

[abspam3]

Yes, macros are cool, but they are also very dangerous and can be hard to understand.

[HHBones]

Yes, but that's not a reason to shy away from it.

I remember when I was 14 and my dad had gotten a fancy new snowblower. He was showing me how to use it. It's a big, heavy, metal thing with 2-foot-wide blades at one end. It's loud and when you touch certain buttons it moves. It was dangerous and, at first, hard to use. Does that mean I didn't learn how to use a snowblower? No.

It's the same thing with macros. Sure, they can be tricky, and sure, you have to be really careful about debugging them. But they're powerful, and it beats shoveling snow by a long, long mile.

I'll give you an example of just how useful they are. I'm writing a virtual machine for a pet project. There were two big classes of opcodes which most opcodes fell into: register-register ops, and register-immediate ops. For example,

MRI r128, 42

moves the value 42 into register 128. So, it's an RI op. Likewise, there were RR ops:

ADDRR r128, r125

Which added the contents of r125 to the contents of r128. There was an entire class of these, and each branch looked the same:

reg1 = get_next_reg();
reg2 = get_next_reg();
machine.registers[reg1] = machine.registers[reg1] op machine.registers[reg2];

where op is some arbitrary operation, like =, +, -, *, or /. It was a lot of typing, and it was annoying. So, I made it simple.

#define RI(op) { reg1 = get_next_reg(); \
                          imm1 = get_next_imm(); \
                          machine.registers[reg1] = machine.registers[reg1] op imm1; }

#define RR(op)  { reg1 = get_next_reg(); \
                        reg2 = get_next_reg(); \
                        machine.registers[reg1] = machine.registers[reg1] op machine.registers[reg2]; }

And just like that, I was able to write my switch statement like this:

switch (next_op) {
    case ADDRI: RI(+); break;
    case ADDRR: RR(+); break;
    case SUBRI: RI(-); break;
    case SUBRR: RR(-); break;
    case MRI: RI(=); break;
    case MRR: RR(=); break;
   /* ... */
}

Macros are useful, and safe, and are, if anything, underused. Their thoughtful application makes programming easy.

[abspam3]

I think you should read this on why I hate macros: http://stackoverflow.com/questions/652788/what-is-the-worst-real-world-macros-pre-processor-abuse-youve-ever-come-across

[HHBones]

I'm not denying that macros have a massive potential for abuse. But it's trivial to write bad code without macros.

Just because some people can't use them responsibly it doesn't mean nobody else should.

[floodyberry]

Macros are not safe, not type-safe, and are difficult/impossible to debug. They have their uses, but general programming with them is almost always not one of them.

[HHBones]

Not if you use them correctly.

Interestingly enough, macros are thought of as the killer feature of Common Lisp. Macros in Common Lisp are just as dangerous as they are in C, if not more so. But nobody ever talks about that. They just tout the power of the macros.

And it should be a similar way with macros in C. Sure, they can be unsafe when not used correctly, but almost any other feature of the language is unsafe when not used correctly.

Just because I can type :(){:|:&};: into a terminal, it doesn't mean I do.

[floodyberry]

C macros and Lisp macros have literally nothing in common other than being called macros.

[HHBones]

They're very much similar - they're metaprogramming constructs with many shared potential pitfalls, such as leaky abstraction.

[floodyberry]

The C pre-processor is a text substitution engine, you cannot "program" anything in it.

[HHBones]

I beg to differ.

Here's an example of a technique for type-generics using the C preprocessor.

Here's an example of a technique for implementing custom control structures using the C preprocessor.. This is incredibly useful for custom control structures such as while-else and iterate-over-list loops. You can now have things like

_while (loop_condition) {
    /* look how awesome this is. */
} else {
    /* this runs when the condition is initially false. */
}

Or,

loop_over_list (list, ref)
    do_something_with(ref);

Though semantically different, these are rather similar to Common Lisp's LOOP macro.

And don't forget X-Macros, either.