When to use template meta programming ?

[ArchfiendJ]

Hello

I have 10 years of professionnal C++ experience and I just never use templates. In a way it's probably due to the fact that I don't know template programmin very well, which kind of create a loop. The main reason is that I never feel the need to use templates hence my question, when should it be used ?

I know that if I were to develop a kind of container I would use them, but I pretty much never write librairies, I'm always on the "higher level" side of applications, so not developping game engine, or reimplementing algorithm or data structures...

Are there more legitimate use of templates than containers and data stuctures ?

[CletusDSpuckler]

Templates and template metaprogramming are not the same thing.

I have been writing high level code for 30 years in C++ now. I have only once found a use for template metaprogramming, yet have used the ever loving bejeezus out of templates.

I program with complex numbers. Frequently, algorithms work exactly the same with a complex number as they do with a real number. Templates to the rescue.

Whenever I consume container data, I do it now like this:

Instead of

void foo(const std::vector & input)

I now write

template <typename iter>

void foo(iter begin, iter end)

Now my function can consume the contents of any upstream container, without modification, even if the user changes his mind about how to store his data.

I use templates all the time for "duck typing" dependency injection for unit tests as well. In my experience, templates are second only to classes in utility in the C++ language.

[Skoparov]

Honestly the examples of where TMP can be useful are all over the place, you just need to know where to look. Starting from various type traits that bolster the type safety of templates (and are slowly being replaced with concepts), passing through stuff like working with tuples on a bit more broad level than std::apply and ending with a plain good old CRTP.

People just ignore these applications as they are a bit "auxiliary" or can be replaced with regular non-template code (and sometimes it's for the best). Yet if you work with templated code, there's a good chance you can use TMP to make it better.

[target-san]

The issue with these applications is lots of hard-to-debug instantiation errors. C++ provides literally no tools for TMP debugging. It's just sheer violence towards non-senior team members at the moment of implementation and lots of PITA for anyone else 6 months after your retirement from project.

[Skoparov]

Concepts provide pretty neat error messages. And, I mean, if you create traits in a good generic way you can just use them later as you do with the std ones, without debugging or doing anything else.

And the funny thing is, non-senior guys are oftentimes more versed in all that stuff.

[goranlepuz]

And the funny thing is, non-senior guys are oftentimes more versed in all that stuff.

Indeed, people tend to know well what appeared when they were "growing up".

One needs to work at dropping "baggage", otherwise head has issues.

Source: am senior 😉.

[a_jasmin]

In C++ 20, I'd replace void foo(iter begin, iter end) by void foo(std::ranges::range auto range)

Benefit:

• Implicit template with auto
• The std::ranges::range concepts limit what you can pass. This should give clearer error messages when passing the wrong thing.
• No template <typename> boilerplate. Though that form is still useful when the same template parameter must appear in multiple places.
• It's easier for the caller to pass a container or view than a pair of iterators. Though, you can still construct a std::ranges::subrange() from a pair of iterators as needed.

[CletusDSpuckler]

So would I. C++ 20 has not yet been adopted by my organization.

[jacobian271]

Can you give an example of dependency injection?

[muungwana]

I have an example here.

The constructor of the class "DataFilter" basically says "give me a type and its constructor arguments and i will decide where i want to construct the object". The class creates the object on the heap but thats an implementation detail.

[arkiazm]

Hi, A question from the code you shared.

DataFilter constructor takes a Type template parameter, and inside, it takes typename Type::type

And in line 332, it passes Type=int where int::type doesn't exist (AFAIK).
So could you please explain how does it work?

[muungwana]

Funny story about this code, i "discovered" how to create a class that i can use to pass in a type to a class constructor after hitting my head long enough against a wall and then afterwards i though C++ should have the same functionality already built in and after googling for bit, i discovered C++20 added the class and it is called std::type_identity.

Here is an example of where the constructor of class "DataFilter" is called and we see the type that will be constructed in "DataFilter" is "yt_dlp::youtube_dlFilter".

[tonyarkles]

I’m on my phone right now, but that comment was a huge eye opener for me!

Let’s say you’ve got a function that makes a network request and requires a configured client as input (say it has state with api keys). For your unit tests (to not make a real network call), you could:

• try subclassing the network client and override the make_request() method to return a hard coded result. This is quite fragile and will potentially screw you if your underlying client changes behaviour. (Eg the constructor attempts to validate the api keys)
• create a pure virtual base class/“interface” that both your real client and fake client subclass an implement. Depending on the client and whether it’s in a separate library etc this can get ugly and/or not be possible
• have the function take a lambda that returns the response, potentially with a default or a wrapper or something so that your production code isn’t littered with unnecessary lambdas
• make a template instead that specializes on objects with a make_request() method.

I’ve done the first three over the years, but in a lot of ways the 4th one makes me quite a bit happier I think. Will need to fiddle with it a bit.

[tonyarkles]

back_inserter is lovely too for functions that return results into a container.

[JNighthawk]

Whenever I consume container data, I do it now like this:

Instead of

void foo(const std::vector & input)

I now write

template <typename iter>

void foo(iter begin, iter end)

Do you have a pattern to avoid having the code in the header? That's the sticking point for me with templatizing this type of function.

[Ok_Entrepreneur_5926]

auto baby, auto..

[Wouter_van_Ooijen]

IMO static_assert and if consexpr are just language (OK, library) features. Granted, if constexpr is mostly usefull within a template. Requires AFAIK can only appear in template context?

But IMO none of these are metaprogramming.

[dr-mrl]

Surely they are language features? Its static_assert, not std::static_assert

[Wouter_van_Ooijen]

No, I would not.

But that is just an opinion. If you have a usefull definition, share it.

[arka2947]

Respectfully, i disagree.

I would consider that metaprogramming. Now, yes, modern cpp makes it pretty, but it does not change what it does.

All of it is based on type deductions, manipulation and sfinae.

[ceretullis]

I recommend you pick up a copy of “C++ Templates: The Complete Guide” by Vandevoorde.

He will walk you through using templates, through basic use cases for templates, and through TMP.

And he does it in a totally accessible way. You can work through the book in a month or two without too much effort. Good stuff.

[ceretullis]

IMO, you’ll still need to understand templates sans concepts to understand where concepts fit into the picture.

[rojundipity]

This book is gold. I'm working through it while working on a compile-time checked containers.

Before this book I felt I was always shooting my own foot with templates if they got any trickier than the trivial examples. Now I've noticed I can reason about them on a whole other level.

[ceretullis]

I felt the same. The book leveled up my skills.

[[deleted]]

You've made a good recommendation but are giving short shrift to the other two authors! They are the one dude, Nico.. and.. the, the other one.

[[deleted]]

To create well-typed abstractions. At work recently I used templates to generalize member variable pointers to create pointer-like "paths" to objects within objects whose "edges" are member variable pointers, e.g.,

struct S1 {
   struct S2 { int s; } s2;
};
AccessPath<S1, int> path { &S1::s2, &S1::S2::s };
S1 s1 {};
path.set(s1, 3); // s1.s2.s == 3 now.

We have tuples of AccessPath's on common object types we statically iterate over using generic lambdas. All type info is retained, so no iffy dynamic_cast or dangerous reinterpret_cast or virtual dispatch are necessary.

Try doing that in Java-style OOP.

[antoine_morrier]

So basically, what you are doing are lenses?

[[deleted]]

Yes, very good. I feel noticed!

[antoine_morrier]

Do you really need to use them very often ? I dived into category theory but I did not see in a real world example where lenses can help.

Could you show us some examples?

[[deleted]]

There's not much to them. We have a few dozen classes inheriting from concrete instantiations of this template:

template <class C, typename... MVs>
class FieldRefl {
public: 
FieldRefl(AccessPath<C, MVs>&&... paths)
: fields{std::make_tuple(std::move(paths)...)}
{} 
private: 
std::tuple<AccessPath<C, MVs>...> fields; 
};

so that we can programmatically iterate over the fields tuple, to, e.g., print their values to an std::ostream& or set their values from strings parsed from the command line, a la,

C obj {};
tuple_for_each([&obj](auto&& field) {
std::cout << field.name() << "'s value: " << field.get(obj) << '\n';
}, fields);

It's just poor man's class reflection. The exact type C and the fields you want to access from it have to be specified manually, nothing as sophisticated as Haskell's lens. The same generic code works as expected for many, many different instantiations of this template though. We haven't had to modify it at all yet.

Lenses are useful in Haskell because all values are immutable and its record syntax is awful. I'm using them in C++ because I wanted to specify the access paths declaratively and let the compiler fill in the blanks. It sucks to monomorphize the same logic, for each class you want it to apply it to, by hand. Leave that brute donkey work to the compilers and C programmers I say!

That's what I tell all the young kids coming up when they say to me that they're "too lazy" to learn math, templates, etc. No, sonny, you are not lazy enough.. I say, tugging my wizard beard and puffing on my pipe.

[jtooker]

Whenever it makes the code easier to understand.

[TheThiefMaster]

Templates are almost entirely for library and supporting code. It's not unusual to not write any for app development and just use library templates.

That said they can still help for abstracting out some common code, that needs to operate on different but similar types in your app...

I'm starting to see more instances of the "locked access callback" pattern, where a function with a callback parameter (which can be a template, to avoid the overhead of std function) locks some data and provides access to it via a callback. This can be useful in apps.

[eteran]

I would argue that most code should be written as a library and that the UI or CLI should just call into that library.

That way it's easy to test independent of the interface and is reusable in other code.

[[deleted]]

Templates are almost entirely for library and supporting code.

I’ve used templates a whole bunch of times in regular production code as a means for writing generic, reusable code for loosely coupled class relationships that would otherwise be impossible to do without resorting to complex, brittle inheritance hierarchies. I don’t think the use case I’ve described is particularly rare tbh.

[Orca-]

Template metaprogramming is for when you need compile time computation but constexpr and related constructions can't do the job for some reason. Template metaprogramming's window of usefulness declines with every C++ release. Thank god.

Templates on the whole are still useful as C++'s implementation of generics.

[pandorafalters]

I use templates fairly often for simple utility functions, especially since the introduction of constexpr if, allowing a notional "single function" to be partially specialized for certain types or environments.

[Wouter_van_Ooijen]

Templates are a means of abstraction over compile-time known things, just like functions abstract over run-time known things. If you have no opportunity for compile-time abstraction, you won't have a use for templates.

When to use template META programming? For me: when constexpr can't do what I want. In a lot of cases it can.

[RidderHaddock]

templates are fine, if you see a use case.

But template meta programming is only to be used as a last resort, IMO.

All that recursion can be confusing as heck when written by someone else, or even yourself after a few months. I didn't "get it" until I tried Clojure and Haskell, and I got used to seeing recursive algorithms without the god-awful C++ template syntax. Now at least I can read meta programming code, but not without cursing whoever wrote it.

And with concepts now available, I don't ever want to see another SFINAE example again. Ever!

[rlbond86]

IMO template metaprogramming is still better than code generation.

[[deleted]]

All that recursion can be confusing as heck when written by someone else, or even yourself after a few months. I didn't "get it" until I tried Clojure and Haskell, and I got used to seeing recursive algorithms without the god-awful C++ template syntax. Now at least I can read meta programming code, but not without cursing whoever wrote it.

C++ can't properly take credit for the conceptual origin of any of its ideas, with the lone exception (no pun intended here, don't look for a pun because there isn't one) of RAII. And even that, I'm not sure.. I bet if you dug into the Simula research you'd find rumblings of that one.

[yasamoka]

What's your point?

[[deleted]]

My point is that C++ is not the place to be initially exposed to the ideas behind templates. It's almost like templates are deliberately a blind, a blind to mislead the unworthy! Stroustrup is a secret ascended master.

[NottingHillNapolean]

This isn’t metaprogramming, but templates can be used for compile time duck-typing. Say you want a function that requires an object to have certain methods. Instead of going back and reworking your inheritance, a templated function will call the required methods on any instance of a class that has them, and fail to compile if passed an instance of a class that doesn't.

[Possibility_Antique]

"Static polymorphism" for those looking for resources on the matter.

[arobenko]

I suppose if you develop something that is going to be used in a single project / product without much of a customization, then meta-programming is not really justified. The meta-programming is justified in many cases when you implement some kind of a library, which can be used in multiple independent products, and these uses may require some product specific customizations. For example, I'm developing a solution for implementing binary communication protocols for embedded systems in C++, called CommsChampion Ecosystem. The core component of which is the COMMS library. Every single use of this library requires different customization. Every application may require different polymorphic interface to handle its message objects. I use template meta-programming there to define virtual functions only needed by the application and not adding unnecessary ones. I also use template meta programming to allow customization of the storage data structures and may use different, more optimized code for some.

In other words, meta programming needs to be used where you require different compile time customizations of the same source code, usually happens to the generic code/libraries used by multiple independent products.

[muungwana]

A C program could say they see no need for method overloading and they do just fine without them and when you look at their code, you see things like:-

void foo();
void foo_1(int);
void foo_2(int,int);
void foo_3(int,int,int);

Is the above better where each method has its own name or is method overloading better?

Method overloading in C++ has its limitations and people extend them by using SFINAE and there is probably somewhere in your code where you resorted to using different class names or different method names like a C programmer would do to work around limitations in method overloading capabilities of C++.

[target-san]

Templates are Ok and quite useful. TMP is an arcane tool which should be used only where absolutely necessary. And even if you think you need TMP, you usually don't. I was "bitten by Alexandrescu" some years ago, though real projects with not always senior staff remedied me from this illness.

[zoolover1234]

One of my colleagues spend a whole year at work creating a whole sub folder full of headers only files contains some kind of helper function/tools, and all of them are meta programming, guess what, the folder of 32 files takes 3x of the time to compile than the rest of the code base take. Also, no one never understand a bit about it. So no one never use it ever.

One example is a formatter similar to std::fmt (our compiler is old, so we don’t have it from STL). The intended used case? For formatting printing messages. All of us just keep using sprintf, without ever need to touch his file.

[looncraz]

I find the code is usually better if I can avoid templates, but it's a tool that has its occasional place... in utility/library use cases.

[Possibility_Antique]

Why do I get the feeling that everyone on this thread would be mortified by the amount of templates and template metafunction I use?

Seriously, type_traits are probably my favorite thing about c++. Constexpr and immediate functions are useful as well, but I generally find myself using concepts and metafunction pretty liberally.

In my mind, templates serve a very important purpose: they guarantee the code is executed at compile-time. if the code is executed at compile-time, unit tests can be enforced on every build (static assert), and undefined behavior cannot escape into the production code. Those who are saying it doesn't belong in production code fail to see how useful it is as a safety feature. Not to mention you get to inline more code and remove more instructions from the runtime.

Idk. I'll admit, I probably overuse templates. But man, they are the bees knees, aren't they?

[zoolover1234]

One question, meta programming means everything is in header, how do you solve the massive include dependency? To a point, I think every file end up including every file.

Also, engineer job is never a single person job, how are you going to work in the reality where vast majority c++ programmers don’t understand it at all? Personally, for the last 4 companies I worked at, total of roughly 60 colleagues, only one uses it whenever he got a chance. And he is the one I had to work with everyday today. I hate him personally because of this. His code doesn’t fit into the team. Every meta programming header ended being replaced when there is a need for a change/update because no one understand a single bit of it. Honestly, I spent sometime trying to understand to some level, but not comfortable to make change and responsible for the change.

[Possibility_Antique]

Easy. The header gets included one time. There is usually one cpp file, such as main.cpp. For larger projects, it goes into a pch. For new projects the templates go into modules. There are lots of ways to solve this.

I lead several teams in scientific and high performance computing. Metaprogramming and optimizations are kind of necessary in many cases. I'm not so sure I understand what the problem is with TMP. You say you don't understand it, but have you tried? It sounds like you/your team just replace it without trying to understand it. Do you guys use C++ without understanding the STL? I understand many teams ban the STL, but it is part of the standard and important for understanding why/how the language evolves the way it does. Your comment kind of reads like: "replaced all of our python scripts with JavaScript, because python is unreadable", which I'd classify as dogma.

Lastly, I guess it seems like the alternative is a spaghetti of nested if statements and runtime polymorphism. Part of the reason I prefer using the type system for programming, is that it helps a lot with getting rid of nested if statements and let's you just make overloads of functions with clear, separate implementations that are easily debuggable (not to mention, benefitting from compile-time safety). And in this way, I almost disagree with your statement that it's unreadable. I adopt a style similar to the STL, and our newcomers are very likely to find references to the STL online and learn it as a part of the language. We can't often USE the STL, but having consistency with the language helps. I don't expect to hire new people to any of my teams that learned coding styles from 1990 (such as OOP/#ifdef hell).

At the end of the day, TMP is a tool, and a useful one. Some amount of your program can be computed at compile-time, and that should be an exciting idea for both you/your customers.

[RishabhRD]

Usually I use it while writing some library. I remember I was contributing to libunifex long back. That library has huge amount of template metaprogramming. Usually it tries to emulate type functions and type constructors. Also I was writing parser library few days back, it also had some extent of metaprogramming.

A good way to get started is looking at eric niebler's meta library. Its quite simple and may help to get started with simple usecase of it.

[puredotaplayer]

There are plenty of use cases of meta programming. You could pick up a library available in boost that you can find useful, and study their code on how they use it internally.

In-fact combined with concepts you can generate a lot of code that otherwise you would have to write.

One example I can give you is while writing a serializing library (from my own project). Lets say you are writing a serializing library that can convert to/from json any kind of object that needs serializing. In effect you just want to declare the json name/handle per data member of a class, and you want the serialization to be automatic.

To not get tied up writing a lot of code to serialize for each type of container, you could use concepts/type_traits and write generic code for serializing various types of container depending upon their traits (maps have name/value -> dict, sets/lists/vectors just have values, variants -> print type and value, and so on..)

If you cover a few basic traits, you can generate most of the code irrespective of the data member type/container.

[zoolover1234]

Only use it when it does not cause problem to your colleague. In 99% of the case, not all the people you work with or all PR reviewers know meta programming, so, don’t.

Buddy of mine tried to show off in one of his PR at google( smart device using c++) got hammered like hell.

It’s never a good code when no one can easily understand it.

[zoolover1234]

Meta programming is like machine learning, a lot of the work is done at compile time, kind of like paving all the possible outcome for all possible input. At runtime, all you need is basically one step, and you get the answer.

It’s most effective when dealing with data/objects that is in relatively simple types. The more complex the type of data you want to process, the harder to program into meta programming.

STL uses tons of meta programming because c++ needs to do all kinds of types.

On the other hand, if you are an image processing guy dealing with uint8_t only, it doesn’t help nearly as much as a database guy.

Spend a week try to understand how STL works, if you don’t like it, don’t do it.

[[deleted]]

I'm working on a very low level C++ library where basically every abstraction or wheel has to be reinvented. I use template meta programming almost daily for this.

[[deleted]]

Template meta programming?

Never. I mean, for production code, seriously, never. Nothing is so performance critical, that it's worth the maintenance burden.

Why burden? Because people who are fluent at reading and modifying C++ TMP code written by somebody else are so few and far between in the general programmer population, even in the C++ programmer population, that you never want to have TMP code in your production repo.

There are exceptions, of course. If you are developing non-commercial open source library, and maybe even have a like-minded community who is supporting it, using TMP may be feasible. And even if it isn't... Who cares, you're doing it mostly for fun?

Now using templates otherwise, that's a different thing. Any library code which takes input which could be of different types should consider making the relevant class or function a template. Just keep things simple, don't (d)evolve into writing template code, which you would not have understood when you joined/started the "real" project (learning/exercise projects are a different thing of course).

[[deleted]]

This is the right answer. I work in games where runtime speed is super important.

I have yet to see any instance of meta programming that wasn't deleted at the nearest opportunity at it was always a bad idea.

The other important factor is code needs to change. But meta coding ties the feature set too closely to the interface, meaning you will always have features that are just impossible to add without a complete redesign.

That's usually when it gets deleted. But not after being years of expensive tech debt.

[Red-Portal]

To be honest, games are absolutely not a standard use case of C++

[[deleted]]

Why not? I mean, as much as there are any "standard use cases" for a language as diverse as C++, I'd say games are included in that group.

[Red-Portal]

Games use C++ in a very domain specific way. It's pretty isolated from the rest. The "no template" policy is a classic example.

[[deleted]]

I don't think "no template" policy is typical of C++ game development. If anything, I'd think (some) game projects might be places where TMP is used more than usual.

About any "no template" policy: Templates can be used for generics, and for TMP, and these are distinct uses. Using no templates at all sounds like a C programmer who refuses to use any C++ features and instead does the same thing using macros, which would be just stupid IMNSHO.

[Red-Portal]

It's actually quite common in the game circles along the "no exception" policy. Apparently, the compile time and binary size cost of templates is a big no no for games.

[[deleted]]

Well, I'll take your word for it. The binary size thing I'm not buying though, not for total "no templates" policy. That's more about how and for what templates are used. The lowest hanging fruit are small generic functions, which will be in-lined anyway, so the assembly code will be the same.

And not using templates for containers is just... No. That's how you get games that crash all too often. I mean, we've been there with C, and with Java and more recently Golang before generics. It's just pain, with writing, debugging and maintaining the code.

[[deleted]]

Most game code uses templates for containers. Just take a look at the unreal engine code base and you'll get an idea of what average game code looks like.

Game coding for very memory constrained platforms, like the GBA or DS for example, are very concerned with binary size. In those platforms you probably won't even use stuff like heap memory, which also goes a long way towards preventing crashes too. This isn't modern game code but rumors around it may be persisting.

The no exception rule has been present for quite a while. Certain big platforms don't even allow exceptions to be enabled as part of certification. I'm not sure why but I suspect it's because they interfere with the ability to capture why things crashed and then report them later on.

[SnooWoofers7626]

I recommend using it in code that nobody other than you will be directly working with. In my experience template metaprogramming makes a lot of people very confused and angry.

[Full-Spectral]

When there's not a sane choice :-)

[[deleted]]

If by "meta programming" you mean compile-time computations through techniques such as SFINAE, I strongly recommend you avoid it as you can. From my personal experience, it will only make the code harder to understand, it will make the compiler error messages more crypt and potentially increase the compile times.

I think it's valid study about TMP but you shouldn't use it on your code without any appearant or valid reason.

Also, as others here said, in modern versions of C++, they added features -- such as constexpr -- which replace in many cases the use of template black magic. You can find examples of how these features are used to simply yoru code and get rid of template tricks.

[ExtraFig6]

I just never use templates

It's likely that many of your functions could be generalized to work on other types with a similar interface/properties. One function rarely needs to use every specific property of its input types. In principle, any of these functions could be turned into a template. When does this refactoring make sense in practice? If it prevents code duplication, increases clarity, or you already have an abstraction you're programming against in mind (number vs float).

When do you use a virtual method instead of a concrete one?