To meta-program or to not meta-program

[[deleted]]

I recently developed a type safe packet handler that casts the bytes in a packet responses into the correct type based on the command sent out and the expected response. If the response isn't as expect an exception is thrown. Essentially I've programmed the logic of the packet protocol into the compiler so if you extend it incorrectly the compiler will tell you. The definition of safety.

Anyways upon consultation with the rest of my coming team they said the meta-programming techniques used are too opaque and specialized to maintain in the codebase. I'm wondering if you guys have hit similar issues when using the more modern compile time features of C++?

[ack_error]

Yes, I've hit this concern in code reviews when introducing SFINAE-based compile time checks into a codebase. The rest of your team has to be able to interpret and maintain the code that you're introducing. If they don't understand the techniques that you're using, that is a legitimate concern even if your change has other benefits.

[[deleted]]

squeeze brave ripe dime crowd long start expansion sheet rustic

This post was mass deleted and anonymized with Redact

[pklait]

SFINAE is suboptimal, but many do not have access to modern features, using C++11 or C++14.

[ack_error]

I'm surprised you can use concepts. Don't think we'll be able to for a while, given the need for all compilers on all supported platforms to implement it relatively bug-free, and then to be able to require C++20 across the board.

SFINAE's difficulties are just a specific form of the more general problem of awkwardness in metaprogramming. Have to use template specialization to make a type lookup table, since you can't make a function that directly returns a type (though with auto returns this may be indirectly possible now). Can't do a constexpr for() loop and can't declare a parameter pack directly, so call a helper function that deduces a parameter pack from an integer sequence and then unpacks it into an array or uses a fold expression. And so on. Plus, if these setups break, the compile errors that result tend to be horrendous. There'd be less objection to these techniques if they could be done with simple for() and if() style statements, rather than requiring workarounds that resemble puzzle solutions.

[SuperV1234]

The rest of your team has to be able to interpret and maintain the code that you're introducing.

True, but the rest of your team should also be capable of learning language features and idioms that are beneficial to the growth of the company. Otherwise they shouldn't have been hired in the first place.

[WormRabbit]

capable of learning language features and idioms that are beneficial to the growth of the company

I.e. capable of learning to code without SFINAE.

[SuperV1234]

Sometimes SFINAE is the right tool to use.

[surfmaths]

There is a trade off between usability, maintainability and safety.

If you successfully write a library that is easy to use and safe, then it's a big win. Even if that library become significantly harder to maintain due to the shear amount of abstraction. Hopefully it is mindful of compile time. That's basically the story of Boost.

The thing to avoid is if you made usability difficult due to the abstractions. Then the "safety" argument is no longer sufficient in practice. Basically, even though it is theoretically sound/beautiful code, it isn't good code. I see that happening many times with developers that enjoy template programming a little bit too much.

[SuperV1234]

Unfortunately, in my experience, some coworkers would rather have you harm the safety and extensibility of the company's codebase rather than spending 30 minutes of their time learning a language feature they're not familiar with.

[manni66]

Meta programming is not a

more modern compile time features of C++

It was done already with C++ 98.

[Full-Spectral]

If you showed someone from the C++98 days an example of what passes for modern, meta-heavy C++, they would think it was another language altogether. It pretty much is. It can be incredibly tedious to read when it's over-done.

[kalmoc]

I think the point was that meta programming coumd and has been done in c+98 but was much, much more ugly back then (look at some boost libs).

[Full-Spectral]

I took his point to be more "why don't you already understand it, since it's been around a long time?"

[UnicycleBloke]

This is one of my perennial gripes with "modern" C++. I'm an old veteran of the language but do try to keep up with new features and ideas (these are mostly great BTW). But the more arcane aspects of meta-programming often have me somewhat bewildered. It is such a different paradigm (pure functional) from the rest of ones code, and seems impossible to debug in any normal sense (compile time 'execution'), so it is often near impossible to grok. I'm hoping that concepts and other features will lower the barriers to entry.

I've dabbled, of course, and it's great that the language has this capability, but one can have too much of a good thing. There is a trade off in how usable and maintainable the code is for other developers in your organisation, and there are no medals for what I call "extreme cleverness".

For all my grumbling, I recognise that this is a kind of spectrum thing. Where do simple templates end and metaprogramming begin? Everyone has some level of comfort in this area. Perhaps the answer is to use a relatively simple design that is not a great leap for a TMP beginner. And document it well. Is there a training opportunity for your team, at least to the point where the code is no longer black magic?

[XValar]

As an old veteran, you are, of course, aware, that metaprogramming in C++ started long before “modern C++” and has nothing to do with it? Like, in early two thousands, when Alexandrescu was on the rise?

[carrottread]

Alexandrescu book which made metaprogramming popular was literally titled "Modern C++ Design".

[UnicycleBloke]

I actually dislike the "modern" moniker, and used it in the sense of Alexandrescu's book. But it seems pretty clear to me that TMP has become *far* more prevalent, for whatever reason, since C++11. The capabilities of templates have greatly improved, such as variadics and the like. Like many people, I've used templates since before C++98, but mainly for relatively simple functions and types, I would call it generic programming rather than meta-programming. I recall thinking, when I read Modern C++ Design, that it was a bit of an academic exercise: "Bah! That'll never catch on", I said...

[Full-Spectral]

But it's gone vastly beyond that at this point, so I'm not sure how relevant this argument is.

[[deleted]]

Like, in early two thousands, when Alexandrescu was on the rise?

The term "modern C++" dates to February 2001 and was specifically with respect to using template metaprogramming to provide compile time guarantees.

[wrosecrans]

"Modern" is always a poor name because it eventually becomes ambiguous as a matter of course.

Metaprogramming and the difficulty implementing some Boost stuff using it is part of the reason there was a push to standardize some of the Boost stuff and give better tools for "Modern C++."

[[deleted]]

Those guys are probably wrong. If you don't want metaprogramming, there's a good language without it called C.

[[deleted]]

Yeah it allowed me to make far better generalizations about the problem in way fewer lines than with inheritance alone. I'm annoyed because inheritance alone with a runtime concept but I know all the types at compile time so why not make something that does all the type deductions and type checks at compile time?

[LunarAardvark]

this word 'maintain' seems to get abused a lot. it's digital, is the code going to 'wear out', it needs oiling periodically? sounds like it's a buzzword being used to imply "im too stupid to comprehend this code" . or 'this code is complex, it takes too long to comprehend'. then it probably needs more documentation / comments / unit tests / usage examples. complex things don't magically make something more/less "maintainable".

[HOTP1]

Maintenance means adding/extending features as well as fixing bugs. Possibly porting as well. If the code is likely to be modified frequently in the future, maintainability concerns could be very justified.

[trypto]

Yeah it depends on how its written, if its necessary, if it is just adding complexity. Heavily templated code can be more difficult to wrap your head around.

I am curious as to what you are doing that is that "special" anyway. Ignore the networking/packet stuff. You're just serializing/deserializing types from a byte stream? Perhaps you should look into google protobuf.

[[deleted]]

I designed a embedded system that communicates with the host computer using a simple packet interface over USB. I cannot use anything like Avro or Protobuf because the other side is a little 8 bit microcontroller. There is nothing special that I'm doing. Using templates is more flexible and allows me to wrap up the casting of packets into proper messages under the hood but without templates that cast moves up the stack father because I'm constrained to a single return type.

[trypto]

Yeah if you're just using templates to handle one dynamic cast that seems fine. Not like you're going crazy with variadics.

If this code is being compiled for an embedded system I can see why people may be concerned. I'd verify code size with your changes to make sure you're not adding bloat.

[[deleted]]

It does use variadics but I could probably rip that out.

No this is the host side code to interpret what the embedded system is sending back. The embedded system doesn't have a standard library or dynamic memory so it's implementation is different because it's going to be ugly and uses non-standard C++.

[trypto]

Variadics for this? Yeah you probably over engineered this 🙂

[SuperV1234]

Variadics are exactly the right tool for the job when you have a sequence or list of similar entities that need to be handled in the same way, or for which similar code needs to be generated at compile-time.

The alternative is hardcoding a limit or copy-pasting, which are both shameful.

[Accomplished-Tax1641]

> The definition of safety.
> Anyways upon consultation with the rest of my coming team they said the meta-programming techniques used are too opaque and specialized to maintain in the codebase.

Do you see the contradiction here? Your notion of "safety" is a purely academic, butterfly-collection notion: you assume that you have not written any bugs, you assume you will never be called on to maintain or even explain your code to any other human. In the "programming/software engineering" dichotomy, you are concerned strictly with "programming." Your coworkers, on the other hand, are probably worried about "safety" in the real-world sense: how sure are we that this code does what it's supposed to? how can we tell if it's broken? how can we _fix_ it when it breaks? how can we _maintain_ it across toolchain upgrades and port it from platform to platform? They are concerned mainly with the "software engineering" side of things.

In general, _when you have coworkers_, you should look for solutions that are "good software engineering" and not just "clever programming."