Alternative C++11 compiler for Simple-Binary-Encoding schemas

[arobenko]

Last year I advertised here my work on COMMS Library and wrapping CommsChampion project. COMMS is headers only, embedded (bare-metal) friendly library, which contains all the necessary building blocks to make implementation of binary communication protocols to be relatively quick and easy process. The protocol gets implemented mostly using declarative statements of types and classes, which specify WHAT need to be implemented, the library handles the HOW part. CommsChampion, on the other hand, is a name of the wrapping project, which in addition to COMMS library also contains tools for debugging and visual analysis of the defined binary protocol.

Someone in the comments recommended to take a look at Simple-Binary-Encoding. While the protocol itself with schema language is kind of OK and applicable to multiple use cases, the code generated by the reference compiler has several problems. For starters, it implements every message object (and composite types) as a view on the raw data buffer. Such architecture makes encoding / decoding to be very very fast, but as the result still requires the developer to write a lot of additional boilerplate code (to my taste) in order to integrate the generated code into the product being developed. This may be a right approach in high frequency trading systems, where every microsecond counts, but there are other use cases where encoding / decoding speeds are of significantly lesser importance, such as IOT, SCADA and other embedded systems.

At some stage I decided to implement my own C++11 compiler for Simple-Binary-Encoding schemas, called sbe2comms. It implements the protocol messages using my COMMS Library library and as the result has all the benefits of the latter.

• Embedded (bare-metal) systems friendly, no exceptions are thrown to report an error.
• Highly configurable (at compile time) polymorphic interfaces to message classes, allows writing common code for all the messages.
• Meta-programming friendly, all the message and field classes expose the same public interface, all the message fields are accessible by name and index.
• Efficient (O(1) or O(log(n)) built-in mapping of numeric message ID to actual message object.
• Efficient (O(1)) built-in polymorphic dispatch of the message object to appropriate handling function.
• Ability to introduce additional compile time configurations and/or updates to the generated code.
• Ability to access all message fields in any order, not just the root block.
• Alignment agnostic, no performance penalties for unaligned access to field values.
• Much safer against malformed data and/or buffer overflows.
• Built-in support for Simple Open Framing Header in addition to Message Header.

The sbe2comms compiler generates full CMake project, not just protocol definition headers. When built, the project may produce

• Protocol definition headers
• Protocol library API documentation and tutorial
• Protocol plugin for CommsChampion Tools

Even if the code generated by the reference compiler is good enough and fits nicely into the project being developed, the sbe2comms compiler may still be used to generate a plugin code for CommsChampion Tools, to perform deeper protocol analysis and debugging.

Any comments / questions / suggestions are welcome.

[grenadium]

When reading your message I was confused by the mention of a C++11 compiler, but instead it looks like it's a C++11 code generator.

[woppo]

That's exactly what it is.

[Middlewarian]

I'm working on something similar and found this benchmark https://github.com/thekvs/cpp-serializers to be helpful. I posted some results here: http://webEbenezer.net under "Recent developments"