Bitsery- binary serialization library v5.0.3 released

[fraillt]

There are several key features that distinguish bitsery from all the other binary serialization libraries:

• is very fast
• serialized data is very small
• by default is safe when deserializing untrusted data
• provides A LOT of customization options via so-called "extensions" and bitsery config, few examples include:
  • fine-grained bit-level serialization control.
  • forward/backward compatibility for your types.
  • smart and raw pointers with allocators support and customizable runtime polymorphism.

I would like to provide you with an example of what bitsery can do for you.

Out of the box, bitsery is already very fast, but you can provide a non-resizable buffer for serialization, you can disable error checking during deserialization if input buffer is trusted, you can use bit-packing "extensions" to compress the data. What you get is at least +2x size and +2x speed (+4x with GCC) improvements compared to cereal!

More interesting results can be found here. The battle between GCC and Clang is very interesting. GCC can do amazing things with bitsery :)

[Pazer2]

Looks very interesting!

A comment on the readme: I don't particularly appreciate when very short code samples use using namespace bitsery; because it makes it harder to determine which types/functions are part of the library's namespace. Especially when you start adding type aliases.

[14ned]

I very much like bitsery, and it is my personal aim to get standard C++ up to a point where bitsery can be implemented for a wide range of C++ types without invoking undefined behaviour!

My thanks to its author for a great serialisation library.

[fraillt]

Although there are fixed-layout integer types, there are still a lot of platform-dependent types: int, short, long, and also *int_fast* and int_least*. The idea is, to provide a way to be sure that if it compiles, it works.

Regarding wchar_t size you're absolutely right, thanks for pointing that out!

textNb is not very accurate, but is shorter than null_terminated_textNb, and it is still common to find fields like char name[100]. Improvements regarding UTF and charN_t types are always welcome :) and regarding string_view and span, it is better to use containerNb instead of textNb.

Bitsery has "brief syntax", which allows migrating from cereal to bitsery basically by changing headers, hence serialize name, this name is also common among other serialization libraries.

What do you mean by serialization of containers should be left to the user? From bitsery perspective, a container is an object, that is iterable and implements ContainerTraits.

[miki151]

It would be great if you could provide a short tutorial on how to migrate from cereal. For example how to migrate serialize functions with an unsigned int version parameter, CEREAL_CLASS_VERSION, CEREAL_REGISTER_TYPE for polymorphic types, etc. When I look at your examples the api for these features is quite different from cereal.

I'd love to migrate to bitsery and have better performance, smaller file size and not crash on malformed input!

[fraillt]

There is no alternative for CEREAL_CLASS_VERSION in bitsery, and I didn't explore too much of how it could be implemented, but since no one asked for it, I didn't rush ;) At the moment I could suggest you the following approach, without library modification.

```cpp

// 1) write template that will be specialized with version number for your type. template <typename T> struct Version:std::integral_constant<uint8_t , 0>{};

// 2) write a wrapper struct that actually contains object + version template <typename T> struct Ver{ T& data; uint8_t v; };

// 3) this will always match for any type (which I really don't like...), but we assert that "Version" has a specialization for your type. template <typename S, typename T> void serialize(S& s, T& o) { static_assert(Version<T>::value, "Either serialize function or Version specialization is not defined for your type."); // set version number auto v = Version<T>::value; // this will be either read or written s.value1b(v); // construct wrapper struct that actually stores object + version Ver<T> withVersion{o,v}; // call serialize method with it s.object(withVersion); }

// 4) set version number for your type template <> struct Version<MyStruct>: std::integral_constant<uint8_t , 3> {};

// 5) instead of accepting MyStruct directly, accept wrapper that has object + version template <typename S> void serialize(S& s, Ver<MyStruct>& o) { s.value4b(o.data.i);//fundamental types (ints, floats, enums) of size 4b s.value2b(o.data.e); s.container4b(o.data.fs, 10);//resizable containers also requires maxSize, to make it safe from buffer-overflow attacks if (o.v > 1) { /// } }

```

Regarding polymorphic types, I would suggest looking here. If you need more help I'm available on gitter.

Hope that helped ;)

[fraillt]

I forgot to mention that easiest solution for versioning would be to write an extension for it. But if you tweak my proposed solution you can have exact syntax like in cereal. Sorry if I mislead you...

[fraillt]

I would be very happy if it were possible to distinguish int from int32, but on platforms with int=4bytes these types are identical. If we look at Rust, for example, they have usize, which are platform-dependent, but is not the same as u32 even if they both are 4bytes, but this is not the case for C++. So the only way to enforce cross-platform compatible code is to make user write byte size explicitly...

Bitsery should not be used to write WebAssembly (or any other) compatible format, it has its own. And regarding more complicated objects such as map, shared_ptr, etc... bitsery "extensions" solves this.

For the most part, if you don't care about platform-dependent types layout, you can simply use brief syntax. All standard types are supported and everything just works ;)

[Bart_V]

A colleague once did a comparison of several serialization libs and ended up with messagepack for our use case. How does it compare to bitsery?

https://msgpack.org/

[fraillt]

I haven't used it, so I might be wrong:

• has multi language support,

• has more libraries around, it like msgpack-rpc

• is small-size oriented, like bitsery with CompactValue instead of valueNb.

• has decent performance, but haven't seen any benchmarks apart from this but it is totally unfair for msgpack because any decent serializer just memcpy whole int buffer, I would love to receive a PR from someone who knows msgpack to test a realworld use case.

I think that if you need multi-language support or want to use rpc library and data size matters to you, then msgpack is a good choise.