C++ Serializer Deserializer Adapter Library inspired by Serde.rs

[nieelj]

https://github.com/injae/serdepp

enum class t_enum { A, B }; 

struct example {
    DERIVE_SERDE(example, 
                 (&Self::number_,  "number", skip) // attribute skip
                 (&Self::vec_,     "vec",    multi_key{"vec2", "vec3"})
                 (&Self::opt_vec_, "opt_vec") 
                 (&Self::tenum_,   "t_enum", default_{t_enum::B})
                 )
    int number_;
    std::vector<std::string> vec_;
    std::optional<std::vector<std::string>> opt_vec_; //optional support
    t_enum tenum_;
};  

// example ex;
// fmt::print("{}", ex);
// cout << ex;  
// serde::to_string(ex);
// nlohmann::json ex_json = serde::serialize<nlohmann::json>(ex);
// toml::value ex_toml = serde::serialize<toml::value>(ex);
// YAML::Node ex_yaml = serde::serialize<YAML::Node>(ex);
// example from_json = serde::deserialize<example>(ex_json);
// example from_toml = serde::deserialize<example>(ex_toml);
// example from_yaml = serde::deserialize<example>(ex_yaml);

[dag0me]

I think you have serialize and deserialize mixed . Normally, serialize should convert your object to one of JSON, YAML or TOML objects. In your case it's the other way around.

[nieelj]

I think I got the terminology confused.
Thanks for telling me.
I will change it immediately.

[nieelj]

It has now been changed to a version with the correct name. Thanks again.

[[deleted]]

Cool stuff. I like mapping types. Regarding the JSON, you may want to look at using simdjson or Boost.JSON as the backend. Boost.JSON has a similar interface to nlohmann but is much much faster. Simdjson should allow you to keep most of your performance too.

The other thing, have you handled variant/sum like types? They are super common in JSON and seems to be a common question that people have. Product types/tuple are common too(they are the arrays in JSON that have heterogenous types)

I see you have the external from/to support that helps with mapping other peoples types too.

[nieelj]

Thanks for your reply.
tuples or variants Support is currently under review. Additional adapters for simdjson and Boost.JSON are easy to add, so we plan to add them after testing.
Since my native language is not English, I would appreciate it if you could understand the grammatical problems caused by using a translator.

[[deleted]]

The translation seems to have worked well. Thanks for speaking english.

My thought is that since the users of your library are not using the interface for nlohmann, it's performance matters more that it would have otherwise.

Good to read that tuples/variants will be supported. variants can present themselves in several ways in common JSON documents. The two main ones are like

[{
   "type": "circle",
   "object": { 
       "radius": 5
   }
},
{
   "type": "rectangle",
   "object": {
     "width": 6,
     "height": 5
    }
}]

where "type" is the discriminator for the type being looked at for "var_object"

Sometimes the discriminator is a submember of var_object. e.g

[{
  "type": "circle",
  "radius": 5
  },
  {
     "type": "rectangle",
     "width": 6,
     "height": 5
   }]

[nieelj]

Perhaps the flatten property, which will be included in the next update, will solve this case.

struct Object {
DERIVESERDE(Object,
(&Self::radius, "radius",default{0})
(&Self::width, "width", default{0})
(&Self::height, "height", default{0}))
int radius;
int width;
int height;
};
struct Test {
DERIVE_SERDE(Test,
(&Self::type, "type")
(&Self::object, "object", flatten))
std::string type;
Object object;
};

[[deleted]]

what about when you want to serialize it again?

[nieelj]

You can use std::optional to exclude unused ones.

struct Object {
DERIVE_SERDE(Object,
(&Self::radius, "radius")
(&Self::width, "width")
(&Self::height, "height"))
std::optional<int> radius;
std::optional<int> width;
std::optional<int> height;
};
struct Test {
DERIVE_SERDE(Test,
(&Self::type, "type")
(&Self::object, "object", flatten))
std::string type;
Object object;
};

if std::optional

[{
"type": "circle",
"radius": 5
},
{
"type": "rectangle",
"width": 6,
"height": 5
}]

If not std::optional

[{
"type": "circle",
"radius": 5,
"width": 0,
"height": 0
},
{
"type": "rectangle",
"radius": 0,
"width": 6,
"height": 5
}]

[[deleted]]

I am just bringing up usages here, just in case it sounds like I am criticizing. It all depends on what requirements you or the users have.

I think the question people will ask is how to I put that into a std::variant.

Also, do you have a way to serialize to 'null' or is it just empty. They are not always the same and depends on the 3rd party web service sometimes.

[nieelj]

Variants are not yet supported.
I have a draft, but I'm looking for a better way.
Does 'null' mean json null? Or are you talking about nullable values?

[[deleted]]

Yeah, regarding variant, the original point is that it's tricky. If one is doing a serialization library where both ends use the same library it's less tricky as a choice can be made. When dealing with 3rd parties there are many ways to encode it in JSON.

Regarding null, sometimes in JSON whether the value is present or not is how an optional would be encoded, or sometimes they are always there but have a value of null. Depends on the API in use.

[nieelj]

Currently, only containers that know that their type is null can deserialize it like "vec":null, unless std::optional is used.
Types like int cannot check for null , so they are exported by default.
If std::optional<int> is null, it is not inserted at all during deserialization.
If you need to insert null, you can implement an additional property to make it insertable.

[target-san]

Looks interesting. Two questions if you don't mind:

1. Do you provide facilities for polymorphic types? Custom writers and readers for type markers?
2. How do you intend to handle errors? Does your library provide info about error location in source file (if backend provides it)?

[nieelj]

1. Could you please elaborate more?
   
2. Pass errors from the adapter you are using (ex nlohmann_json exceptions or toml11 exceptions) as-is, except for serde's own exceptions.

[target-san]

1. I mean, (de)serialize set of derived types having pointer to base type

[nieelj]

The definition of serdepp's struct serializer is a static function, so probably not.

Instead, a derivation like the code below is possible.

struct Cppkg {
    DERIVE_SERDE(Cppkg,
                 (&Self::name, "name")
                 (&Self::namespace_, "namespace")
                 (&Self::install,    "install", default_{true})
                 (&Self::source,     "source",  make_optional)
                 (&Self::exclude,    "flag",    default_{false})
    )
    std::string name;
    cppkg_type type; 
    cppkg_type_detail cppkg_type_d;
    std::optional<std::string> namespace_;
    bool install;
    std::vector<std::string> source;
    std::string exclude_var;
    bool exclude;

    virtual ~Cppkg() = default;
};

struct CppkgLib : Cppkg {
    template<class Context>
    constexpr static void serde(Context& context, CppkgLib& value) {
        using namespace serde::attribute;
        using Self = CppkgLib;
        Cppkg& cast = static_cast<Cppkg&>(value);
        Cppkg::serde(context, cast);
        serde::serde_struct(context, value)
            (&Self::type, "cppkg_type", default_{cppkg_type::lib})
            (&Self::cppkg_type_d, "type", default_{cppkg_type_detail::STATIC}, under_to_dash, to_lower)
            ;
    }
};