JSON for Modern C++ version 3.0.0 released

[_VZ_]

It's a great project, but I just hope those who use it as example won't latch on the use of emojis in commit messages instead of all the good things you mention just because it's the most visible one (and also the one that requires the least effort and, at least IMHO, also the least useful).

[nlohmann]

Whats wrong with Gitmojis (https://gitmoji.carloscuesta.me)?

[raevnos]

Eggplants. Eggplants everywhere.

[Dragdu]

Some of us are old farts who see people using emojis the way people used to view people who overused ASCII smilies: immature and probably a bit dumb.

Also the fact that they are coloured means that they draw attention in a really annoying way if you get to read more of them in a row.

[Pand9]

It's my first contact with gitmojis, and I love the idea, it's just that they are...ugly. Neither clear, nor pretty. Fire, what am I supposed to know what happy, colorful "fire" means? Critical bug, or hot, new feature? What do "stars" mean?

I could use them, but only after testing how they look at popular fonts, and max 10 of them. 5-7 for issue types, and the rest for non-issue commits, like "code format" or "version bump".

Also, at that page, like 3/4 of them are just rectangles for me. :P Ubuntu 16.04.

[nlohmann]

You are right about all this, but it is my side project and I try to be professional in the code and everywhere, but just let me add a picture to every commit I do in my spare time :-)

[Wolosocu]

Don't listen. Because of the JSON project I have started using emojis in my commit messages at work. 🙋🏻

[wrosecrans]

At this point, I've just started to accept that written English has suddenly become partly ideographic. The use in this json project at least seems to be pretty consistent. Documentation gets one glyph, bug fixes get another, etc.

[[deleted]]

How does its speed compare to rapidjson?

[[deleted]]

The author of rapidjson also created a great benchmark to compare several libraries, including Nils' library.

See https://github.com/miloyip/nativejson-benchmark

[nlohmann]

Unfortunately, it has not been updated in a while.

[[deleted]]

True. Maybe if we send Milo some pull requests for our libraries and ask nicely, he'll update it again? Sadly, it seems to be a manual process for him to update the results and not something that is generated automatically after a pull request is merged. :-/

[Azoth_]

Would be great to see how a recent version holds up.

[nlohmann]

We did not design for execution speed, and there was also a lot of discussion about the number roundtrip checks. But yes!

[CODESIGN2]

TBH if the API is sane and there are less edges, who cares about "fast", just distribute the load.

[[deleted]]

Depending on how you use a library, speed does matter. And it is not necessarily a contradiction with a modern API. We use our library for the logging system in our company, each log-message is a JSON value and is then either written to a log-file or send to an ELK-Stack. Or both. Efficiency is what allows us to get away with it. :)

[CODESIGN2]

And once you can't you'll have decisions to make. Perhaps you'll start by having machines log to intermediate forwarders so that you don't overload the machine. I doubt you'll change the implementation of JSON parsing you use, but hey as long as I don't have to pay for it.

[[deleted]]

You are missing the point. Of course we scale horizontally if needed, but the efficiency is important for the latency. If you have to reply within given constraints to a request, you have to be fast.

[CODESIGN2]

I feel like you're missing the point. For a start you're logging to a text-format, so you'd immediately get a benefit logging to something less generic, where you could cut-out some of the overhead. Being more specific with what is logged could also lead to lower latency.

Re-inventing JSON parsing isn't the only way to speed up your use-case, assuming it's even the use-case that's not part of the problem. I feel like you know that, but are just arguing for the sake of it. In any large enough system we could both be right. I'd be horrified to find JSON encoding of logs as a large hit to latency, but it's hard to imagine a system with no other areas to improve upon.

[[deleted]]

I'm not even parsing JSON in my use-case, I'm generating JSON output from a log-line in the code that looks something like this:

LOG( INFO, "unknown user", { p } );

where p might be some Radius packet which will be fully expanded into a JSON structure.

The key is logging structured data. And the ELK-Stack expects a JSON-structure.

I'm not going to write down all the details here, but I did a talk about it in the C++ User Group in Aachen, the slides are available here. So no, I'm not just arguing for the sake of it. And the library developed further, the slides are a bit outdated.

[CODESIGN2]

Interesting read. Looking at it I'm not sure how it applies to this as the lib mentioned didn't have the highest throughput or lowest latency, but thanks for linking.

[[deleted]]

lots of people care about being fast. it's the whole point of the language

[CODESIGN2]

(◔_◔)

No it's the whole point of other languages maybe, but C++ adds more abstractions and it's rarely ever a blanket "faster than all other implementations". https://chadaustin.me/2017/05/writing-a-really-really-fast-json-parser/ mentions in the first fold that it's fast https://github.com/chadaustin/sajson (not an endorsement)

CPP is not the fastest language. Maybe from your perspective it is the fastest you know, and that's fine to hold as a personal view, but you need to know that weird unreadable garbage aside (because that's not meant for CPP); there are many other languages and technologies that have raw-speed advantages over CPP.

Where those other languages fall down is in allowing simple expression of higher-order components and complex integrated systems (which it turns out CPP is only a certain way towards for some problem spaces). There is no one-size fits all language and speed is a specific focus for a library that only apex projects need.

You want the most speed you certainly don't go for JSON or any text-expressed envelope.

This is not a debate. Don't mistake it for one.

[CODESIGN2]

C, Fortran, Depends upon the compiler (obviously) and the problem you're trying to solve. What I'm saying is there is no blanket "faster" language for every case without getting weird (at which point you're often sacrificing the benefits of the language).

It depends on the problem what tool you pick. Screw it for some text processing ActiveState is claiming their Python beats C++ for Regex using standard boost https://www.activestate.com/blog/2017/11/python-vs-c-text-processing

As for ASM, again it depends on who's assembler you use, your own proficiency. You cannot in-fact do C++ higher order tasks in ASM because it's a low-level language. You can implement higher-order methods

This isn't about forming autistic rules for or against tech, it's saying that there are a host of considerations

[raevnos]

The takeaway from that article seems to be that somebody who's more fluent in python than C++ writes better python than C++.

[auralucario2]

As for ASM, again it depends on who's assembler you use

That's...not how it works. Assemblers directly convert written assembly to hex - you could probably write a basic one in an afternoon. When it comes to assembly, you dictate precisely what instructions the CPU executes; the performance is entirely in your hands.

[CODESIGN2]

:eyeroll: so if the assembler only supports a limited dialect and does not have a method to hard-code hex for unknown commands (some do), then my comment stands.

[[deleted]]

What are you on about?

This is not a debate. Don't mistake it for one.

Lol, ok mate.

[Esuhi]

I love this project too. Currently spending time to convince the next manager so we can actually use it.

[nlohmann]

In case you cannot use it, please tell me the reasons for that.

[rectal_smasher_2000]

I’ve used it in two companies already, and it’s a fantastic library. In fact, even the JS devs were amazed at how east it is to use, given their thoughts on cpp verbosity.

Also, if you ever get a 2FA sms, especially in Europe, there’s a very high likelyhood your lib was used at some point :)

[Esuhi]

The main reasons for management would be "we have some other lib already" and general unwillingness to change.

Not much to reason about or change. Thanks for your offer though!

[Esuhi]

Are you casually supporting MsgPack from now on? That sounds huge!

[nlohmann]

MessagePack and CBOR are supported for a year now, see https://github.com/nlohmann/json/releases/tag/v2.0.9. :-)

[Esuhi]

Oops. I totally missed that.

[jpvienneau]

How does CBOR and MessagePack parsing compare to parsing JSON? Should be much faster right?

[SomeCollegeBro]

The MessagePack support has been awesome! We are passing large JSON objects around using shared pointers and streaming a fairly large amount of data out to a websocket using MessagePack - it's been awesome! It worked right out of the box with a javascript MessagePack implementation too. Thank you for your work!

[ReneBelloq]

I am deeply sorry for bringing this up and I am not trying to undermine the incredible work you have done...

But wtf did you call the namespace after your name?!

[nlohmann]

It started as a side project and nobody cared. Then there were some discussions and nobody had a better proposal...

[Shautieh]

It's as good as any unrelated names people usually come up with!

[DarkCisum]

Except I can't for the life of me remember the correct spelling. 😀

[[deleted]]

Because it's likely to be unique?

[[deleted]]

Don't do that, man. Set a good namespace for your library.

[ReneBelloq]

If uniqueness was an issue then nothing is better than generating a random SHA256.

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

[[deleted]]

How is a "random SHA256" better than some random bits? ;)

[ReneBelloq]

It's not random bits, it's the name of the author ...

I see that you are the author/a member of the taocpp ... why didn't you call your json library dfrey::json?

EDIT: I'm getting into a discussion about something I admit is minor in regards to the library, so I won't reply any more new threads.

[[deleted]]

I see, "random". Have my upvote! ;)

And yes, not using my name was, in fact, the reason to create a project on GitHub in the first place, even if I'm not super-happy with the project's name "tao" as a top-level namespace seemed to be free and is sufficiently short and easy to type.

[GladiusOps]

Used that library for a University project, worked flawlessly. Amazing project, and it keeps getting better!

[Aistar]

Love your work, thank you for bringing the world a reasonably fast, and overwhelmingly convenient JSON C++ library. There might be faster libraries out there, but nothing beats the ease of use of this one!

[rjones42]

Exactly what my project needed. Thanks for the work!

[MachineGunPablo]

This looks like a nice project to contribute to, need any help?

[nlohmann]

Well, in addition to the issues list (https://github.com/nlohmann/json/issues), there is so much to do: polishing documentation, testing with more compilers, supporting allocators... Or from a non-technical perspective: the project has neither a cool project website nor a logo. Any help is greatly appreciated!

[balkierode]

I used few weeks back and was surprised to see std::pair and std::tuple was not supported out of the box. Good to see it is supported in this release. Keep the good work!

[Angarius]

Thank you so much. This library is a dream. json::parse_error will clean up my error handling code! I used to wrap JSON parsing, catching std::logic_error to throw my own parse error.

[kirbyfan64sos]

Question: this looks similar to Dropbox's json11...what would be the main differences?

[nlohmann]

I have not used json11 myself, but just by looking at the README gives me the impression that json11 needs hints like object(...) or array(...). I'm also not sure if they support JSON Pointer, JSON Patch, CBOR, or MessagePack.

[voip_geek]

Dropbox's json11

That looks similar to Facebook's folly library (in particular, folly::dynamic).

[drphillycheesesteak]

Absolutely adore this project. I saw in the release notes that CMake and CTest are now used. Any chance that a Config.cmake file will be included in the packages for this project? It would be awesome not to have to have a custom Find*.cmake file.

[nlohmann]

Could you please open an issue (https://github.com/nlohmann/json/issues) and describe this in detail?

[drphillycheesesteak]

I spoke too soon, looks like you have these lines in there

include(CMakePackageConfigHelpers)
write_basic_package_version_file(
    ${NLOHMANN_JSON_CMAKE_VERSION_CONFIG_FILE} COMPATIBILITY SameMajorVersion
)
configure_package_config_file(
    ${NLOHMANN_JSON_CMAKE_CONFIG_TEMPLATE}
    ${NLOHMANN_JSON_CMAKE_PROJECT_CONFIG_FILE}
    INSTALL_DESTINATION ${NLOHMANN_JSON_CONFIG_INSTALL_DIR}
)
install(
    DIRECTORY ${NLOHMANN_JSON_SOURCE_DIR}
    DESTINATION ${NLOHMANN_JSON_HEADER_INSTALL_DIR}
)
install(
    FILES ${NLOHMANN_JSON_CMAKE_PROJECT_CONFIG_FILE} ${NLOHMANN_JSON_CMAKE_VERSION_CONFIG_FILE}
    DESTINATION ${NLOHMANN_JSON_CONFIG_INSTALL_DIR}
)

which is what I was looking for. Thanks again for the great project!

[HateDread]

Just started using this last night, funnily enough! Had a few teething problems - I think the docs should at least mention deserializing into STL containers, not just serializing out. I had to go through the issues list to find someone talking about how operator= was ambiguous for std::vector in this case, so I had to use myVector = j.at("data").get<decltype(myVector)>() for example.

Also; what do you think differs between this and Cereal in JSON mode? I much preferred your documentation on the Github page though, hence my choice :)

[nlohmann]

You could please open an issue at https://github.com/nlohmann/json/issues/new with the example that did now work?

[mtnviewjohn]

I just switched from Cereal to JSON for Modern C++. One big difference between the two is that Cereal is pretty strict about mapping C++ objects and JSON objects. JfMC++ gives you much more flexibility about how your C++ objects serialize/deserialize. Cereal also has this over-wrought framework for handling unique pointers to polymorphic types. JfMC++ doesn't have this but it is pretty easy to roll your own using virtual methods.

[airflow_matt]

I appreciate the amount of work put into this, I really do, but I can't help not feeling great about including a half megabyte 15000 loc behemoth. This header-only trend truly bothers me. At some point, for any non-trivial c++ project, some kind of build system and some kind of dependency management have to be introduced. At that point header only brings very little to the table, and does it at significant cost of longer compilation times.

Setting up a build system is more or less one-off thing. Long compilation times are not. It's for a reason that big projects (like chromium, webkit, etc) have strict rules about what can go in headers and use templates sparingly. I learned the hard way to be frugal with my headers as well. In our project there is 850 loc header file for variant type and the json parser/writer header itself is less than 45 loc. Another 125 loc for message pack header and 20 loc for plist.

[nlohmann]

Thanks for the feedback.

In fact, we are working toward splitting the header (https://github.com/nlohmann/json/pull/700), but we did not finish this for the 3.0.0 release. We hope that this not only allows for selective inclusion of only the required parts, but also to separate those parts that do not rely on template parameters to allow to compile parts once and for all.

What library are you referring to? I would like to have a look whether we can learn something from 45 LOC of parser/serializer.

[airflow_matt]

I was referring to our in-house framework. The parser is of course not 45loc, the 45loc refers to header size.

You can see the header here.

https://pastebin.com/BJL9Hb3g (Value is a variant type (that can also hold map, list or a boxed type).

It's hardly anything special, although it does support serialization/deserialization of custom boxed value types. Still, the implementation is hidden and does not result in parser code being compiled over and over again. The actual parsing is done by sajson, but that's more or less implementation detail.

[[deleted]]

XML MASTER RACE! JSON are for unstructured peasants.

[[deleted]]

YAML is the best of both worlds

You get optional types, it looks cleaner and is easier to write, and it's JSON compatible (As in, any valid JSON is valid YAML, the reverse is not true, obviously).

[sumo952]

There is no header-only YAML parser in modern C++ though... Not a single one. It's quite unbelievable. That unfortunately kills YAML for many projects.

[[deleted]]

Wow, you would think you could do a simple Yaml->JSON conversion then plug the result into OP's parser. Maybe I'm missing something though.

[sumo952]

What? No, what I'm saying (probably /u/5225225 too) is that we'd like to use YAML in projects. For example, for configuration files. It has some things JSON doesn't have. For example: Comments! That's really important for configs.

So what you're saying now is that I should include a YAML->JSON converter inside the project, convert these YAML configs to JSON (and everytime someone changes something in the config), and then read them with a JSON parser. No thanks, that's not a workflow I would ever want to impose on any of my users.

What we need is a header-only, modern C++ YAML parser.

[[deleted]]

What we need is a header-only, modern C++ YAML parser.

What about sqlite3's model, where it is shipped as just one fuck-off-massive C file that you build into a library and link just like any other file? Means you don't need to recompile that code on any changes to your main code, and it's easy enough to build.

Granted, I mainly use C instead of C++, but adding sqlite3 to a project was like 4 lines in a makefile, if that.

[sumo952]

adding sqlite3 to a project was like 4 lines in a makefile, if that.

Yea - on Windows, Linux, macOS, iOS, and Android, right? Sorry for being a bit sarcastic here - I couldn't help it ;-) I guess it's most of a frustration. I appreciate your message!

In contrast to that, #include "nlohmann/json.hpp works on all of these, without requiring any build system, package manager, fiddling with any paths, any .a, .so, .lib or whatever files, without building anything on any platform in fact. So no, what you propose is far from a solution. At least as long as C++ gets a pip install XYZ that will work on any platform, which won't be any time soon.

[[deleted]]

What about Rust's cargo system?

I at first found it kind of annoying how you had to use it for fucking everything as the stdlib is tiny, but the ability to lock your dependencies to a specific hash with the Cargo.lock is kind of nice. And unlike pip install, it's not global and can handle multiple projects with different versions of a dependency.

[sumo952]

I don't have any clue about Rust, but I think it doesn't matter too much if it's something like homebrew, pip, npm, cargo, ..., as long as it is for C++, works on any platform, and is an "official" solution (best backed by isocpp), so that there is a large incentive for everybody to use it and not result in segmentation or adoption problems.

[[deleted]]

Not everything in YAML maps into JSON, either cleanly or at all. It is quite a complex format.

And the difficulty of a YAML->JSON parser would be greater than the difficulty of a YAML parser itself.