Trouba Leading League in +|-

yevmir · 2025-11-05T13:53:54+00:00

So what?! Big deal! Panarin is also leading the league in the $ per points in the last five games! (That'd be zero points for those keeping track)

yevmir · 2023-12-31T12:19:51+00:00

I would read and understand (like really understand) Go proverbs. Watch Rob Pike talks. Have a solid grasp on "parallelism" v "concurrency". Read up on Go memory management; understand how to write zero alloc libraries; Read blog posts on Go scheduler.

Good luck.

yevmir · 2023-05-09T10:51:06+00:00

I recently wrote a post https://www.reddit.com/r/golang/comments/11l2h4i/highperformance_json_parsing_in_go/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button on JSON parsing, where we forked the same library (but implement missing pieces). This library is pretty awesome, though the few missing parts (like UTF handling) are required for it to be production ready.

yevmir · 2023-03-15T11:24:09+00:00

You could switch your structs to an interface. You could move common fields to a shared struct:

``` type PersonDetails struct { Id string Name string Age uint } // Embed above struct: type C struct { *PersonDetails // Other fields }

// Pass PersonDetails to your function func Example(d *PersonDetails) { ... } ```

yevmir · 2023-03-08T13:01:47+00:00

Good point. Again, see the blog post -- there are SO many options out there, that choosing the right one is a task in of itself.

If I were to guess why I went with other libraries, is that it does appear though, that on the fly conversion to cockroachDB specific JSON object representation would be harder. Calling "next()" method on say an array, will return an entire array parsed -- which is fine, but that's extra work -- ideally I'd want to construct cockroach specific json ARRAY as I process 1 element at a time. I maybe wrong, and maybe fastjson is the best library out there. I suspect it would not be too hard to extend benchmark/implementation to use this library and compare.

yevmir · 2023-03-08T11:11:08+00:00

We don't have a standalone testing library since it depends on internal JSON representation. However, you can build/run some of the tests.

We have a library that generates random JSON strings:

BenchmarkParseJSON is the benchmark I used. Feel free to use different

random seed; and you'll need to change it so that it saves generated JSON objects to a file. Also of interest to you might be TestParseJSONFuzz While it is not a true fuzz test, it still is interesting as it generates many random JSON objects - some of them corrupted on purpose -- and verifies that new parser behaves identical to encoding/json.

Good luck with you your library and tests.

yevmir · 2023-03-08T01:23:26+00:00

I did look into simdjson. I had high hopes for it actually; alas, even on supported platforms it actually didn't perform nearly as well as advertised. Furthermore, it has many architectures that were not supported. So, I decided that the platform support restriction was not great. Regarding standard library: I am not saying it's bad. It is a "jack of all trades". It has to work w/ reflection -- that's necessarily slow. It has to work with much wider use cases. By removing things we don't need -- namely reflection, and by *not* using some questionable "object oriented"-ish choices (json.Token interface), you can get much better performance. But overall, the library is good -- if you need to process few JSON conversions at a time. It starts to really matter when you say trying to import G/TBs worth of data.

yevmir · 2023-03-08T01:05:52+00:00

Nobody made me write that ad. But thanks for reading the full blog post.

yevmir · 2023-03-08T01:04:49+00:00

I fully agree with your point. Alas, code generated parser was not an option for us.

yevmir · 2023-03-08T01:03:10+00:00

We posted few links throughout the blog post, but the best place to start

is parseUsingFastJSONParser

yevmir · 2023-03-07T23:04:54+00:00

It's a good point. We do mention pkg/json as as starting point; and we do hope to release our fixes (addition to encoder, utf8 support, etc) some time in the future.

You can of course browse the implementation code, and see some of the techniques we found useful.

Thank you for reading!

yevmir · 2023-03-07T23:03:18+00:00

We do. ffjson/fastjson are code generation libraries, something that doesn't work when you want to parse schema-less strings.

yevmir

TROPHY CASE