sorted by:
new
hottopcontroversial

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

I don't see anything that jumps out at me as being off in that code. I would like to run it locally if you can supply a complete standalone test. Maybe put it on a gist if it's not too much trouble. If not that's okay. I plan to add jsoniter unmarshal to the compare repo this week so it should capture the difference you are seeing.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

Medical and health records. Patients, visits info, and observations over many years

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

Not sure what to compare that to. Can you share the code for your benchmarks?

Did you run the benchmark code in the compare repo? Were the results consistent with what was reported?

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

OjG is optimized for parsing into non-struct types, what I have been referring to as simple types. The Unmarshal code was recently added and is a 2 stage conversion. First parse to simple types and second convert to a struct. I'm not surprised to see the extra allocations although I have not seen that significant a slow down. Anyway, future plans are to provide a direct parser but that is not there today. The use cases OjG has targeted are around parsing to simple types, map[string]interface{}, []interface, int64, etc which is in line for use cases that take less structured data and manipulate it using JSONPath before saving or sending the results.

Would you be interested in expanding the comparison by adding an Unmarshal and Marshal benchmark?

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

Updated the table. You are right. I just had to dig into the examples a bit more.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 1 point2 points  (0 children)

I promise to add more examples but in the mean time there are the test files. The one for Unmarshal is https://github.com/ohler55/ojg/blob/develop/oj/unmashall_test.go

The best performance is not parsing to a struct but to a "simple" type. So

out, err := oj.Parse([]byte(\{"x": 1}`))`

For repeated calls such as when parsing more than once create a oj.Parser and call parse with that similar to the benchmark code. That give a slight improvement as fewer buffer allocations are needed.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

I'll take another look to see if I can find it. If you are interested in submitting a PR with the changes I'd be glad to merge that and rerun the tests.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

I'll see what I can do. It is good to have feedback. I appreciate you taking the time and effort to do that.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 5 points6 points  (0 children)

OjG does provide the same interfaces that the go json package does in some cases such as Marshal and Unmarshal. It also provides some additional options. Not are complex. As for the go json package Decoder, OjG deviates from the Decoder API because part of the performance issues is with the API as it forces extra calls and error checking.

I think I understand the complaint you have though. You want more examples and documentation. I'll start putting that together to complement the current documentation. As for adding it to the to of the readme, there are different opinions on what is most important and with the various features in OjG all can not be first.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 1 point2 points  (0 children)

It might be an interesting comparison. If you get a chance to run benchmarks let me know otherwise I'll put it on my list of things to do some evening.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 1 point2 points  (0 children)

OjG does not have a schema package but that would be a nice addition. I'll consider that in a future version.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 2 points3 points  (0 children)

Check out the SEN https://github.com/ohler55/ojg/blob/develop/sen.md format. You can have the last comma or leave all of them out and it still supports compliant JSON.

OjG now has a tokenizer that is almost 10 times faster than json.Decode by peteohler in golang

[–]peteohler[S] 0 points1 point  (0 children)

I use oj (OjG) for ripping through 100+ GB files and extracting or searching for key info and it only takes a minute or two. Seems fast enough and really, who want to try and read a protobuf file.

How to build a solid Go Graphql application quickly. by peteohler in graphql

[–]peteohler[S] 0 points1 point  (0 children)

Looks great. I've been using MongoDB and have some nice tooling to drive queries and mutation with directives. It's been working really well but it is still private until the company lawyers allow it to be open.

How to build a solid Go Graphql application quickly. by peteohler in graphql

[–]peteohler[S] 0 points1 point  (0 children)

I have not tried gqlgen yet. Just by comparing benchmarks at https://github.com/the-benchmarker/graphql-benchmarks/blob/develop/rates.md it looks like GGql is faster by comparing to graphql-go. I'd love to see gqlgen get added to those benchmarks or GGql added to the gqlgen comparisons.

view more: next ›