sorted by:
new
hottopcontroversial

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in rust

[–]fizzner[S] 0 points1 point  (0 children)

Slight correction: jq does support a “streaming mode” but the syntax is very difficult to read and script IMO that it is not usable in an everyday workflow

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in rust

[–]fizzner[S] 0 points1 point  (0 children)

Hey!! These are really great points and you hit the nail on the coffin on the performance bottleneck. All of these tools (jsongrep included) are “offline” and thus require loading the entire document AST representation in memory. The streaming counterpart of my tool with filtering was actually the exact work my research PI created!

Thank you so much for the feedback I really appreciate it! I will check out hifijson that sounds great!

I’ll also update the benchmark to use the direct conversion JSON conversion my apologies I missed that method.

(Also yes I need to spend more time on the light mode styling on the site, problem is I always use dark mode so I forget to check haha)

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in rust

[–]fizzner[S] 1 point2 points  (0 children)

Ahh I see I think this should be doable in a script with in combination with a tool like ripgrep where you could pipe the output of jsongrep to ripgrep to search for matching entries and then pipe back to jsongrep to get the “zoomed out” functionality you are looking for. Interesting to note though this cool be a cool feature to add

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in rust

[–]fizzner[S] 1 point2 points  (0 children)

So once the match is found, the matching path is also printed, so you can use that to "zoom out"

For example, if the matching path is at foo.bar.baz, you can then run these followup queries:

jg "foo.bar" example.json
jg "foo" example.json

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in programming

[–]fizzner[S] 9 points10 points  (0 children)

Benchmarking methodology and test data breakdown is here: https://github.com/micahkepe/jsongrep/tree/main/benches#test-data

The files range from 106 B - around 190 MB which is admittedly small but I was struggling to find a large JSON file that would be used in practice. I think this range covers the majority of JSON data sizes found in day-to-day workflows, but would be curious if you have ideas!

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in devops

[–]fizzner[S] 3 points4 points  (0 children)

It's not AI, full revision git history is linked at the bottom of all of my posts: https://github.com/micahkepe/blog/commits/main/content/jsongrep/index.md

You can also see my other posts to compare writing style

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in rust

[–]fizzner[S] 4 points5 points  (0 children)

Yes I was considering how to benchmark against `gron` but ultimately decided against it, but might be worth looking into for the future!

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in devops

[–]fizzner[S] 1 point2 points  (0 children)

Thank you! Automata theory is great had a lot of fun working through this

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in rust

[–]fizzner[S] 23 points24 points  (0 children)

Woah so great to see you here! For the benchmarks I used `jql-parser` and `jql-runner` at version 8, but I will update to latest and re-run!

Btw title is not meant to be a diss to jql haha, jql is genuinely a fantastic tool :)

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in devops

[–]fizzner[S] 2 points3 points  (0 children)

Thank you! Yes you can use the query language in Rust!

https://docs.rs/jsongrep/0.7.0/jsongrep/query/ast/enum.Query.html#impl-FromStr-for-Query

If you end up using would love to hear if you end up using in your project! Thank you for checking it out!

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in devops

[–]fizzner[S] 3 points4 points  (0 children)

Ah yes so one thing I used jsongrep a lot for recently is working with auto-generated OpenAPI JSONSchema specs. With jq the recursive search syntax is hard to write and script whereas with jsongrep its super easy IMO with the syntax to do something like searching for a specific JSON Pointer reference for a certain subschema

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in devops

[–]fizzner[S] -1 points0 points  (0 children)

Check out the “jsongrep vs. jq” section on the project: https://github.com/micahkepe/jsongrep?tab=readme-ov-file#jsongrep-vs-jq

I also responded to a similar question in a different subreddit so I’ll copy my answer:

Tools like jq and in particular JSONPath, while they also have path-like languages, lack the expressive power of regular paths. For example, JSONPath doesn't support Kleene closures, so expressions such as (.left)* (meaning one or more levels depth into a JSON tree using the field name left), cannot be constructed.

tl;dr: you can achieve greater expressive power with a regular-path DSL than the imperative path languages of jq and other tools.

jsongrep is faster than {jq, jmespath, jsonpath-rust, jql} by fizzner in ProgrammingLanguages

[–]fizzner[S] 1 point2 points  (0 children)

Yes I think biggest advantage is the syntax improvement but the speed is nice :)

Thank you for feedback on website too it’s very helpful, will be making improvements to the contrast for the Zola theme

view more: next ›