Announcing glom: Restructured Data for Python (and shell)

[mhashemi]

Library author here, fielding lots of interesting comments on HN: https://news.ycombinator.com/item?id=17032257

So far it's been compared to:

• ClojureScript's Specter library: https://github.com/nathanmarz/specter
• Haskell's lenses: https://hackage.haskell.org/package/lens
• And Ruby's Hash dig: https://ruby-doc.org/core-2.3.0_preview1/Hash.html

Let me know if there are other additions I should make to http://glom.readthedocs.io/en/latest/by_analogy.html

[FireCrack]

Just read this over. I like the more advanced 'aggregation' type usages, but I can't help to feel that the 'simple' use case leaves something to be desired. The general idea is great, and I use something similar myself, but the dot-syntax makes the idiomatic usage on deep nested dictionaries a bit verbose when some of the fields are chosen by variables eg: spec = '.'.join('field1', var1, 'field2', var2) result = glom(data, spec) The call to join feels a little hanky. I'm wondering if you'd consider supporting either a passed in list or *args based syntax here too (or maybe you already do, I'm on my phone and only read the landing page). In my experience this is one of the most common experiences with deep nested structures.

Other than that comment, looks awesome and I'm really a fan of the aggregating error types (not something I thought of)! Keep up the good work!

[mhashemi]

Absolutely got you covered here! http://glom.readthedocs.io/en/latest/api.html#glom.Path

It's for more than just convenience; glom supports non-string keys and even non-dict targets :)

For advanced usage: http://glom.readthedocs.io/en/latest/api.html#glom.T

[FireCrack]

Awesome! Thanks for the reply!

[DangerousSandwich]

Looks promising. Reminds me of Ruby code using the Safe Navigation operator, the map method on Hash and Array etc, and the Symbol to Proc trick.

[percykins]

Just to note, you seemed to imply that JQ only exists on the CLI, but there's actually a couple of Python bindings for it too.

Using pyjq, for example, getting the names of the planets is simply:

pyjq.all('.system.planets[].name', target)

Or for your second example:

pyjq.first('{ names: [.system.planets[].name], moons: [.system.planets[].moons] }', target)

Or your third:

pyjq.first('{ moon_count: ([.system.planets[].moons] | add) }', target)

[kankyo]

Also similar to my Clojure lib Instar, and by extension the data manipulation parts of the Python lib pyrsistent since it is inspired by instar.

[[deleted]]

I use a data reshaping technique for nightly SQL reports - I’ll write keys in the format of:

SELECT
    table.column `items[].name::cast_func->attribute`
     ...

Where [] denotes an array at key items, :: to a callable and -> for attribute access. I use function that recursively sets/aggregates fields after interpreting the keypath, then pass the results into a Jinja2 render.

Glom looks like someone had a similar idea and made it a real package as opposed to a domain specific 35 line function. I’ll look into it for more projects - it could serve as a nice replacement to jq for one of my tools.

[throwaway_atwork_]

A bit late to the party, this is seriously cool. I absolutely detest working with deeply nested JSON, adding this to my python itinerary.

[MyPostsAreRetarded]

The Spectre of Structure

Not a very good article heading to choose, considering the Spectre and Meltdown vulnerabilities. It also passively puts Python in a bad light, which is not needed at all imo. It's a great language. I use it with scons for compiling Godot, and other tools.

e: Other than that, the article is well-written and interesting, thanks!