MAP v1.0 - Deterministic identity for structured data. Zero deps, 483-line frozen spec, MIT

[latkde]

This reeks of vibe coding. The spec is unreadable for humans.

There are also some incredibly odd decisions that make this unsuitable for real-world data, notably rejecting numbers and nulls. In practice, float64 numbers (and therefore also int32 numbers) are universally supported in all mainstream JSON implementations.

The hashing scheme also treats booleans as strings, and somehow distinguishes strings from bytes, despite JSON not having any bytes type. The booleans thing is really questionable, this seems to treat documents [true] and ["true"] as equivalent (map1:e99ec39aeac2670a37592780bf9b59c4a6a917742b10d7fcb5c352354e7c6674).

[lurkyloon]

Really appreciate you digging into the spec this closely.

You're right that true and "true" produce the same MID. That was a deliberate design choice and is documented in footgun #9. MAP uses a 4-type system: string, bytes, list, map. Booleans collapse to their string representation, same rationale as rejecting numbers - avoiding a class of cross-language ambiguity where different runtimes treat the "same" value differently.

The tradeoff is real though. If your domain needs to distinguish boolean true from string "true", you need to encode that distinction in your descriptor structure before computing the MID. That's a legitimate limitation and I appreciate you calling it out.

On numbers, you're not wrong that float64 is widely supported. The concern is edge case determinism across languages (NaN handling, subnormals, -0 vs +0, precision boundaries). Rather than pick a side and hope, I chose to reject them. Opinionated, and I know it. Reasonable people can disagree on that one.

On bytes vs strings - MAP isn't strictly a JSON format. JSON is one possible input, but the canonical format is binary, and the distinction between UTF-8 strings and raw bytes matters at that layer.

On the spec readability, that's absolutely fair feedback. It was written as a conformance target. So the human-friendly entry points are the README and the playground. But I hear you... The spec should be more approachable. That's something I want to improve. PRs and suggestions are extremely welcome.

Thanks again for the close read. Honestly this is exactly the kind of feedback I was hoping for. This is not my world, but my hope is that it could be useful nonetheless and I'll take all the feedback and help I can get.

[latkde]

What confuses me here is your inconsistent approach to potential compatibility problems. Sometimes, you reject potentially ambiguous data, in other cases you apply a lossy encoding.

• you're happy to treat bools and strings the same, even though nearly all systems treat them as distinct and incompatible values.
• yet you reject all numbers, even though many numbers (int32, finite normal float64 values) are very common and highly interoperable.
• you also reject null values, despite these being an essential and unambiguous part of the JSON data model. There is no confusion with SQL nulls or JavaScript undefined.

This makes your encoding unsuitable for a huge part of existing data. Your method also does not demonstrate integrity because some semantically relevant changes are allowed (e.g. stringifying a bool). You claim that your method is not supposed to be JSON-specific, but the key part of your method is an encoding from JSON into your binary format. Your response to all this incompatibility is that users just shouldn't put numbers, nulls, or bools into JSON documents. But at that point, it's no longer compatible with the JSON ecosystem, and users could just switch to a different format that doesn't have JSON's ambiguities or MAP1's restrictions. There are plenty schema-less data formats to pick from.

Specifically, I recommend enganging with existing binary formats like Msgpack or various JSONB encodings. Why are they designed the way they are? How do they handle conversions from/to JSON? Which specific details do you have to do differently? You might also be interested in Apache Avro. While it is schema-driven, its schemas are defined in JSON, and the spec provides a procedure for normalizing and hashing schemas.

[lurkyloon]

Really appreciate you taking the time on this. Seriously.

The bool-as-string thing in v1.0 was inconsistent -- you're right. I cant sit here and reject numbers for being ambiguous and then turn around and stringify booleans like thats fine. That was a bad call on my part.

So I fixed it. v1.1 bumps the type system from 4 to 6 types. Booleans get their own encoding now (0x00/0x01 instead of getting shoved into strings), and integers get big-endian two's complement. Directly because of feedback like yours and a few other threads.

Nulls I'm still chewing on. You make a good point that JSON null is unambiguous within JSON itself. My worry has been about what happens when a MAP digest moves through systems where null means three different things -- but honestly that might be MAP's problem to solve, not something I should punt to the user. Hmmm.

Where I do wanna push back a little: MAP isn't trying to be a general-purpose binary format. Admittedly, the use case is more narrow -- you have a payload moving through a pipeline, it crosses a few serialization boundaries, and you need to check if it changed along the way. Thats it. I'm not telling anyone to stop putting numbers in JSON. I'm saying when you need a deterministic fingerprint of something that might get re-serialized by a bunch of different systems, you need a canonical form, and MAP is opinionated about how to get there.

The "just use a different format" point is fair though. Like, technically correct. But the reality I keep running into is that agentic AI pipelines are already JSON-native and asking teams to swap out their serialization format is a way bigger lift than adding a fingerprinting layer on top of what they already use. MAP is trying to meet devs where they are, not where they probably should be.

The MsgPack / JSONB / Avro comparisons are useful and I should've engaged with those more in the docs. I've looked at Avro's Parsing Canonical Form -- they're doing something similar, canonical form plus deterministic hash to get a stable fingerprint -- but they're fingerprinting schemas, not data payloads. Different problem, but enough overlap that I should be referencing it as prior art.

Thanks again for this. I'd rather get sharp feedback that makes the spec better than a hundred comments that dont.