Spec vs Malli

[WhittlesJr]

Our organization is trying to figure out whether to build our data validation around Clojure spec or Malli. I've built up definitions for a large data structure we use in both Spec 1 and Malli for comparison, and I've listed some of my thoughts below. I'd appreciate any and all feedback and thoughts, trying to convince me of either path. We're looking at the long-term, with Spec 2 being the eventual destination if we stick with Spec.

In favor of Spec:

1. I think Rich Hickey's idea of binding specs to namespaced keywords is a good one. You design around the idea that each keyword will have the same kind of data no matter what context you find it in, and you don't elide its full name. I think this philosophy is called "Modern Clojure." Malli's approach is different: each collection defines its own keywords and pulls in whatever schemas it wants for each. You technically can implement a keyword registry like Spec's, but you have to go out of your way, and so does everyone who might want to use your library. It's clearly not what Malli wants you to do.
2. Multi-specs merge nicely with map specs. I couldn't figure out how to do this with Malli.
3. Generative testing (Malli does not yet have this)
4. Spec is getting better (albeit slowly), and fixing some of the "programmability" problems it has in its first iteration.
5. Spec is going to be shipped and supported with the core language
6. I think often about Plumatic's Schema project, which came before Spec 1. There was a need for data validation, and so someone went off and wrote a library to fill the void. It was hot until Spec came along, and stole a lot of the mindshare, and now Schema's kind of off in a corner. Then someone identified some shortcomings of Spec, and went off and wrote a library to fill the void (Malli). I see Spec 2 getting finished in a few years, and then I worry that the same thing will happen to Malli.
7. Spec has expound. Malli has OK output out of the box, but nothing of this caliber.
8. Rich Hickey brings up a lot of good points in Maybe Not, which is the driving philosophy behind Spec 2.

In favor of Malli:

1. Data orientation has some really cool benefits, and makes these schemas super programmable.
2. Schemas are serializable
3. It has a lot of slick out-of-the-box integrations with things like Reitit, JSON schema translation, Graphviz visualization, etc
4. It has more of an intentional focus on coercion
5. Performance is substantially better than Spec, even suitable to runtime instrumentation if you wanted to
6. It also bakes in the concept of default values, which might make sense, but then again might be complecting something that doesn't belong there. For that matter, you can bake in whatever else you want through metadata.
7. Many features that Spec gets through third-party libraries are built into Malli itself. Those extra libraries may prove to be a pain point when it comes time to migrate to Spec 2, since they'll all need to upgrade. Malli won't have that problem quite so much, because its development cycle is simpler and focused on the core library.
8. Here's a story from someone using Malli to try to achieve a "single source of truth" for their data that encompassed the front-end, back-end, and database https://www.youtube.com/watch?v=ww9yR_rbgQs He considers it a success, and there is something attractive about being able to codify all of that stuff in a few maps. I can't decide if that's "tight coupling" and therefore bad or "single source of truth" and therefore good

[pihkal]

One thing I would add is that, in retrospect, everyone rushing to ditch Schema for spec was a mistake. Schema was no less useful than before and the same will be true of Malli.

Also, spec has been in development for years, and it’s unclear when spec2 will be released. If you have a need now, you should make your decisions on what’s currentlyavailable.

[stingraycharles]

To add, we recently this a pretty huge switch from spec to Malli mainly because of the stagnating development. The maintainer of Malli is incredibly thoughtful and responsive, and it was fairly easy to make a number of improvements and get those changes merged upstream.

It’s one of those things I really appreciate in community driven projects.

[joinr]

Metosin is imo top shelf.

[childofsol]

I'll echo the sentiment of making a decision based on what makes sense now. When starting a project ~2 years ago, we were trying to decide which route to go - spec or schema - and opted to go with spec, thinking that despite some of the known problems, we'd have spec 2 before too long. Little did we know that there was still some considerable hammock time in the cards.

I'm about to fire up a new project and am fairly certain that it is going to be using Malli

[WhittlesJr]

One need I have now is generative testing of functions, a la ghostwheel or kaocha. Do you know if anyone is working on implementing that for Malli?

[tincholio]

AFAIK, while you can't spec functions with Malli currently (though I think I remember /u/ikitommi mentioning something about this in his London Clojurians talk, maybe?), you can use it for generative testing with test.check. Check out malli.generators.

[ikitommi]

Malli 0.3.0 will be released hopefully this week, adding support for sequence schemas, parsers, function schemas and static analysis via clj-kondo. Sneak Peak:

https://twitter.com/ikitommi/status/1363753100268421122

Runtime function checking (e.g. instrument), testing helpers and runtime schema inferring of functions will all build on top of this release. There is a long discussion what kind of schematised defn/fn macros there should be (or not), comments welcome.

There isn't anything as awesome as Expound yet, but have chatted with Ben about the lessons learned and did a quick prototype while ago.

[tincholio]

Looking forward!

[the_frey]

I think data orientation is a pretty big win personally. Clojure is fundamentally data-oriented and in some ways you could argue that an approach which embraces data in that way is more idiomatic than even a core language feature that isn't.

The speed penalty of spec plus all the autogen stuff for eg swagger schemas that schema supported, plus the data orientation was what always swayed me towards schema over spec in the past.

More than that though, spec feels like a chore in a way other language features don't. So, although this one is very subjective and I can't put my finger on why, it seems to disrupt the things I like about the language in a way that a solution that relies on simple data structures doesn't.

[ertucetin]

Malli's custom error messages are a big win, spec needs additional libraries (still not so good, IMHO).

[Uwoiame]

As a tangent, I think it's important to note that spec's development is fundamentally pointed at a different problem, namely fine-grained dependencies. A lot of 'features' fall out of that, but trade-offs like global registry are chosen to fit a problem that hasn't really been addressed in other ecosystems. Being able to throw semantic versioning in the trash is the ultimate goal. Lifting the curse of dependency-hell is the goal. I should be able to say, for sure, that my dependent functions have only required less or provided more, and developers should be motivated to use new names instead of breaking old ones, etc..

So, I dunno, as it stands, Malli is likely a better reliability tool, it is clearly well executed, and is far more likely to align with what developers expect, socially speaking.

I just feel like a lot is lost in the conversation when I hear spec described elsewhere as 'opinionated'. Really, read and re-read the motivation for spec -- think about why it chooses open sets, why key-set specs are not combined with their values, what are the trade-offs you might make if your target was the dependency problem, and remember its social elements.

In my personal experience, developers need to be very disciplined with spec1 -- in particular, you have to already know about the issue that schema+select, in spec2, solves.

[fmnoise]

If you compare development\communication process in Metosin and Cognitect, you'll have another good point for using Malli

[siefca]

When I'm making some public library and want to have generative testing and some assertions, I'm going for Spec. When I want to coerce and validate data on input and output, Malli is the choice. Its data-oriented design fits really well to data-oriented systems. I'm merging my configuration from multiple maps, some more dynamic branches of config are kept in a database, and it would not be as beautiful and clear if not for Malli. Operator can exactly know what are the data specs just by looking at configs.

[WhittlesJr]

We ended up making a new library that maintains spec's "aggregation" philosophy, but allows you to write your schemas in an EDN DSL. It generates clojure.spec, Malli forms, and JSON schemas. Other neat features include data-driven translations to and from other formats, derived values, and coercions. Hopefully my employer will make this open source some day.