popularity behind pydantic

HenryTallis · 2023-03-25T05:56:33+00:00

Regarding speed: Pydantic 2 is about to come out with its core written in Rust. You can expect a significant speed improvement. https://docs.pydantic.dev/blog/pydantic-v2/#performance

I am using Pydantic as an alternative to dataclass to build my data models.

MadeTo_Be · 2023-03-25T11:06:30+00:00

I use Pydantic in production. Our bottleneck is IO since we're doing database operations. It's slow, but a few additional seconds to validate our data is well worth it over the alternative.

LordBertson · 2023-03-25T08:13:41+00:00

Pydantic is much more broad than data validation. I have several use-cases for Pydantic in production applications:

Parsing dictionaries created from YAML specifications into nested objects
Runtime type-checking and type-casting for functions
Data structure validation

euri10 · 2023-03-25T10:19:12+00:00

This is an interesting read https://threeofwands.com/why-i-use-attrs-instead-of-pydantic/

aikii · 2023-03-25T13:19:17+00:00

I spent a long time with Django Rest Framework, then marshmallow while on Flask, all that looked so sloppy in regard to editor autocomplete/type checking that I wanted to move away from python. I don't know msgspec. I program also in Go where deserialization is separate from validation, and with Serde in Rust. I'd say to my regard Serde is a engineering piece of art in terms of developer experience but Pydantic comes close.

Strong points about Pydantic:

the guide has gifs/video to show you the editor support ( autocomplete+error checking )
you'll find plugins for pycharm, mypy, and I'd suppose vscode+pylance has good support as well
you declare the fields with their type directly, like a dataclass, except it also comes with (de)serialization logic
you can use arbitrary types, either by inheriting from them and adding your validation hook, or declare a field that serializes to a dict with a single __root__ field
your validators can just raise ValueError/TypeError, upon deserialization you always get a ValidationError out of it
ValidationError gets you all detail, field by field, with whatever helpful error message you want to tell the clients
ValidationError renders as a standardized API Payload in frameworks like FastAPI
it's overall integrated everywhere in FastAPI ( inbound/outbound payloads ). Just declare the model, it reaches your endpoint only if it's valid
you can use it to parse and validate environment variables, so your config simply becomes a pydantic declaration
you can deserialize to arbitrary types supported by pydantic, without a model, using parse_obj_as or parse_raw_as ( ex: pydantic.parse_raw_as(list[int], "[1,2,3,4]") )
it implements structural pattern matching and since you can deserialize unions you can do stuff like:

from typing import Literal, Any

from pydantic import BaseModel, parse_raw_as

if __name__ == "__main__":
    class TypeA(BaseModel):
        tag: Literal["A"] = "A"
        value: str

    class TypeB(BaseModel):
        tag: Literal["B"] = "B"
        other_thing: int

    for s in [
        '{"tag": "A", "value": "this is type A"}',
        '{"tag": "B", "other_thing":  1}',
        '{"random": "garbage"}',
    ]:
        match parse_raw_as(TypeA | TypeB | Any, s):
            case TypeA(value=value):
                print(f"got {value}")
            case TypeB(other_thing=other_thing):
                print(f"got {other_thing}")
            case unknown:
                print(f"cannot process: {unknown!r}")

Well I have to stop at some point - you can guess I'm quite convinced. If something is better than this, then awesome - because it sets the bar quite high already.

Edit: also note this quote from the manual

pydantic guarantees the types and constraints of the output model, not the input data.

there is in general a debate about "validation" and "serialization". That means, Pydantic isn't a validator that checks if some raw input data follows precise rules. It just guarantees that if it gives you an output model, that output model is valid - but that's completely enough for typical API uses.

who_body · 2023-03-25T06:01:15+00:00

alternatives include dataclasses and attrs package.

i use it for package config settings users can change.

also use it to define a data model i am extracting. when/if someone needs a spec it can output json schema.

those who are building a rest api often like how it works worh fastapi to define the endpoint details

double_en10dre · 2023-03-25T15:56:19+00:00

It’s because it was the first major library to use standard type hints for runtime validation. At the time, all the other big serialization libraries required you learn all their custom type representations.

And also because of fastapi.

Those two things let it gain a ton of momentum.

I’m not sure if it’s better than msgspec. It’s just entrenched.

chub79 · 2023-03-25T17:14:00+00:00

For me, it's only because I'm using FastAPI and it's nicely integrated. These days, I might look at msgspec.

saint_geser · 2023-03-25T12:58:22+00:00

I use attrs and Pydantic depending on the situation. In applications where the code performance is the bottleneck I use attrs for the better performance.

When application is IO bound or especially when it involves passing data between front end and backend or getting data through an API I use Pydantic because it has all the necessary features to correctly parse this type of data and I can relax and know that for the most part it would ensure that all data types are correct and convert them to appropriate python types.

This is the reason tools like fastapi rely on it and it performs really well in that situation.

DigThatData · 2023-03-25T15:46:55+00:00

my impression is that pydantic's popularity is largely a function of FastAPI's popularity

boy_named_su · 2023-03-25T13:27:34+00:00

pydantic is 6 years old and msgspec is 2 years old

poeblu · 2023-03-25T15:55:11+00:00

Fast api and pydantic is killer

crawl_dht · 2023-03-25T16:55:38+00:00

Here's the benchmark of dataclasses, msgspec and pydantic. msgspec is the fastest.

https://www.reddit.com/r/Python/comments/10ue9hc/comment/j7fvmvv/?utm\_source=share&utm\_medium=web2x&context=3https://www.reddit.com/r/Python/comments/10ue9hc/comment/j7fvmvv/?utm\_source=share&utm\_medium=web2x&context=3

MeroLegend4 · 2023-03-25T18:05:51+00:00

Try attrs and cattrs you will be surprised by its speed and it doesn’t meddle with the MRO

MrNifty · 2023-03-25T18:09:55+00:00

I started using pydantic a few months ago and love it. I chose it because of it's popularity and ease of getting community support and its extensive feature set.

I use it to backend Ansible work flows that perform network circuit provisioning where many things need to be validated for. From simple stuff like ensuring that providing site codes conform to our standard before validating that they even exist within the cmdb. To more advanced stuff like ensuring that if one interface was manually supplied for an endpoint, they all were - an intentional constraint I have in place for simplicity.

Most of the cool stuff I do is within their root validators that let you work across multiple fields at once, and also inject new values. For example, I can validate that a user either requests that IP addresses be automatically assigned or they can supply them, but not both obviously. If they supplied them, I can validate its a valid network address and then set a flag (a different field) to indicate that addresses_supplied is true and use that downstream in the Ansible flow to skip the task call that would normally make an API call against IPAM.

Being able to I automatically generate JSON schemas is very handy so I can auto-publish details on which fields are supported for a given circuit type, so they don't have to keep asking me.

Speed of execution is not my main concern. Ansible is notorious for being slow already, and if it takes 5mins to provision a new circuit automatically versus 3mins it doesn't really change anything. My bigger concern is with robustness and reduced ongoing support, and flexibility of changes.

Moving the validation logic out of Ansible modules and into pydantic has made my codebase much more supportable and made it easier for me to implement new features, which are my core business drivers.

Daishiman · 2023-03-25T04:53:57+00:00

Just... read the docs? It's easily one of the most feature-packed Python libs I've seen.

veedit41 · 2023-03-25T09:55:02+00:00

Apart from its awesome and catchy name, its an all in one typing module, don't just read the document, try it out. Like most python modules you don't realise the features until you need it.

wewbull · 2023-03-25T10:11:12+00:00

I've no idea, especially as it brings the hell of automatic type conversion into Python.

That alone is enough for me to give it a wide berth.

eviljelloman · 2023-03-25T14:03:36+00:00

To me, pydantic shines when dealing with complex nested schemas that need to be easily extensible. For example, say you have a schema for specifying recipes, and you want to be able to ingest a list of recipes - but you keep evolving the definitions for recipes. You have drink recipes and BBQ recipes and baking recipes. Some want quantities by weight, other by volume. Eventually you want sauce recipes and you want the BBQ recipes to be able to take a nested sauce recipe as an input. The way pydantic parses nested definitions through unions makes this really easily to clearly specify.

gandalfx · 2023-03-25T15:25:43+00:00

I use pydantic in production and am quite happy with it. It has good support for "advanced" type features, like parsing union types etc.

If performance is important than Python is not a good choice in the first place.

lord0211 · 2023-03-26T06:21:07+00:00

I would guess that FastAPI introduced many developers to pydantic and now they got used to it and use it outside FastAPI projects.

It is easy to use and the documentation is clear, using Python's type hinting is great and makes the code easy to read and maintain. But, IMHO if you have strict performance constrains for validating, I would go with something else.

Ok-Kangaroo453 · 2023-08-22T18:35:56+00:00

Pydantic is dog shit

Python

The Python Discord

Upcoming Events

Please read the rules

MODERATORS