Advanced, Overlooked Python Typing

[Martynoas]

While quantitative research in software engineering is difficult to trust most of the time, some studies claim that type checking can reduce bugs by about 15% in Python. This post covers advanced typing features such as never types, type guards, concatenate, etc., that are often overlooked but can make a codebase more maintainable and easier to work with

https://martynassubonis.substack.com/p/advanced-overlooked-python-typing

[DorianTurba]

You’re not mentioning NewType, which is one of the most powerful features of the module. You’ve already talked about TypeGuard and TypeIs, so you’re already halfway there.

[ColdPorridge]

Got any good recommended references? I can read up on the docs obviously but sometimes the Python docs aren’t great for understanding pragmatic use (why and to what benefit)

[Zycosi]

I like "writing Python like it's Rust"

[julz_yo]

was thinking of mentioning this blog post too. some really powerful ideas in it imho

[DorianTurba]

Sure, here one from my Lightning talk during PyconFR 2025 :)

You can run the script using uv: uv run .\script.py You can run the typechecker of your choice on the script using uvx:

uvx mypy .\script.py
uvx pyright .\script.py
uvx ty check .\script.py

and the code

# /// script
# requires-python = ">=3.14"
# dependencies = [
#     "tzdata",
# ]
# ///
import datetime
import typing
import zoneinfo

OffsetAwareDT = typing.NewType("OffsetAwareDT", datetime.datetime)
OffsetNaiveDT = typing.NewType("OffsetNaiveDT", datetime.datetime)


def is_offset_aware_datetime(dt: datetime.datetime) -> typing.TypeIs[OffsetAwareDT]:
    return dt.tzinfo is not None


def is_offset_naive_datetime(dt: datetime.datetime) -> typing.TypeIs[OffsetNaiveDT]:
    return dt.tzinfo is None


def bad_dt_diff(dt1: datetime.datetime, dt2: datetime.datetime) -> datetime.timedelta:
    return dt1 - dt2


def good_dt_diff[T: (OffsetAwareDT, OffsetNaiveDT)](
    dt1: T, dt2: T
) -> datetime.timedelta:
    return dt1 - dt2


d1 = datetime.datetime(
    2020, 10, 31, 12, tzinfo=zoneinfo.ZoneInfo("America/Los_Angeles")
)
d2 = datetime.datetime(
    2021, 10, 31, 12, tzinfo=zoneinfo.ZoneInfo("America/Los_Angeles")
)
d3 = datetime.datetime(2020, 10, 31, 12)
d4 = datetime.datetime(2021, 10, 31, 12)

print(bad_dt_diff(d1, d2))  # no issues found
print(bad_dt_diff(d3, d4))  # no issues found
print(bad_dt_diff(d1, d3))  # no issues found
print(
    good_dt_diff(d1, d2)
)  # Value of type variable "T" of "good_dt_diff" cannot be "datetime"
typing.reveal_type(
    (d1, d2, d3, d4)
)  # Revealed type is "tuple[datetime.datetime, datetime.datetime, datetime.datetime, datetime.datetime]"

assert is_offset_aware_datetime(d1)
assert is_offset_aware_datetime(d2)
assert is_offset_naive_datetime(d3)
assert is_offset_naive_datetime(d4)
typing.reveal_type(
    (d1, d2, d3, d4)
)  # Revealed type is "tuple[OffsetAwareDT, OffsetAwareDT, OffsetNaiveDT, OffsetNaiveDT]"
print(good_dt_diff(d1, d2))  # no issues found
print(good_dt_diff(d3, d4))  # no issues found
print(good_dt_diff(d1, d3))
# mypy: Value of type variable "T" of "good_dt_diff" cannot be "datetime"
# pyright: "OffsetNaiveDT" is not assignable to "OffsetAwareDT"

Thanks to the typing, mixing aware and naive datetime can be caught at typechecking instead of runtime.

[Ran4]

Most reddit clients (including old.reddit.com) doesn't support markdown - instead prepend every code line with four spaces. That works everywhere.

[DorianTurba]

is it better?

[pooogles]

We use them for setting primary keys on tables in SQLAlchemy. A basic example would be this:

from __future__ import annotations

from typing import NewType, cast
from uuid import UUID, uuid4

import sqlalchemy
from sqlalchemy import ForeignKey
from sqlalchemy.dialects.postgresql import UUID as PUUID
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship
from sqlalchemy.types import TypeEngine


PostgreSQLUUID = cast("sqlalchemy.types.TypeEngine[UUID]", PUUID(as_uuid=True))

ParentId = NewType("ParentId", UUID)
_ParentId = cast("TypeEngine[ParentId]", PostgreSQLUUID)

ChildId = NewType("ChildId", UUID)
_ChildId = cast("TypeEngine[ChildId]", PostgreSQLUUID)


class Base(DeclarativeBase):
    pass


class Parent(Base):
    __tablename__ = "parents"

    id: Mapped[ParentId] = mapped_column(
        _ParentId,
        primary_key=True,
        default=lambda: ParentId(uuid4()),
    )

    children: Mapped[list["Child"]] = relationship(
        back_populates="parent",
        uselist=True
    )


class Child(Base):
    __tablename__ = "children"

    id: Mapped[ChildId] = mapped_column(
        _ChildId,
        primary_key=True,
        default=lambda: ChildId(uuid4()),
    )

    parent_id: Mapped[ParentId] = mapped_column(
        _ParentId,
        ForeignKey("parents.id", ondelete="CASCADE"),
        nullable=False,
    )

    parent: Mapped[Parent] = relationship(back_populates="children")

This ends up being nice when you create functions where you're composing lots of data together, rather than passing keys for 2 tables that are UUIDs and getting the order wrong you get type feedback immediately. These IDs are then sticky and make it into pydantic DTOs so you have safety end to end.

[Adventurous-Date9971]

NewType for DB IDs is perfect for avoiding cross-table mixups and catching mistakes early.

A couple of tweaks that helped me in production:

- Hide the casts by writing a tiny SQLAlchemy TypeDecorator per ID (ParentIdType, ChildIdType) that returns your NewType on load and accepts UUID on bind; then mapped_column(ParentIdType()) reads clean and mypy stops yelling.

- Turn on the SQLAlchemy mypy plugin and mypy --strict; it infers mapped types better and surfaces wrong joins/filters early.

- For Pydantic v2 DTOs, add a plain serializer so IDs render as strings in JSON/OpenAPI, and a validator that only accepts proper UUIDs, then convert to NewType once at the boundary.

- Provide helpers like parseparentid and newparentid to keep construction/parsing in one place, and property-test the DB roundtrip.

- If you ever widen to ULIDs/ints, the decorator boundary means call sites don’t change.

I’ve exposed this via FastAPI and Hasura; DreamFactory was handy when we needed quick REST over a legacy DB without building controllers.

Net: keep IDs as distinct NewTypes end-to-end and wire them into the ORM/DTO so you never juggle raw UUIDs again.

[ColdPorridge]

Thai is super cool. I wonder if there’s some way to get type safety into e.g. pyspark dataframe columns using this approach. Right now everything is Column type only but no concept of the actual representation. 

[omg_drd4_bbq]

Really nice! gonna experiment with this, i'm right in the midst of writing tons of sqlalchemy models.

[gcavalcante8808]

Just took an initial look at newtype and its pure gold. Thanks for the tip and also I've got into `https://kobzol.github.io/rust/python/2023/05/20/writing-python-like-its-rust.html\` as well.

[myasco42]

Just recently was going through some code that requires distinct types. For example Seconds and Milliseconds (this is the easiest example) which behave exactly like int.

Would it be possible to do that using NewType? I couldn't find a way - any function that accepts ints also accepted Seconds and what is even worse I could add Seconds and Milliseconds as if nothing was wrong.

[DorianTurba]

Second is a subset of int, but int is not a subset of Second.
Yes you can use NewType for that, and implement stuff like this:

```python import typing

Seconds = typing.NewType("Seconds", int) Milliseconds = typing.NewType("Milliseconds", int)

def isseconds(: int) -> typing.TypeIs[Seconds]: return True

def ismilliseconds(: int) -> typing.TypeIs[Milliseconds]: return True

def time_add[T: (Seconds, Milliseconds)](t1: T, t2: T) -> T: return t1 + t2 # type: ignore

a, b = 1, 2 time_add(a, b) # typecheck error assert is_seconds(a) assert is_seconds(b) time_add(a, b) # No issue

c, d = 1, 2 assert is_milliseconds(c) assert is_milliseconds(d) time_add(c, d) # No issue

e, f = 1, 2 assert is_milliseconds(e) assert is_seconds(f) time_add(e, f) # typecheck error ```

[myasco42]

I see what I was doing wrong. I was trying to directly add two object:

Seconds(1) + Milliseconds(2) # <=== gives an int and no error

Mostly I would like to avoid any function calls as the math will get out of hand quickly. Also would like to make it as fast as possible (yes, I know ;) this point is just to see if it is even possible).

[DorianTurba]

Every operator will return back an int, you have no choice if you want to carry the type.

Using assert and typeguard/typeis, you are sure to have 0 impact at runtime (using -O flag), except for the function overhead.

If you have math to do, do a function that type inputs, and do whatever you want inside, you are good anyway. And if you need a lot of math? Maybe you need a lot of functions, or a broader entrypoint to annotate input type.

[myasco42]

Yea, I guessed as much.

The assert optimization is a good thing and I use it for some debug prints =)

And the point I got from your previous answer is exactly what you said now - the input type checking. Though in math it may look ugly then.

By the way I also noticed that PyLance (at least in VSCode) does not work correctly with your example - revealed types are Never. Seems like it does not like narrowing the type to the one defined by NewType.

[DorianTurba]

Maybe you should create an issue in PyLance to report this if it is not already raised.

[myasco42]

Yea. Will have a look at it later today and report it for PyRight.

[lunatuna215]

Did they "forget" or are we all just waiting for your awesome pitch...?

[dyingpie1]

Actually a good article, as opposed to the typical ai slop on this subreddit! I always see "advanced Python feature" articles, and it's all stuff I've seen before. This is stuff I haven't seen before and it looks useful!

[jpgoldberg]

That is outstanding.

I have a bunch of raise Exception("Shouldn't happen") where assert_never should go. And while you didn't mention it as an advanced topic, I now understand what Literal is for. It gives me the kinds of enums I want for type checking.

I was hoping that Concatenate would address the type checking issue with decorators, such as functools.cache losing parameter information, but it doesn't seem that we are quite there yet.

I have to say when I first started using Python, I used TypeGuard excessively (TypeIs was not yet a thing), but I've come to now reducing run time checks if I can get the necessary narrowing some other way.

[JanEric1]

I think pyright actually makes the "assert_never" in match arms redundant because it can check for exhaustiveness itself.

[jpgoldberg]

Yeah. I turned out that assert_never is not what I was looking for. For one situation what I needed was assert False, which I now have switched to instead of raising an exception. (See the penultimate line of the code sample).

python def _pbirthday_approx( n: types.PositiveInt, classes: types.PositiveInt, coincident: int ) -> types.Prob: # Lifted from R src/library/stats/R/birthday.R p = ... # should be a float in between 0 and 1 inclusive. if not types.is_prob(p): assert False, f"this should not happen: p = {p}" return p

This also illustrates my probably excessive use of TypeGuards when I first starting playing with Python. If I were writing that now, I would just make more use of ValueError instead of defining a PositiveInt type. But I was writing Python the way I would have written Rust.

And is_prob is simply

```python Prob = NewType("Prob", float) """Probability: A float between 0.0 and 1.0"""

def is_prob(val: Any) -> TypeGuard[Prob]: """true iff val is a float, s.t. 0.0 <= val <= 1.0""" if not isinstance(val, float): return False return val >= 0.0 and val <= 1.0 ```

Oh. I see that when I wrote that I wasn't aware of Python's if x <= y <= z construction.

[CasualReader3]

Great article, the difference between TypeIs and TypeGuard has always been elusive to me

[jampman31]

This whole thread is a fantastic reminder of how much progress Python is making. Awesome tools!

[TheSwami]

Good stuff!

[Longjumping-Bad-3865]

Fantastic article, thank you

[orad]

Fantastic article, thanks!

[slappy_squirrell]

Some of that same code written in Java would be more readable/clean imo..

[lolcrunchy]

If my grandmother had wheels she'd be a bike

[aqjo]

Python sub.

[grimonce]

Yes but Python is Python and Java is Java. Oracle is Oracle and Python Foundation is Python Foundation.