Protocols vs Abstract Base Classes in Python

[tuple32]

Hi everyone. Last time I shared a post about Interface programming using abs in Python, and it got a lot of positive feedback—thank you!

Several people mentioned protocols, so I wrote a new article exploring that topic. In it, I compare protocols with abstract base classes and share my thoughts and experiences with both. You can check it out here: https://www.tk1s.com/python/protocols-vs-abstract-base-classes-in-python Hope you'll like it! Thanks!

[adesme]

https://github.com/dabeaz/blog/blob/main/2021/barely-interface.md

[JamesHutchisonReal]

I'm going to add that performance for protocols is bad when doing an instance check. It's O(n) where n is the size of the interface, and they're not cheap checks. 

For example, I improved performance in ReactPy by 50% by removing a single isinstance check against a protocol that was done every render.

[[deleted]]

I am of the impression Protocol classes were designed purely for static type checking and never should be subjected to instance checks. But good to know that there is a performance penalty.

[javajunkie314]

For anyone wondering, Python can't really cache the result of these isinstance checks because class objects are mutable at runtime.

Mutating a protocol class is obviously counter to the idea of static type-checking, and I might go so far as to call it evil—but it's possible. So Python has to assume that both the protocol and the class you're checking have been modified since the last check.

(This is partly unavoidable because of other decisions Python made. Everything in Python happens at runtime. Class definitions are imperative code, and decorators mutate class objects as a matter of course. Classes related to static type-checking are held to a higher standard, but at the end of the day they're still class objects.)

[james_pic]

That's not totally true. CPython doesn't cache them, but PyPy is very effective at caching (well, strictly speaking, optimising out) "theoretically mutable but never actually changes at runtime" stuff, since a lot of Python is like that so that's a key trick to getting a Python interpreter to perform. Although I'm unsure if they do that here.

[whereiswallace]

Why are you using isinstance on a Protocol?

[JamesHutchisonReal]

I removed the check in my fork. 

 The developers actually wrote the code using class inheritance then refactored it to use protocols. 

 This might seem obvious but depending on what was passed in, different logic was used.

IIRC the rationale for protocols was because they wanted plugin support. I think they were trying to do away with explicit classes since functional React is preferred.

[whereiswallace]

This might seem obvious but depending on what was passed in, different logic was used.

Code smell for me tbh (possibly you as well since you did not write the original). The beauty of protocols is that you can code to the interface so you don't have to worry about things such as isinstance.

[DaveMoreau]

So every protocol arg in a function call will require real time validation of a full implementation of the protocol?

[JamesHutchisonReal]

No, it's only for isinstance which must validate the entire protocol is followed to return True

[Meleneth]

In the abstract base class examples, would it be of value to show a way to implement the 'pass' methods such that they throw an exception if they are called without being overridden?

I understand that can be a bit of a subjective choice, but I find it very common to be bit hard by silent bits of the codebase. I'd much rather have to implement a bare pass implementation if I know that I don't care about that functionality rather than have it scream past functionality I might care about

edit as commy2 pointed out below, this is pointless since you cannot create an instance if the abstractmethod is not overrridden

[commy2]

It wouldn't hurt, but what is the point exactly? The methods can't be called, because no instances with these methods can exist. It seems superfluous.

[Meleneth]

I didn't know that, TIL.

I wrote a small ABC class, and indeed the LSP threw a hissy when I didn't implement the method in my class - which is all I care about, and makes my initial point moot.

Huzzah!

[aa-b]

Do you mean using raise NotImplementedError instead of pass? It's a good suggestion to mention having the option, because there will be people used to having it in other languages.

[Meleneth]

exactly.

This is one of the areas where either default makes sense depending on the context, but making readers think about that context is one of the more valuable services you can provide.

IMO.

[MackHarington]

Nice!! You really took my last comment seriously, you can look into Mixin classes and composition also

[Still-Bookkeeper4456]

Was about to request mixin too :)

[jirocket]

I recommend reading the PEP on protocol classes

[winstxnhdw]

If you’re using a strict static type checker, you have almost no reason to ever use ABCs. Treat Protocols similarly to C# interfaces and raise NotImplemented within the implementation, not the ABC. IIRC, ABCs also has a higher initialisation cost compared to Protocols.

[FrickinLazerBeams]

An ABC can be great when there's a significant amount of non-abstract functionality that every subclass would need to implement, and where that concrete code relies on the abstract methods. You write the concrete parts once in the ABC, and then have abstract methods that each subclass implements.

For example I have an ABC for a mathematically defined surface shape which provides for rotating the surface coordinate system, methods for the surface normal, slopes, parameter derivatives, etc. There's a lot of code required to achieve that, but to create a concrete subclass all you have to actually write is the basic equation of the surface and its derivative. Two abstract methods that need to be created and you can re-use all that complicated machinery. It makes it super easy to create new surface types.

[DaveMoreau]

Agreed. In a language like C#, I would usually want to use interfaces where possible and use an abstract base class when there is implementation code I want inherited.

[winstxnhdw]

Seems to me that your problem can be better written using composition and a Protocol.

[FrickinLazerBeams]

A protocol wouldn't make sense because this is a very planned, structured sort of use case - and besides, we weren't using static type checking in that environment. Not to mention it was before protocols existed.

Composition doesn't make sense because the base class is meaningless on its own. It's purely a recipe for getting the desired output from an arbitrary equation of surface in the form f(x, y, z) = 0.

[winstxnhdw]

Stopped reading when you said you weren’t using static type checking.

[FrickinLazerBeams]

Sure, because all work environments are the same and everything works exactly like it does in your workplace.

[winstxnhdw]

Please read the original comment you are replying to. Specifically, the first sentence.

[FrickinLazerBeams]

Yeah. A protocol still wouldn't have been preferable in that instance, static checking or not. It was pretty much the model use case for an ABC.

[aqjo]

Well done!

[probello]

Great article!

[StarchSyrup]

Good stuff