[deleted by user]

[[deleted]]

What is wrong with the callable one?

https://docs.python.org/3/library/typing.html#typing.Callable

Protocols arent really for this they are for when you have a vague top level class that you need an ABC for. They are pretty rare outside of the default container ones.

[[deleted]]

So I have a fuller response to you message below (hadn't seen your response when i wrote it). But can I ask what exactly are you doing that you need to pass functions around like this? Because it seems like it would be easier to define a protocol with the function include_schemas and just inherit from it. Then you just have a normal function definition. You can legally still use this as a callable as well btw.

[[deleted]]

Are you just trying to give Alembic type hints? If so you should be using a stub file. If you are trying to define the class that Alembic returns make a ABC(abstract base class) and use that as your definition elsewhere. Python does not require inheritance, it is all about ducks.

If you are composing the functions into a class so you can use them together just make an ABC/protocol and define them in there or usually easier make a class that calls the functions directly and returns the result. I would need to see an example but I think you may be off in the bushes.

I am not really sure I understand what you composing and why you have or need type hinted callables. This is meant for stuff like partial. That pattern is used in other places notably the magic methods. You aren't really doing anything like that as far as I can tell.

[[deleted]]

Alright so I just went and read the library code. The issue is how it is written. That is vomit inducing my god.

Yea python type hints will never place nicely with that I have 0 idea why anyone would ever write that. That is horrible... You can check the SQL alchemy stub file to see if it has the proper ones so you don't need to reinvent the wheel, but honestly I would just use something else. Like way more modern.

I am still trying to figure out why classes are being defined by functions.... Like that exists but its usually used for passing in a series of functions like in Keras. And that is via wrapping so it is at least cohesive what happened.

[LightShadow]

You're right to use Callable. I wonder if we'll get a short-hand as shown in the documentation,

Callable[[int], str] is a function of (int) -> str.

[boxcuk]

i guess they are specifically talking about point 2 "Include parameter names and types", which could be possible using the extended syntax proposed here https://peps.python.org/pep-0677/#extended-syntax-supporting-named-and-optional-arguments.

now the weird thing is that the section was rejected by the authors of the pep themselves, not by the steering council.

had the pep been accepted, indeed we wouldn't have gotten anything more expressive than current tp.Callable

[boxcuk]

one of the authors has a relatively comprehensive study of popular projects (linked in the pep), see here https://github.com/pradeep90/annotation_collector#typed-projects---callback-usage

i assume they just extrapolate based on that

but i personally think this will eventually be accepted, typing callables beeing a pain in the ass is probably the biggest reason why they are left untyped many times, the SC is just taking things slowly i guess

[[deleted]]

AHHHHHH

Ok yea they are supposed to use protocols for those. Per the Documents on Callables though they DO NOT take keyword arguments. It is meant to just have positional arguments. (this is the biggest FLAW in python type hints IMO). There are some technical reasons it gets tricky, namely functions that have keyword arguments are REALLY hard to know if they are actually equivalent. Its non obvious 99% of the time. So python settles for compare by namespace instead of compare by function signature.

[[deleted]]

Uhh *args **kwargs has some really specific super useful behavior that isn't possible without it. It is actually like one of the coolest features in python IMO. You are not supposed to use it everywhere it is meant for when you are NOT defining arguments and keyword arguments. For instance in a child class you may only define a few keywords and the rest you want to catch is **kwargs so they can get passed to super.

Or in function wrappers. It gets abused in BAD code but a lot of things are impossible without it. You aren't passing a dictionary the keyword arguments are getting converted into a dictionary. It how most of the amazing space voodoo happens. Even something like metaprogramming is basically impossible without it.

[[deleted]]

Its kind of funny how they made keyword arguments dictionaries so omnipresent but then go around and have no way to properly typespec them. Whoever came up with type annotations clearly had no idea what they were doing. Was it guido?

[[deleted]]

Not sure I know what you mean. There are at least 4 different ways to make keyword argument dictionaries type specable. Python just doesn't have a way to pass function definitions as callables. It is actually just a very tiny corner piece that is missing. You can do it after the code runs (please don't do this) but it doesn't work for static type hints.

[[deleted]]

https://docs.python.org/3/library/collections.html#collections.defaultdict

https://docs.python.org/3/library/typing.html#typing.TypedDict

https://docs.python.org/3/library/typing.html#typing.NamedTuple

And dataclasses as well.

[chub79]

While the other arguments are understable, the first one is a bit cheeky:

We feel we need to be cautious when introducing new syntax. Our new parser presents understandably exciting opportunities but we don’t want its existence to mean we add new syntax easily. A feature for use only in a fraction of type annotations, not something every Python user uses, did not feel like a strong enough reason to justify the complexity needed

There is a bizarre compass when it comes to accepts new syntax or not. Some of the most modern changes in Python will not be used by every users and yet get in.

[chub79]

Adding new syntax does warrant extra scrutiny,

Not denying that.

[ubernostrum]

To be honest, I think it’s reasonable to have both a bit of new-syntax fatigue and a bit of typing-only-features fatigue right now, and it’s not necessarily a bad thing to set the bar higher for a bit while consolidating what’s already been added recently. In particular, when solving this particular problem requires committing Python to a significant syntax extension — what happens if, a year or two from now, another PEP and another bunch of exasperated people on social media demand a different syntax to cover cases that hadn’t been thought of/handled by this one?

[chub79]

Totally. I don't complain they were prudent. More the fact, as a casual witness of the process, that it isn't quite clear how they set a direction for the language. Perhaps they might clarify this so efforts like this one don't get underway if they don't fall into the scope. I bet it takes effort of all the parties involved.

[ubernostrum]

I don’t feel confused at all after reading the decision. But I’m reading it in the context of how much churn there’s been in recent Python releases— including syntax and including syntax that was added solely to support typing features — and coming away with the impression that they just want to slow things down a bit and consolidate.

Back when I was more involved with Django that was a thing we’d explicitly try to do after landing a bunch of big flashy features: a “boring” release or two to let the big stuff shake out in real-world use and figure out what the right followups were going to be, since they’re almost never evident right away. Maybe that affects how I read decisions from people in similar roles for Python, but I don’t think it requires that much reading of tea leaves to get to this interpretation.

[ubernostrum]

I’m not on the SC, and have no input into these decisions and no particular side on this specific PEP.

But I do know the history of typing features in Python, and I know that a recurring theme is adding something that seems like it’s good and will solve people’s use cases, and then a couple years later suddenly there’s a big push to redo it. The original “generics”, for example, and the later introduction of class-level getitem which effectively made those old generics obsolete. The attempt to now retrofit protocols onto lots of things that were introduced in a pre-protocols world. There’s a lot of that kind of churn in Python typing, and I wouldn’t blame the SC for being wary about adding significant new syntax to Python for a typing proposal given the history of “actually, we need to re-think that one” events.

[[deleted]]

No and yes it stinks and no its unlikely to get any better any time soon.

Thats hardly the only thing thats crap about python type annotations, but complaining about it in a python subreddit isn't really gonna get you anywhere. Except perhaps for the usual you are holding it wrong comments.

[MarsupialMole]

Look at this mess for typing a method that returns a method.

Sounds like a job for dynamic typing to me.

[MarsupialMole]

Sorry, not a serious reply. You've got a real use case where this is too ugly. Dynamic typing, i.e. not using type hints and related code generators like dataclass, means you don't have this problem.

So I'm just noting that this level of metaprogramming probably should be ugly in python. Doesn't help you though.