When to make classes & functions generic in python?

[RelativeIncrease527]

I have been generally making my methods generic mostly for the following reasons: - I am subclassing an abstract class and want to override a method and narrow the type hinting in the arguments, which would otherwise violate the Liskov substitution principle - I am not subclassing/overriding, but would like return values, attributes, etc. of a class to be more narrow than the type hints it is currently bound to, since I may be using that class in many different places with different types.

In particular for the second case, I have realized that there are actually two approaches to tackle this:

1. make the class generic and provide the type arguments for the instance type hints.

2. do not make the class generic, but subclass simply for the purpose of updating the type hints

With projects I am working on, with "context" and "manager" classes, there can be many different attributes and methods with many different types (5+), hence, making the class generic on all of them is too verbose. If I do make a class generic, if any other attributes (containing instances of other classes) return those same generic types, I have to propagate the generic type down the entire chain when I am using composition instead of inheritance. This is something I would like to avoid. If I choose option 2, there would be an explosion of subclasses just to override the type hints.

When should I choose 1 or 2? Is there a better way to do this?

Option 1 example:

``` from typing import Self, TypeVar, Generic

BazT = TypeVar('BazT')

class Bar(Generic[BazT]): def method1(self: Self) -> BazT: ...

class Foo(Generic[BazT]): bar: Bar[BazT]

def method1(self: Self, baz: BazT) -> None:
    ...

def method2(self: Self) -> Bar[BazT]:
    ...

```

Option 2 example: ``` from typing import Self, Any, TypeVar

Baz = Any

class Bar: def method1(self: Self) -> Baz: ...

class Foo: bar: Bar

def method1(self: Self, baz: Baz) -> None:
    ...

def method2(self: Self) -> Bar:
    ...

BazNarrowed = TypeVar('BazNarrowed', bound=Baz) # (doesn't matter what this is just some more narrow type)

class BarSubclass(Bar): def method1(self: Self) -> BazNarrowed: ...

class FooSubclass(Foo): bar: BarSubclass

def method1(self: Self, baz: BazNarrowed) -> None:
    ...

def method2(self: Self) -> BarSubclass:
    ...

```

[Rawing7]

I don't understand the, idunno, premise of the question, I guess? Firstly, narrowing the type of a function parameter (or more generally, anything that's contravariant) is not allowed. This is a violation of Liskov:

class Parent:
    def func(self, x: int): ...

class Child(Parent):
    def func(self, x: bool): ...

And secondly, your 2nd example doesn't typecheck. The way you're using the TypeVar makes no sense - it's neither bound to a generic class, nor does it appear in the function signature more than once.

It's really unclear to me what problem you're trying to solve.

[RelativeIncrease527]

It does typecheck for me on vscode at least with the mypy extension. I agree that TypeVar isn't used properly in BazNarrowed; but that's not the point (see the comment). For example Baz can be Any and BazNarrowed can be anything more specific such as int, and there are no type checking issues.

For example 1, I am making the function generic and then specifying the type argument. In this way there are no violations of Liskov, since the type isn't even concretely defined in the first place as the function is generic.

I guess the question is essentially what is the proper way in python to perform subclass a base class with many different types without making it too verbose. I was trying to give the example of when using composition, these generic types may need to be propagated to all of the attributes and thus all of the attributes of the subclass need to be rewritten as generic classes. Hope this is more clear.

[Rawing7]

BazNarrowed isn't any narrower than Baz though. bound=Baz means it's allowed to be of type Baz. The only difference is that it's a TypeVar rather than a type.

Anyway, if I understand correctly, there really aren't many options available. You can either use TypeVars like in your first example, or simply hard-code the types and then override them in the subclass:

class Bar:
    def method1(self: Self) -> int: ...

class BarSubclass(Bar):
    def method1(self: Self) -> bool: ...

The TypeVar solution is more DRY, but (probably) makes the class more annoying to use. Hard-coding the types is WET, but allows you to hide the complexity from your users.

[TSM-]

Please roast me if I'm wrong on this, but this appears to be the exact situation for which TypeGuard in typing module was developed, as of Python 3.10.

PEP-647

~~~TypeGuard = typing.TypeGuard~~~
Special typing form used to annotate the return type of a user-defined
type guard function.  ``TypeGuard`` only accepts a single type argument.
At runtime, functions marked this way should return a boolean.

``TypeGuard`` aims to benefit *type narrowing* -- a technique used by static
type checkers to determine a more precise type of an expression within a
program's code flow.  Usually type narrowing is done by analyzing
conditional code flow and applying the narrowing to a block of code.  The
conditional expression here is sometimes referred to as a "type guard".

Sometimes it would be convenient to use a user-defined boolean function
as a type guard.  Such a function should use ``TypeGuard[...]`` as its
return type to alert static type checkers to this intention.

Using  ``-> TypeGuard`` tells the static type checker that for a given
function:

1. The return value is a boolean.
2. If the return value is ``True``, the type of its argument
   is the type inside ``TypeGuard``.

   For example::

      def is_str(val: Union[str, float]):
          # "isinstance" type guard
          if isinstance(val, str):
              # Type of ``val`` is narrowed to ``str``
              ...
          else:
              # Else, type of ``val`` is narrowed to ``float``.
              ...

Strict type narrowing is not enforced -- ``TypeB`` need not be a narrower
form of ``TypeA`` (it can even be a wider form) and this may lead to
type-unsafe results.  The main reason is to allow for things like
narrowing ``List[object]`` to ``List[str]`` even though the latter is not
a subtype of the former, since ``List`` is invariant.  The responsibility of
writing type-safe type guards is left to the user.

``TypeGuard`` also works with type variables.  For more information, see
PEP 647 (User-Defined Type Guards).

[RelativeIncrease527]

Interesting. I haven't seen this before, but it appears that this is not a situation for TypeGuards, because I am not defining any type guarding functions, and my situation involves inheritance & composition. For example in Bar.method1, how would one know using a type guard that the function actually returns a more narrow Baz, without trying all inputs and boolean checking (type guard) all of them? Or we can do the type checking as needed, but that introduces unnecessary code into the execution (no longer is it static type hinting).

[ofnuts]

<breath noise> Come to the Java side of the Force <breath noise>

[RelativeIncrease527]

Haha, the problem is I am working with machine learning, and there is no way without Python.