Modern Python Advanced #6: Advanced typing — Variance, ParamSpec, Self, overload

In Intermediate #2 typing in earnest we covered Generic, Protocol, TypedDict, and Literal. This post is the next step — the harder parts of the typing system: variance, ParamSpec/Concatenate, Self, TypeGuard/TypeIs, and @overload.

This is mostly the library author’s territory and isn’t required for everyday code. But if you read library code or need to write precise types, you should know it.

Variance — covariance/contravariance #

The most confusing part first. Is list[Cat] a subtype of list[Animal]?

Intuition says “Cat is an Animal, so list[Cat] is a list[Animal]” — wrong. Here’s why.

def add_dog(animals: list[Animal]) -> None:
    animals.append(Dog())

cats: list[Cat] = [Cat(), Cat()]
add_dog(cats)    # ✗ if allowed, a Dog ends up in cats

list[Cat] is not list[Animal]. Because both reading and writing happen (mutable), neither direction is safe.

Three kinds of variance #

Intuition #

• Read only → covariant. Iterable[Cat] can be used as Iterable[Animal] (a Cat read out is an Animal)
• Read and write → invariant. list[Cat], dict[K, V]
• Argument slot → contravariant. Putting “a function that takes Cat” into “a function that takes Animal” is dangerous (a Dog might come; the Animal-taking slot must accept it)

From the function-type perspective #

def feed_animal(a: Animal): ...
def feed_cat(c: Cat): ...

# Can we put 'a function that only takes Cat' into 'a function-that-takes-Animal' slot?
fn: Callable[[Animal], None] = feed_cat   # ✗ — breaks if Dog comes
# the other way?
fn2: Callable[[Cat], None] = feed_animal  # ✅ — Cat is an Animal, OK

A function’s argument slot is contravariant. The return slot is covariant.

How is variance specified? #

The new PEP 695 syntax def fn[T]: from Intermediate #2 is inferred automatically. The old style (TypeVar) was explicit.

from typing import TypeVar

T_co = TypeVar("T_co", covariant=True)
T_contra = TypeVar("T_contra", contravariant=True)

class Producer(Generic[T_co]):
    def get(self) -> T_co: ...

class Consumer(Generic[T_contra]):
    def put(self, item: T_contra) -> None: ...

In new code you usually don’t write it explicitly — tools handle it. As a library author, just be aware of the variance of standard ABCs like Iterable and Callable when using them. That’s the everyday case.

ParamSpec and Concatenate #

Briefly seen in Intermediate #5 for preserving decorator signatures. Going deeper.

What ParamSpec really is #

def log[**P, R](fn: Callable[P, R]) -> Callable[P, R]:
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        ...
    return wrapper

P isn’t a type parameter — it’s the parameter signature itself. It expresses one bundle of (positional argument types + keyword argument types).

Concatenate — adding arguments #

When the decorator wants to add an argument.

from typing import Concatenate, Callable

def with_logger[**P, R](
    fn: Callable[Concatenate[Logger, P], R]
) -> Callable[P, R]:
    logger = make_logger()
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        return fn(logger, *args, **kwargs)
    return wrapper

@with_logger
def do_work(logger: Logger, x: int, y: int) -> int:
    logger.info(f"{x} + {y}")
    return x + y

do_work(2, 3)   # logger is auto-injected

Meaning of Concatenate[Logger, P]: “first argument is Logger, then P as-is.” After decoration, the signature drops the logger argument.

A pattern where the decorator removes an argument so callers don’t write it — a form of dependency injection.

Self — the calling class as a method’s return #

Briefly seen in Intermediate #2.

from typing import Self

class Builder:
    def add(self, item: str) -> Self:
        ...
        return self

class SubBuilder(Builder):
    def special(self) -> Self:
        ...
        return self

b = SubBuilder().add("x").special()
# b's type is correctly inferred as SubBuilder

Using -> "Builder" (a forward reference) makes the return type Builder even when called from SubBuilder, blocking .special(). Self solves this.

Class methods too #

class Item:
    @classmethod
    def from_dict(cls, data: dict) -> Self:
        return cls(**data)

Calling from a subclass returns the precise subclass type.

@overload — same function, different signatures #

A tool to give precise types to a function whose return type changes by argument.

from typing import overload

@overload
def parse(value: str) -> str: ...
@overload
def parse(value: int) -> int: ...
@overload
def parse(value: list[str]) -> list[str]: ...

def parse(value):
    # the actual implementation — under the overloads
    return value

a = parse("hello")        # str
b = parse(42)              # int
c = parse(["a", "b"])      # list[str]

Definitions decorated with @overload are for type checking only. The body is usually .... Below them comes one actual body.

When to use? #

• Return type changes with argument type
• Branching on Literal arguments

@overload
def get(key: Literal["count"]) -> int: ...
@overload
def get(key: Literal["name"]) -> str: ...
@overload
def get(key: str) -> object: ...

def get(key): ...

get("count") is precisely inferred as receiving int.

Standard functions like dict.get are defined this way #

@overload
def get(self, key: K) -> V | None: ...
@overload
def get(self, key: K, default: V) -> V: ...
@overload
def get(self, key: K, default: T) -> V | T: ...

d.get("k") → V | None, d.get("k", 0) → int (the type of 0). The same call, inferred precisely by argument.

TypeGuard and TypeIs — type-narrowing functions #

Tools for building user-defined type-narrowing functions beyond isinstance.

TypeGuard (3.10+) #

from typing import TypeGuard

def is_str_list(items: list[object]) -> TypeGuard[list[str]]:
    return all(isinstance(x, str) for x in items)

def process(data: list[object]) -> None:
    if is_str_list(data):
        # data narrowed to list[str] here
        print(", ".join(data))

A function returning TypeGuard[T] makes the promise the argument is T when True.

TypeIs (3.13+) — TypeGuard’s improvement #

from typing import TypeIs

def is_str(x: object) -> TypeIs[str]:
    return isinstance(x, str)

def handle(x: int | str) -> None:
    if is_str(x):
        print(len(x))     # str
    else:
        print(x + 1)      # int  ← this is the difference!

Difference: TypeIs also narrows the False branch. TypeGuard narrows only on True. TypeIs is more intuitive and safer in many places.

TypeGuard remains for compatibility; new code should use TypeIs (3.13+).

Annotated — metadata on types #

from typing import Annotated

UserId = Annotated[int, "UserID — DB 의 users.id"]
Email = Annotated[str, "이메일 형식"]

def find(id: UserId, email: Email): ...

Type checkers ignore the extra metadata in Annotated[T, ...]. But libraries can read it at runtime to drive behavior. FastAPI’s Depends and Query are the most famous examples.

from typing import Annotated
from fastapi import Depends, Query

def get_db(): ...

def search(
    q: Annotated[str, Query(min_length=3)],
    db: Annotated[Database, Depends(get_db)],
): ...

The types themselves stay str and Database, with extra behavior specs bundled in.

LiteralString — defending against SQL injection (3.11+) #

from typing import LiteralString

def execute(query: LiteralString) -> list[Row]:
    ...

execute("SELECT * FROM users WHERE id = 1")    # OK
execute(f"SELECT * FROM users WHERE id = {user_input}")  # ✗ — f-string result isn't LiteralString

LiteralString accepts only strings known at compile time. Strings built from user input (f"...", + user_id, etc.) are rejected. A tool for statically guaranteeing SQL injection defense.

NewType — same shape, different type #

from typing import NewType

UserId = NewType("UserId", int)
ProductId = NewType("ProductId", int)

def get_user(id: UserId): ...

uid = UserId(123)
pid = ProductId(456)

get_user(uid)    # OK
get_user(pid)    # ✗ — treated as a different type
get_user(123)    # ✗ — raw int can't pass

NewType is just int at runtime. Only the type checker treats it as a different type. Powerful where the same int has different meanings (UserId vs ProductId, USD vs KRW).

Difference from type aliases #

The type alias (type UserId = int) from Basics #2 is just an alias — UserId and int are interchangeable both ways. NewType is one-way: int → UserId requires explicit cast; not automatic.

type alias is just a different name; NewType separates into a different type.

Generic class — deeper #

class Stack[T]: from Intermediate #2. Going further:

Multiple parameters #

class Cache[K, V]:
    def __init__(self):
        self._data: dict[K, V] = {}

    def get(self, key: K) -> V | None: ...
    def put(self, key: K, value: V) -> None: ...

cache: Cache[str, int] = Cache()

Constraints and bounds #

class SortedList[T: (int, str)]:    # int or str only
    ...

class Container[T: Comparable]:     # only subtypes of Comparable
    ...

Variable-length — TypeVarTuple (3.11+) #

def stack[*Ts](*args: *Ts) -> tuple[*Ts]:
    return args

t = stack(1, "hello", 3.14)
# t's type: tuple[int, str, float]

A variable number of type parameters — used in libraries like numpy to express dimensions as types.

cast, assert_type, assert_never #

cast — force a type #

from typing import cast

raw: object = get_data()
data = cast(dict[str, int], raw)
# at runtime it's still raw; only the type checker sees dict[str, int]

Last resort when you know but the checker can’t infer. Misuse can cause runtime issues, so add a comment with the reason.

assert_type — verify inference #

from typing import assert_type

x = some_function()
assert_type(x, int)    # error if checker doesn't infer x as int

A declaration to the checker that “this point must be exactly this type.” Useful in library tests.

assert_never — guarantee all cases handled #

from typing import assert_never

def handle(event: Literal["click", "key"]):
    if event == "click": ...
    elif event == "key": ...
    else:
        assert_never(event)    # if a new case is added, this breaks

The checker verifies that every union case is handled. Really useful in discriminated union branches.

Wrap-up #

What this post covered:

• Variance — invariant (list), covariant (Iterable, read), contravariant (Callable arg)
• ParamSpec + Concatenate — preserve decorator signatures, inject arguments
• Self — the calling class as a method’s return; builder/alternative constructor
• @overload — precise signatures when return type depends on arguments
• TypeGuard vs TypeIs (3.13+) — user-defined narrowing; TypeIs is the new standard
• Annotated — metadata on types; the core of FastAPI/Pydantic
• LiteralString (3.11+) — compile-time string enforcement; SQL injection defense
• NewType — same shape, different type (UserId vs int)
• Variable params TypeVarTuple (3.11+)
• cast / assert_type / assert_never — explicit assertions

In the next post (#7 Performance — profiling) — the last in the Advanced series — we cover tools for finding and fixing slow code: cProfile, py-spy, line_profiler, and memory profiling.