A type-hint is a kind of comment in Python, which is used by developers to indicate which class they expect certain objects to be instances of.
A simple example can be:
a: int = 5
b: int = 3
c: float = a/b
However, there are a few more subtleties to it.
1 – Type-hints do nothing
From a practical sense, the type-hint is, well, a hint, and is not enforced in any way.
For example, the following code is perfectly legitimate, and works with no problems:
def f(a: int) -> list:
return a*5
print( f("abc") )
# abcabcabcabcabc
This is an example of a function which expects an int, and tells us that it will return a list, but actually, it can get a string and return a string.
Why does it work?
Because, as the title hinted us, type-hints do nothing. They are ignored by Python.
Can we prove it?
We can prove it like so:
# When Python first sees a line of code, it parses it into an AST.
# In the AST, we do see that there is an `annotation`.
>>> print( ast.dump( ast.parse("x: int = 'a'"), indent=2) )
Module(
body=[
AnnAssign(
target=Name(id='x', ctx=Store()),
annotation=Name(id='int', ctx=Load()),
value=Constant(value='a'),
simple=1)],
type_ignores=[])
# If we compile the line of code
# (meaning that Python will convert it into an AST,
# and will then convert it into byte-code),
# then we still see the annotation
>>> dis.dis(compile("x: int = 'a'", "file_name", "exec"))
0 0 RESUME 0
1 2 SETUP_ANNOTATIONS
4 LOAD_CONST 0 ('a')
6 STORE_NAME 0 (x)
8 LOAD_NAME 1 (int)
10 LOAD_NAME 2 (__annotations__)
12 LOAD_CONST 1 ('x')
14 STORE_SUBSCR
18 LOAD_CONST 2 (None)
20 RETURN_VALUE
# However, once we place the code in a function,
# we see that the annotation is gone
>>> def f():
... x: int = 'a'
>>> dis.dis(f)
1 0 RESUME 0
2 2 LOAD_CONST 1 ('a')
4 STORE_FAST 0 (x)
6 LOAD_CONST 0 (None)
8 RETURN_VALUE
What’s that opcode?
What’s that magical STORE_SUBSCR ( __annotations__ ) thing?
Well, it seems that it adds annotations to variables defined in the interpreter:
Why is it ignored?
It may seem a bit weird at first that the type hints are somewhat available internally (in the AST, and in code-run-by-the-interpreter-and-not-inside-some-function), while being invisible (not showing in the function), and ignored (there’s no enforcement that x will be int).
But there’s some logic behind that.
Python, as a language, has every object (in the C level) inherit from the same type – everything is PyObject*.
For example, here’s: PyObject_SetItem:
/* o[key]=v. */
PyAPI_FUNC(int) PyObject_SetItem(PyObject *o, PyObject *key, PyObject *v);
This function signature cannot tell the type of the objects coming in. It simply receives PyObject*.
If the function really wants to know the type of the object, then it can do the C equivalent ofif isinstance(obj, typ): ....
And that’s by design.
However, it is sometimes not totally ignored
When assigning type hints to variables, like shown above, the type-hint is totally ignored by Python.
However, when assigning type-hints to functions, or classes, the type hints are stored somewhere.
>>> def f(a: int, b: float) -> str:
... return a+b
>>> f.__annotations__
{'a': int, 'b': float, 'return': str}
likewise, for classes:
>>> class A:
... a: int
... b: str = 3
>>> A.__annotations__
{'a': int, 'b': str}
>>> a = A()
>>> a.__annotations__
{'a': int, 'b': str}
Other than storing the type-hints in the __annotations__ dict, they are ignored.
While mostly ignored, there are some things that use type-hints
Who’s using type-hints?
There are 2 obvious answers:
The first, and the most important, is: programmers.
Type hints are a way of adding comments to the code, and explaining the code in a short, concise, and readable way. They are great.
(One such example I personally like is when working with physical calculations that have units.
Writing stuff like: earth_radius: KiloMeter = ... or rotation: Radians = np.pi)
The second is: static code analysis.
The most common 2 are PyCharm, and mypy.
Yet there are more uses to type hints. famous examples are pydantic and dataclasses.
How’s the dataclasses module use __annotations__?
The first thing CTRL+F found is the following from the _process_class function:
Next, it sets up all the annotations into the fields property:
If you’re curious about the fields property, then here it is:_FIELDS = '__dataclass_fields__'setattr(cls, _FIELDS, fields)
And, lastly, the dataclass creates the __init__ function:
Now, what peaked my interest here is that there’s an option to choose the name of the self argument. And it’s passed as a string.
Answering how it works will also tell us how dataclasses dynamically create a function.
Onto _init_fn
Basically, the whole function enters a single screenshot, and is pretty self explanatory
The interesting thing is that it generates strings!
As a consequence, we can pretty much predict what _create_fn does:
Oh my! an exec in the wild!
Basically, it is merely an automation to writing self.%s = %s
which is, fairly, exactly what’s promised.
Yet there’s a nice trick they’re doing – they create a function inside a function.
Line 428 creates our __init__ function
and line 431 creates a different function, whose output is our __init__ function.
This is done so that the newly created class will have access to the local scope (hence passing locals)
Note that it also has access to the same global scope, as passed in exec.
How’s the pydantic module use __annotations__?
When pydantic creates a new model object, it calls a function called collect_model_fields.
The fields are exactly the annotations of that class:
Another place where pydantic uses annotations is when generating a scheme:
Other worthy mentions
There’s functools.wraps, that now has __annotations__ in its list-of-properties-to-copy-from-wrapped-to-wrapper
There’s inspect, which exposes get_annotations.
It also has _signature_is_functionlike, which checks that obj.__annotations__ is either dict or None.
(for functions, it’s always dict or None. But for classes that behave like functions, we can alter that)
There’s typing, (duh).
It exposes a function called get_type_hints, which not only returns the annotations dict, but also handles strings as annotations.
We’ll talk about strings as annotations later, but I’ll just point out here thatdef f(a: int)
anddef f(a: "int")
have the same meaning to us, as developers, and also for static analysis tools.
But, to Python, the former is a dict (with one key: "a") and a value of the type int itself, whereas the latter has a string as the value.
Thus, the get_type_hints function handles that:
How does this function work?
A simple exception reveals it all:
typing also exposes a version of NamedTuple with annotations (that is, the programmer has to define a class, inherit from NamedTuple, and put the type hints him/herself.)
All typing does is define the class, and makes it behave like collections.namedtuple,
as well as adding the following magic (if you’re not familiar with metaclasses, I can suggest my own posts about it. Otherwise, the following code will be pure magic, and can be ignored)
How’s __annotations__ implemented?
In fact, there’s not much to it. The implementation is rather simple.
For functions, for example, there are but a few results, non of which reveal something interesting.
Modules appear to have annotations.
That’s just like the example we had above, where a variable we defined in the interpreter created the global variable __annotations__.
Likewise, defining variables in the module’s scope (i.e. not inside functions/classes) will create an __annotations__ variable for the module.
The place where annotations are created is in the byte code.
Creating annotations for functions
In order to generate byte-code that creates annotations, let us define a function, f, whose code creates a function (g) with annotations:
def f():
def g(a: int):
return 1
return g
Looking at the byte code, we see:
Starting from MAKE_FUNCTION, we see the following implementation:
Basically, it takes the top item, that’s code object g (at 18) – this will be the codeobj.
Then, it takes the next top item – this will be the annotations.
What’s that next-top-item?
We see that load a ; load int ; build tuple does exactly that – it builds a tuple (which will be converted to dict) that has the annotations.
Neat.
Side note: Python versions
The above byte-code is generated in Python 3.11.8.
For Python 3.13, we see a slightly different byte code:
Simply put, this splits MAKE_FUNCTION to the actual making of the function, and to setting its attributes. There are minor changes in the implementation, but the main point is that the opcode has been split into two.
Creating annotations for classes
The byte code for creating classes is a bit more complicated (compared to load const codeobj ; make function).
However, we’re going to see that the part relevant to us – creating class annotations – is composed of 2 parts:
first, create the annotations property (this will be done by SETUP_ANNOTATIONS)
second, populate the dictionary with values.
Let’s see it in action:
The second step is in front of us – the STORE_SUBSCR does __annotations__['a'] = int
The first step is inside the SETUP_ANNOTATIONS opcode, which simply creates an empty dict
from __future__ import annotations
In [another post], I expanded more on how __future__ imports work.
I’ll just mention here that using from __future__ import annotations changes the compiler’s behavior so that annotations won’t be parsed to the objects they point to, but rather to the actual string that’s written.
In other words, def f(a: int) wont have the object int as the annotation, rather the string "int".
We can see it in the following code:
When the future feature is turned on, the annotation value (string) is used.
When turned off, the value it points to is used
Summary
It’s that time again – the end of a post.
Is this post too long? too short? too detailed? I’m not sure, but I hope that it was right for you 🙂
In this post, we started looking at Python’s type hints.
At the syntax, and at the no-enforcement of it.
At the use cases, such as dataclasses and pydantic.
And at the implementation of it.
In the next post, we’re going to look at some special types that were created specifically for type hints.
See you next time 🙂
Harpaz's blog 