Python — Decorators
This post covers Python decorators. Before reading, make sure you are comfortable with first-class functions and closures. If not, start with the posts below:
Python — First-Class Functions
Python — Closures
What is a decorator? In dictionary terms it means “one who decorates” or “interior designer.” Just as you decorate a room with nice wallpaper or curtains, in Python a decorator is a syntax that adds functionality to existing code. Many Python beginners struggle with decorators, but the concept isn’t actually that hard — don’t worry. Let’s learn how to use them through examples.
Create a Python file called decorator.py in any directory you like and save the code below.
def outer_function(msg):
def inner_function():
print(msg)
return inner_function
hi_func = outer_function('Hi')
bye_func = outer_function('Bye')
hi_func()
bye_func()After saving, open a terminal or command prompt, navigate to the directory, and run:
$ python decorator.py
Hi
ByeIf you read the previous lesson on closures, you should understand the code above. Decorator code looks very similar — the only twist is that a function is passed as an argument to another function. Let’s look at the simplest decorator example. Save and run the following code:
def decorator_function(original_function): # 1
def wrapper_function(): # 5
return original_function() # 7
return wrapper_function # 6
def display(): # 2
print('display function ran.') # 8
decorated_display = decorator_function(display) # 3
decorated_display() # 4$ python decorator.py
display function ran.Here’s what’s happening: at #1 and #2 we defined the decorator function decorator_function and a regular function display. Then at #3 we assigned the result of calling decorator_function(display) to the variable decorated_display. That return value is, of course, wrapper_function. At this point wrapper_function hasn’t run yet — it’s sitting in decorated_display waiting to be called. When we call decorated_display() at #4, the wrapper_function defined at #5 runs, which at #7 calls original_function (i.e., display), which finally executes print at #8 and outputs the string.
Phew… mouthful. So why do we use this seemingly complicated decorator pattern? As mentioned in the intro: it lets us add functionality to existing code without modifying it, by using a wrapper function. Let’s see:
def decorator_function(original_function):
def wrapper_function():
print('Before calling {}.'.format(original_function.__name__))
return original_function()
return wrapper_function
def display_1():
print('display_1 function ran.')
def display_2():
print('display_2 function ran.')
display_1 = decorator_function(display_1) # 1
display_2 = decorator_function(display_2) # 2
display_1()
print()
display_2()$ python decorator.py
Before calling display_1.
display_1 function ran.
Before calling display_2.
display_2 function ran.With one decorator function we added behavior to two functions, display_1 and display_2. In practice you usually don’t write display_1 = decorator_function(display_1) style — you use the simpler @ syntax with the decorator’s name. The previous code can be written more concisely as:
def decorator_function(original_function):
def wrapper_function():
print('Before calling {}.'.format(original_function.__name__))
return original_function()
return wrapper_function
@decorator_function # 1
def display_1():
print('display_1 function ran.')
@decorator_function # 2
def display_2():
print('display_2 function ran.')
# display_1 = decorator_function(display_1) # 3
# display_2 = decorator_function(display_2) # 4
display_1()
print()
display_2()$ python decorator.py
Before calling display_1.
display_1 function ran.
Before calling display_2.
display_2 function ran.Instead of #3 and #4, the @ syntax at #1 and #2 makes it cleaner.
But what if we want to decorate a function that takes arguments, like display_info below? Let’s modify and run:
def decorator_function(original_function):
def wrapper_function():
print('Before calling {}.'.format(original_function.__name__))
return original_function()
return wrapper_function
@decorator_function
def display():
print('display function ran.')
@decorator_function
def display_info(name, age):
print('display_info({}, {}) function ran.'.format(name, age))
display()
print()
display_info('John', 25)$ python decorator.py
Before calling display.
display function ran.
Traceback (most recent call last):
File "decorator.py", line 21, in <module>
display_info('John', 25)
TypeError: wrapper_function() takes 0 positional arguments but 2 were givenA TypeError saying wrapper_function takes 0 arguments but 2 were given. Fix by changing the code as follows:
def decorator_function(original_function):
def wrapper_function(*args, **kwargs): #1
print('Before calling {}.'.format(original_function.__name__))
return original_function(*args, **kwargs) #2
return wrapper_function
@decorator_function
def display():
print('display function ran.')
@decorator_function
def display_info(name, age):
print('display_info({}, {}) function ran.'.format(name, age))
display()
print()
display_info('John', 25)$ python decorator.py
Before calling display.
display function ran.
Before calling display_info.
display_info(John, 25) function ran.Adding arguments at #1 and #2 fixed the issue.
Decorators can also be defined as classes, not just functions. Take a look:
# def decorator_function(original_function):
# def wrapper_function(*args, **kwargs):
# print 'Before calling {}.'.format(original_function.__name__)
# return original_function(*args, **kwargs)
# return wrapper_function
class DecoratorClass: # 1
def __init__(self, original_function):
self.original_function = original_function
def __call__(self, *args, **kwargs):
print('Before calling {}.'.format(self.original_function.__name__))
return self.original_function(*args, **kwargs)
@DecoratorClass # 2
def display():
print('display function ran.')
@DecoratorClass # 3
def display_info(name, age):
print('display_info({}, {}) function ran.'.format(name, age))
display()
print()
display_info('John', 25)$ python decorator.py
Before calling display.
display function ran.
Before calling display_info.
display_info(John, 25) function ran.Defining DecoratorClass at #1 and using @DecoratorClass at #2 and #3 produces the same result as the function-based decorator. Class-based decorators aren’t used as often as function-based ones in practice — just be aware they exist.
Got a feel for what a decorator is? Many people understand the concept but don’t know how to use decorators in real projects. Let’s look at typical use cases.
Decorators are commonly used for logging, for checking a user’s login state and redirecting to the login page if not authenticated, and for benchmarking. Linux/Unix server admins often combine the date and time commands to measure how long a script takes:
$ date; time sleep 1; date
Thu Sep 8 00:13:28 JST 2016
real 0m1.007s
user 0m0.001s
sys 0m0.001s
Thu Sep 8 00:13:29 JST 2016We can build similar logging functionality with a decorator. Let’s try:
import datetime
import time
def my_logger(original_function):
import logging
filename = '{}.log'.format(original_function.__name__)
logging.basicConfig(handlers=[logging.FileHandler(filename, 'a', 'utf-8')],
level=logging.INFO)
def wrapper(*args, **kwargs):
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
logging.info('[{}] called with args - {}, kwargs - {}'.format(timestamp, args, kwargs))
return original_function(*args, **kwargs)
return wrapper
@my_logger
def display_info(name, age):
time.sleep(1)
print('display_info({}, {}) function ran.'.format(name, age))When you run this, the terminal prints the result and a log file display_info.log is created in the same directory as decorator.py. Run the program:
$ python decorator.py
display_info(John, 25) function ran.The log file contains:
INFO:root:[2021-10-03 21:13] called with args - ('John', 25), kwargs - {}Now let’s add another decorator to measure execution time:
import datetime
import time
def my_logger(original_function):
import logging
filename = '{}.log'.format(original_function.__name__)
logging.basicConfig(handlers=[logging.FileHandler(filename, 'a', 'utf-8')],
level=logging.INFO)
def wrapper(*args, **kwargs):
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
logging.info(
'[{}] called with args - {}, kwargs - {}'.format(timestamp, args, kwargs))
return original_function(*args, **kwargs)
return wrapper
def my_timer(original_function): # 1
import time
def wrapper(*args, **kwargs):
t1 = time.time()
result = original_function(*args, **kwargs)
t2 = time.time() - t1
print('{} ran in: {} sec'.format(original_function.__name__, t2))
return result
return wrapper
@my_timer # 2
def display_info(name, age):
time.sleep(1)
print('display_info({}, {}) function ran.'.format(name, age))
display_info('John', 25)At #1 we defined a new decorator my_timer and at #2 applied it to display_info. Run:
$ python decorator.py
display_info(John, 25) function ran.
display_info ran in: 1.00157594681 secIt took 1 second.
Now let’s stack both decorators, my_logger and my_timer:
import datetime
import time
def my_logger(original_function):
import logging
filename = '{}.log'.format(original_function.__name__)
logging.basicConfig(handlers=[logging.FileHandler(filename, 'a', 'utf-8')],
level=logging.INFO)
def wrapper(*args, **kwargs):
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
logging.info(
'[{}] called with args - {}, kwargs - {}'.format(timestamp, args, kwargs))
return original_function(*args, **kwargs)
return wrapper
def my_timer(original_function):
import time
def wrapper(*args, **kwargs):
t1 = time.time()
result = original_function(*args, **kwargs)
t2 = time.time() - t1
print('{} ran in: {} sec'.format(original_function.__name__, t2))
return result
return wrapper
@my_logger # 1
@my_timer # 2
def display_info(name, age):
time.sleep(1)
print('display_info({}, {}) function ran.'.format(name, age))
display_info('John', 25)$ python decorator.py
display_info(John, 25) function ran.
display_info ran in: 1.00419592857 secINFO:root:[2021-10-03 21:13] called with args - ('John', 25), kwargs - {}We stacked two decorators at #1 and #2. Terminal output looks normal, but nothing was logged to the expected log file. Instead a file called wrapper.log was created with the log entries:
INFO:root:[2021-10-03 21:17] called with args - ('John', 25), kwargs - {}Hmm — why? Let’s swap the decorator order:
from functools import wraps
import datetime
import time
def my_logger(original_function):
import logging
filename = '{}.log'.format(original_function.__name__)
logging.basicConfig(handlers=[logging.FileHandler(filename, 'a', 'utf-8')],
level=logging.INFO)
@wraps(original_function) # 1
def wrapper(*args, **kwargs):
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
logging.info(
'[{}] called with args - {}, kwargs - {}'.format(timestamp, args, kwargs))
return original_function(*args, **kwargs)
return wrapper
def my_timer(original_function):
import time
@wraps(original_function) # 2
def wrapper(*args, **kwargs):
t1 = time.time()
result = original_function(*args, **kwargs)
t2 = time.time() - t1
print('{} ran in: {} sec'.format(original_function.__name__, t2))
return result
return wrapper
@my_timer
@my_logger
def display_info(name, age):
time.sleep(1)
print('display_info({}, {}) function ran.'.format(name, age))
display_info('Jimmy', 30) # 3INFO:root:[2021-10-03 21:13] called with args - ('John', 25), kwargs - {}
INFO:root:[2021-10-03 21:20] called with args - ('John', 25), kwargs - {}Logging works this time! But the terminal output is a bit off:
$ python decorator.py
display_info(John, 25) function ran.
wrapper ran in: 1.0019299984 secwrapper was printed instead of display_info. The mystery deepens.
The reason is simple. When you stack multiple decorators, the bottom decorator runs first. In the previous code, my_logger was applied first (closest to the function), so by the time my_timer saw it, it was wrapping a function whose __name__ was already wrapper (the inner function returned by my_logger). Inside my_timer, original_function is in fact the wrapper function returned by my_logger.
To prevent this, the functools module provides a wraps decorator. Let’s use it:
from functools import wraps
import datetime
import time
def my_logger(original_function):
import logging
filename = '{}.log'.format(original_function.__name__)
logging.basicConfig(handlers=[logging.FileHandler(filename, 'a', 'utf-8')],
level=logging.INFO)
@wraps(original_function) # 1
def wrapper(*args, **kwargs):
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M')
logging.info(
'[{}] called with args - {}, kwargs - {}'.format(timestamp, args, kwargs))
return original_function(*args, **kwargs)
return wrapper
def my_timer(original_function):
import time
@wraps(original_function) # 2
def wrapper(*args, **kwargs):
t1 = time.time()
result = original_function(*args, **kwargs)
t2 = time.time() - t1
print('{} ran in: {} sec'.format(original_function.__name__, t2))
return result
return wrapper
@my_timer
@my_logger
def display_info(name, age):
time.sleep(1)
print('display_info({}, {}) function ran.'.format(name, age))
display_info('Jimmy', 30) # 3We applied wraps at #1 and #2 to both wrappers, and at #3 changed the name and age to verify it works correctly:
$ python decorator.py
display_info(Jimmy, 30) function ran.
display_info ran in: 1.013833999633789 secINFO:root:[2021-10-03 21:13] called with args - ('John', 25), kwargs - {}
INFO:root:[2021-10-03 21:20] called with args - ('John', 25), kwargs - {}
INFO:root:[2021-10-03 21:27] called with args - ('Jimmy', 30), kwargs - {}Both terminal and log file output look exactly as expected.
That covers the basics of decorators. Concepts can fade quickly without practice, so try applying them in real projects.
The next post covers Python’s generators.