Python Basics #20 — Modules and Packages Vol. 1

Programming Language Python
6 min read

In this lesson we’ll cover modules and packages.

A lot of people think modules and packages are hard concepts, but they really aren’t. If that’s been your impression, this lesson will clear up both what they are and how to use them.

What does “module” mean in the dictionary?

  1. A module — an instructional unit (especially in UK universities, a unit forming part of a course of study).
  2. A computer module — a unit of a computer system or program with a specific function.
  3. A module — a standardized component used to make machinery, furniture, buildings, etc.

Different domains, but all describe a small part of a larger system or a small component within it. In programming, a module has the same meaning: it represents one feature, or a collection of code written for a specific feature. In Python, a module is a Python file.

So why use modules? When buying a computer, would you prefer an all-in-one model where the GPU, CPU, and memory are all soldered to the motherboard, or a modular model where they’re separate? With the all-in-one, if the CPU or memory breaks, you have to replace the whole motherboard or buy a new computer. With the modular model, you just replace the broken part. Modular wins. Same logic applies to programs.

Why use modules #

  1. Splitting code into files by feature lets you reuse the same feature in other projects.
  2. As projects grow, management gets harder; well-modularized projects are easier to manage.
  3. Coding becomes easier. Imagine a single file with tens of thousands of lines — you’d spend all your time scrolling.

When you install Python, the standard library is installed alongside it. It is made up of modules, divided into built-in modules (written in C) and regular modules (written in Python). To see all available modules, see the official docs:

Python module index

The print and open functions you use all the time live in built-in modules. The Python interpreter automatically imports them, so you can use them without an explicit import. Use the built-in dir function to see what’s currently in the main namespace:

Python code 
print(dir())
Console output 
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']

Notice __builtins__ — that’s the built-in module the interpreter auto-imports. Let’s confirm with print:

Python code 
print(__builtins__)
Console output 
<module 'builtins' (built-in)>

It’s the built-in module. Let’s see what’s inside:

Python code 
print(dir(__builtins__))
Console output 
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EncodingWarning', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'aiter', 'all', 'anext', 'any', 'ascii', 'bin', 'bool', 'breakpoint', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

The built-in module includes the print and dir functions we use all the time. Let’s call print through the built-in object:

Python code 
__builtins__.print('test')
Console output 
test

And dir:

Python code 
__builtins__.print(__builtins__.dir())
Console output 
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']

Built-in modules are written in C and pre-compiled, so you can’t see the source code directly. If you’re really curious, you can browse the C source on GitHub:

Built-in module source

Now let’s look at modules written in Python. They live in the Lib folder inside your Python install — feel free to open them yourself. You import them with the import keyword.

Let’s import pathlib from Lib and print it:

Python code 
import pathlib

print(pathlib)
Console output 
<module 'pathlib' from 'C:\\Users\\CURTIS\\anaconda3\\envs\\aws\\Lib\\pathlib.py'>

Unlike a built-in module, this one prints a file path.

The inspect module includes a getfile function that also tells you a file’s location:

Python code 
import pathlib
import inspect

print(inspect.getfile(pathlib))
C:\Users\CURTIS\anaconda3\envs\aws\Lib\pathlib.py

getsource shows the actual source code:

Python code 
import pathlib
import inspect

print(inspect.getsource(pathlib))
Console output 
import fnmatch
import functools
import io
import ntpath
import os
import posixpath
import re
...

When you import a module, the Python interpreter searches each path in sys.path in order, looking for the module. The order is: the current folder containing the entry script → paths from the PYTHONPATH environment variable → the Python install paths. Let’s print sys.path:

Python code 
import sys

print(sys.path)
Console output 
['C:\\Users\\CURTIS\\Dev\\Python\\aws', 'C:\\Users\\CURTIS\\anaconda3\\envs\\aws\\python311.zip', 'C:\\Users\\CURTIS\\anaconda3\\envs\\aws\\DLLs', 'C:\\Users\\CURTIS\\anaconda3\\envs\\aws\\Lib', 'C:\\Users\\CURTIS\\anaconda3\\envs\\aws', 'C:\\Users\\CURTIS\\anaconda3\\envs\\aws\\Lib\\site-packages']

Now let’s create a custom module. Create a module file in the current folder. By convention, module filenames describe their feature. Let’s create a module containing attack functionality for a game. Make a file attack.py with a simple print call, and use it from main.py:

attack.py 
print('attack module imported.')
main.py 
import attack

Run it with just the import:

Console output 
$ python main.py 
attack module imported.

The print inside the module ran just from importing! That’s because importing a module loads it into the current namespace, and all top-level code in the module runs at import time.

Now let’s define functions inside the module:

attack.py 
print('attack module imported.')

def arrow():
    print('Whoosh! Arrow dealt 50 damage to the enemy.')

def fireball():
    print('Boom! Fireball dealt 100 damage to the enemy.')

There are several ways to import a module. To import everything in the module like below, what gets stored in the namespace?

main.py 
import attack

print(dir())
Console output 
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'attack']

You only see the module name — not the function names. To use a function in the module, access it through the module name. Let’s attack:

main.py 
import attack

attack.arrow()
attack.fireball()
Console output 
$ python main.py 
attack module imported.
Whoosh! Arrow dealt 50 damage to the enemy.
Boom! Fireball dealt 100 damage to the enemy.

Attack succeeded. 😁

Some modules have very long names. Typing the full name every time is annoying — use as to alias:

main.py 
import attack as att

att.fireball()
Console output 
$ python main.py 
Boom! Fireball dealt 100 damage to the enemy.

If even an alias is too much and you don’t want to use the module name at all, use from with *:

main.py 
from attack import *

print(dir())
Console output 
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'arrow', 'fireball']

This time the module name is gone, and the function names are stored directly. Now you can call the functions without the module prefix:

main.py 
from attack import *

arrow()
fireball()
Console output 
Whoosh! Arrow dealt 50 damage to the enemy.
Boom! Fireball dealt 100 damage to the enemy.

If a module is large and you only need a few of its functions, it’s more efficient to import only those:

main.py 
from attack import arrow, fireball

arrow()
fireball()

That wraps up modules. The next post covers packages.

Reference: Python module list

X