Python Packaging #2 pip and requirements.txt: The Limits of the Traditional Workflow

In the previous post we gave every project its own virtual environment. What remains is recording and reproducing what got installed inside it. The traditional answer is pip and requirements.txt. This approach carried the Python ecosystem for over a decade and you will keep meeting it everywhere, so it deserves precise understanding — and at the same time, its holes are exactly why pyproject.toml and uv were born. This post covers both sides.

What pip install does #

Run pip install requests and pip finds the package on PyPI (the Python Package Index), downloads a wheel (a prebuilt zip-format distribution), and unpacks it into the virtual environment’s site-packages. The packages requests depends on (urllib3, certifi, and so on) are installed along with it. What you named is a direct dependency; what came along are transitive dependencies. That distinction is the core concept of this post.

Versions are controlled with specifiers:

requests==2.32.3     # exactly this version
requests>=2.31       # at least
requests~=2.32.0     # latest within 2.32.x (compatible release)
requests             # whatever is newest

pip list shows what’s installed; pip show requests shows a package’s version and dependency relationships.

requirements.txt: the list as a file #

Write the dependencies to a file and the environment becomes rebuildable:

# requirements.txt
django==5.2.4
requests~=2.32.0
celery

pip install -r requirements.txt

On a new machine or a colleague’s laptop, those two lines rebuild the environment. That is the textbook picture. The real-world picture starts now.

The pip freeze trap: snapshots erase intent #

The traditional answer to “record my environment exactly as it is” is freeze:

pip freeze > requirements.txt

Out comes every package in site-packages at its exact version. You gain reproducibility, at a price:

amqp==5.3.1
billiard==4.2.1
celery==5.5.3
click==8.2.1
kombu==5.5.4
vine==5.1.0
...

From this list, you cannot tell what you installed. You added one package — celery — yet five transitive dependencies stand beside it with equal standing. Give it time and three problems grow:

• You can’t delete: remove celery, and nothing in the file tells you whether amqp and kombu are direct dependencies or leftovers. The list only ever grows.
• Upgrades become scary: there is no distinction between lines that are intent (“we require 2.32”) and lines that are accident (whatever version got installed that day).
• The platform gets baked in: a list frozen on macOS includes macOS-only packages and may not install cleanly on a Linux CI runner.

The opposite trap: unpinned means install-day lottery #

If freeze feels wrong and you write just celery with no version, you fall into the opposite hole: the environment differs depending on the day it’s installed. CI gets celery 5.5 today; next month the new hire’s laptop gets 5.6. A good share of “works on my machine” is exactly this time gap. Even pinning your direct dependencies doesn’t fully close it — the transitive dependencies still float.

To sum up: a single requirements.txt is forced to serve as two different documents — a list of intent and a snapshot for reproduction — and whichever role you choose, the other collapses. The traditional workflow eventually split them into two files (intent in requirements.in, a compiled snapshot in requirements.txt — pip-tools), and that separation of declaration from lock became the standard design of every modern tool. In this series, pyproject.toml (#3) takes the declaration and the lock file (#5) takes the snapshot.

constraints: a helper file that only pins versions #

One more traditional tool worth knowing is constraints:

pip install -r requirements.txt -c constraints.txt

A constraints file installs nothing — it only restricts versions when something does get installed. It suits organization-wide rules like “urllib3 below 2.5 is banned for security reasons,” kept separate from per-project requirements. It is also the only traditional means of pinning a transitive dependency’s version.

Where you’ll still meet requirements.txt #

Knowing the limits doesn’t make the format disappear. You will keep meeting it in Docker base image builds, legacy deploy scripts, the default detection of some PaaS platforms, and collaborators’ repositories — which is why you need to read and diagnose it. Fortunately the next generation is bidirectionally compatible with it (in #4 we’ll watch uv absorb a requirements.txt wholesale).

Summary #

What we covered in this post:

• pip downloads wheels from PyPI into site-packages; your named direct dependencies and their transitive dependencies install together
• Version specifiers to know: == (pin), >= (at least), ~= (compatible release)
• pip freeze buys reproducibility by erasing intent. Direct and transitive dependencies mix indistinguishably, making deletion and upgrades hard
• Unpinned dependencies produce different environments by install date. Pinning only direct dependencies still leaves transitive ones floating
• The root cause: one file serving as both “intent” and “snapshot.” Separating declaration from lock is the standard design of modern tools
• A constraints file restricts versions without installing anything

In the next post (#3 pyproject.toml) we cover the standard home where the “list of intent” will live: pyproject.toml — the process by which everything about a project, from dependency declarations to tool settings, converges into one file.