Python Packaging #1 Virtual Environments: Why Environments Break, and venv

You definitely ran pip install, yet you get ModuleNotFoundError. You upgraded a package to fix project A, and project B broke. You tried sudo pip install like the internet said, and now an OS command is misbehaving. Everyone who uses Python for a while hits this confusion once — and it is a structural problem, not a skill problem. This series untangles that structure from the ground up in 7 parts: starting from virtual environments and how pip works, then climbing through pyproject.toml, uv, dependency locking, publishing to PyPI, and team conventions. It assumes you’ve finished Modern Python Basics, and it also explains the problem that uv — the tool the testing and automation series took for granted — actually solves.

Where import looks: sys.path and site-packages #

When import requests runs, Python walks the directories listed in sys.path in order. You can see it directly:

python -c "import sys; print('\n'.join(sys.path))"

Near the end of the list you’ll see .../site-packages. That directory is where pip installs packages — and one interpreter has exactly one site-packages. This is where the trouble starts: if every project on your machine shares one interpreter, they all share one site-packages.

If project A uses django==4.2 and project B uses django==5.2, there is no way to keep both versions side by side in the same site-packages. Upgrading for B breaks A. The mystery of “this project worked a few days ago” is almost always this: while working on another project, you changed the contents of the shared warehouse.

The system Python is an OS component #

macOS and Linux ship with Python preinstalled. It is not there for you — it is a component that OS tools depend on. Modify its site-packages with sudo pip install and you collide with files managed by the system package manager (apt, brew); in the worst case, OS utilities stop working.

That’s why recent distributions simply block it. On Debian/Ubuntu, attempting a pip install into the system Python greets you with:

error: externally-managed-environment
× This environment is externally managed

This is the guard defined by PEP 668, and it means “this Python belongs to the OS — make a virtual environment.” There is a bypass flag (--break-system-packages), but as the name says, it is a signed confession that you intend to break the system. Don’t. The correct answer is the next section.

venv: an isolated Python per project #

A virtual environment is a lightweight copy of Python with its own project-local site-packages. The standard library venv module creates one:

cd my-project
python -m venv .venv

A .venv directory appears, and its structure is simple:

.venv/
├── bin/            # python, pip executables (Scripts/ on Windows)
├── lib/
│   └── python3.13/
│       └── site-packages/   # this project's own warehouse
└── pyvenv.cfg      # records where the original interpreter lives

It doesn’t copy the whole interpreter — it is a thin shell pointing back at the original with a fresh, empty site-packages attached. That’s why creation is fast and disk usage grows only by what you install. Projects A and B now each have their own .venv, so Django 4 and 5 coexist. The structural cause of the conflict is gone.

What activate really is: PATH manipulation, nothing more #

We talk about “activating” an environment, but what actually happens is modest:

source .venv/bin/activate
(.venv) $ which python
/Users/me/my-project/.venv/bin/python

The activate script’s entire job is prepending .venv/bin to your shell’s PATH. From then on, python and pip resolve to the virtual environment’s executables first — no magic switch. deactivate restores the PATH.

Knowing this buys you two things. First, you can skip activate entirely by using explicit paths:

.venv/bin/python main.py        # run without activate
.venv/bin/pip install requests  # install without activate

In places with no shell initialization — cron, systemd — this is the standard approach. Second, the “forgot to activate” incident is diagnosed with one line: which python. Which path your python resolves to is the first question of every environment debugging session.

Three rules to keep #

1. One .venv per project: the de facto standard location is .venv at the project root. Tools (VS Code, uv) recognize the name automatically.
2. Never commit .venv: add it to .gitignore. Virtual environments have absolute paths baked in and won’t work on another machine; reproduction is done with the dependency records covered next.
3. Install nothing into the system Python: even for tools you want globally, there’s a proper way (the uv tool command covered in #4 is the answer).

Summary #

What we covered in this post:

• import walks sys.path in order, and pip installs into site-packages. One interpreter has one site-packages, so global installs create conflicts between projects
• The system Python is an OS component. The externally-managed-environment error (PEP 668) is its guard — don’t bypass it
• venv is a thin Python shell with a project-local site-packages. Create it with python -m venv .venv, one per project
• activate merely prepends .venv/bin to PATH. Calling .venv/bin/python directly works too, and which python is the first diagnostic when things look wrong
• Never commit .venv. Reproduction comes from dependency records

In the next post (#2 pip and requirements.txt), we cover the traditional form of those dependency records: pip and requirements.txt — how to use them well, and where the approach hits its limits, which is the key to understanding everything that follows.