Building Python Extension Modules with PyO3: From maturin Build to Wheel Publishing

Programming Language Rust Python
4 min read

One of the Python ecosystem’s defining recent trends is “Python on the outside, Rust on the inside.” Polars for dataframes, Ruff for linting, the core of the validation library pydantic — all share this structure, and the Python package manager uv is itself written in Rust. The bridge making the combination possible is PyO3 (bindings between Rust and the Python interpreter) plus maturin (the build and packaging tool). This article walks the minimal path — swapping one slow Python function for Rust — start to finish.

When this card is worth playing #

Judgment first. Most slow Python code is slow for algorithmic or I/O reasons and can be fixed within Python. Rust is appropriate when profiling has confirmed the bottleneck is CPU work, and the logic is custom enough that existing native libraries like NumPy cannot express it (finding the bottleneck is covered in the py-spy article). This is not a full rewrite — start with one bottleneck function. That is the spirit of this route.

Starting out: maturin builds the skeleton #

Project setup 
pip install maturin
maturin new --bindings pyo3 fastcount
cd fastcount

The generated structure is a Rust project with Python packaging attached.

Project structure 
fastcount/
├── Cargo.toml       # crate-type = ["cdylib"] — build as a dynamic library
├── pyproject.toml   # build backend is maturin
└── src/
    └── lib.rs       # the extension module itself

Because pyproject.toml names maturin as the build backend, from the Python side this project is an ordinary package: pip install . works, and as we will see, wheel publishing follows the standard procedure too.

Exposing a Rust function to Python #

The example is a word-frequency counter — the classic CPU-bound loop that pure-Python iteration makes slow.

src/lib.rs 
use pyo3::prelude::*;
use std::collections::HashMap;

/// Count word frequencies in a text
#[pyfunction]
fn word_count(text: &str) -> HashMap<String, u64> {
    let mut counts: HashMap<String, u64> = HashMap::new();
    for word in text.split_whitespace() {
        *counts.entry(word.to_lowercase()).or_insert(0) += 1;
    }
    counts
}

#[pymodule]
fn fastcount(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(word_count, m)?)?;
    Ok(())
}

#[pyfunction] dresses a Rust function in Python’s calling convention, and #[pymodule] registers it on the module. Notice that type conversion is handled by the signature alone: a Python str arrives as &str, and the returned HashMap<String, u64> leaves as a Python dict[str, int]. Vec ↔ list and Option ↔ None-accepting arguments work on the same principle, and passing an unconvertible type raises a TypeError on the Python side. Return a Rust Result and its Err converts into a Python exception — the two languages’ error models connect naturally as well.

The development cycle: maturin develop #

Development cycle 
python -m venv .venv && source .venv/bin/activate
maturin develop --release
python -c "import fastcount; print(fastcount.word_count('a b a')['a'])"  # 2

maturin develop installs the compiled extension straight into the current virtual environment, creating the edit-build-verify loop. When looking at performance, --release is mandatory (the rule from practice Part 4 applies here unchanged). For string-processing loops like this, improvements in the tens-of-times range over pure Python are the usual expectation — and the exact multiple should always be stated from measurements on your own data.

The fit with free-threaded Python #

The relationship with Python’s big shift, free-threading (the GIL-free build), deserves a note. Extension modules traditionally got thread safety for free thanks to the GIL; under free-threaded builds, that premise disappears. PyO3 ships free-threaded Python support, and Rust code has its data races blocked by the compiler in the first place (practice Part 5), which puts this route in a favorable position for the transition. “Truly parallel extension modules under GIL-free Python” is this path’s next chapter.

Shipping: all the way to wheels #

Publish 
maturin build --release        # build a wheel for the current platform
maturin publish                # upload to PyPI as well

The wheels maturin produces ride the standard PyPI publishing procedure as-is. The caveat is the fate of every native extension: per-platform builds. Unlike pure-Python wheels, you need one per OS and architecture, so real distribution conventionally uses CI matrix builds (the same construction as Rust practice Part 8) to produce Linux, macOS, and Windows wheels in parallel. maturin even provides a command that generates the GitHub Actions workflow for this (maturin generate-ci).

Summary #

  • “Python outside, Rust inside” is the standard route validated by Polars, Ruff, and pydantic — PyO3 plus maturin is the toolset.
  • The valid situation: a profiler-confirmed CPU bottleneck plus custom logic beyond existing native libraries. Start with one bottleneck function.
  • Expose with #[pyfunction] and #[pymodule]; the signature handles conversion (str ↔ &str, dict ↔ HashMap, exceptions ↔ Result).
  • Development is the maturin develop --release loop; shipping is maturin build wheels on the standard PyPI procedure, with CI matrices for per-platform wheels.
  • In the free-threaded era, extension-module thread safety stops being free — and Rust, which blocks data races at compile time, holds the favorable seat for that transition.
X