pytest

What it is

pytest is the most-used Python testing framework, originally by Holger Krekel and now maintained by the pytest-dev organisation. It auto-discovers test_*.py files, rewrites assert statements to produce introspected failure messages, and exposes a fixture system that doubles as the framework's plugin contract.

The package itself is small — most of pytest's reach comes from the third-party pytest-* plugin ecosystem on PyPI (pytest-cov, pytest-asyncio, pytest-mock, pytest-xdist, pytest-django, …), each of which hooks into pytest via entry points.

Install

pip install pytest

Output: (none — exits 0 on success)

uv add --dev pytest

Output: dependency added to the dev group in pyproject.toml

poetry add --group dev pytest

Output: updated lockfile + dev install

pipx install pytest

Output: installed to isolated venv, pytest CLI on PATH (useful for ad-hoc runs outside a project venv)

Versioning & Python support

• Current major line is 8.x (released 2024). The project follows a roughly yearly major-release cadence — 7.x (2022), 6.x (2020), 5.x (2019).
• Recent releases support Python 3.8+. Older Python floors are dropped one major at a time; check the changelog before pinning.
• Loose semver — major bumps deprecate APIs that previously raised PytestDeprecationWarning for at least one minor cycle.
• Plugins typically pin to a pytest major (e.g. pytest-asyncio requires pytest>=8.2), so when you upgrade pytest you usually upgrade plugins together.

Package metadata

• Maintainer: pytest-dev GitHub org (community-maintained)
• Project home: github.com/pytest-dev/pytest
• Docs: docs.pytest.org
• PyPI: pypi.org/project/pytest
• License: MIT
• Governance: community / pytest-dev core team, with sponsorship via the Python Software Foundation
• First released: 2004 (as part of the py library, split out around 2009)
• Downloads: tens of millions per week — consistently in PyPI's top 20

Optional dependencies & extras

pytest itself has no published extras — install plugins as separate packages. The most common companions:

Discovery is automatic via entry points — install the plugin and pytest picks it up.

Alternatives

Common gotchas

1. Collection order is filesystem-dependent. Tests within a file run top-to-bottom, but file ordering depends on the OS's directory iteration. Don't rely on cross-file ordering; install pytest-randomly to expose hidden ordering assumptions.
2. conftest.py scope is directory-based, not import-based. Fixtures defined in tests/conftest.py are available to every test under tests/, but a nested tests/api/conftest.py only applies to tests/api/. This is by design but routinely surprises people moving fixtures around.
3. tmp_path is per-test; tmp_path_factory is per-session. Use the factory when multiple tests need to share a fixture-built artefact (e.g. a generated dataset). Mixing them up causes flaky tests that look like cache pollution.
4. Markers vs fixtures are different things. @pytest.mark.slow is metadata you filter on (pytest -m "not slow"); fixtures are dependency-injected values. New users frequently try to "use" a marker like a fixture or vice versa.
5. pytest-asyncio event-loop modes changed. Older releases default to strict, newer ones default to auto. Pin the mode explicitly in pyproject.toml under [tool.pytest.ini_options] to avoid silent behaviour drift on upgrade.
6. rootdir detection drives config discovery. pytest walks up from the test file looking for pyproject.toml, pytest.ini, tox.ini, or setup.cfg. If the wrong file matches first, your config silently does not apply — pytest --collect-only prints the resolved rootdir.
7. Assertion rewriting happens at import time. If a test module is imported before pytest starts (e.g. via a shared helper that imports it), the rich assert introspection is disabled for that module. Keep test files out of the import graph of non-test code.

Plugin & rule ecosystem

The pytest ecosystem is a constellation of pytest-* plugins discovered via setuptools entry points (group pytest11). Install a plugin and it self-registers — there is no manual INSTALLED_PLUGINS list. The trade-off is that a misbehaving plugin can break collection for the entire suite; pytest -p no:<name> disables one and pytest --trace-config dumps the full discovery order.

For authoring custom plugins, register an entry point under pytest11 in pyproject.toml. Hooks live in pytest_plugin_name/plugin.py and are picked up by name.

Testing strategies

The pytest core is small; choosing what to test is where projects diverge. A pragmatic layering for a typical service project:

• Unit tests — pure functions, single classes, no I/O. Sub-millisecond per test. Live next to the code under tests/unit/. Mock external collaborators with pytest-mock.
• Integration tests — real dependencies (database, queue, in-memory HTTP server). Slower (100 ms–1 s). Use pytest-postgresql or testcontainers for ephemeral services. scope="session" fixtures amortise setup.
• Property-based — hypothesis for invariants ("serialize then deserialize is identity for any input"). Shrinks failing inputs automatically.
• Snapshot — syrupy for serialised outputs (rendered HTML, generated SQL, log messages). Cheap to add; reviewable in diffs.
• Contract tests — when consuming a third-party API, record a real response once (pytest-vcr, responses, or pytest-recording) and replay forever.
• Mutation — mutmut runs the test suite against perturbed source to detect untested branches. Slow; run nightly.

# tests/integration/test_user_repo.py — testcontainers + transactional rollback
import pytest
from testcontainers.postgres import PostgresContainer
from sqlalchemy import create_engine

@pytest.fixture(scope="session")
def db_url():
    with PostgresContainer("postgres:16") as pg:
        yield pg.get_connection_url()

@pytest.fixture
def db_session(db_url):
    engine = create_engine(db_url)
    conn = engine.connect()
    txn = conn.begin()
    yield conn
    txn.rollback()
    conn.close()

Output: every test gets a fresh transactional sandbox; the postgres container starts once per session and is torn down on exit.

The fixture composition above — session-scoped container, function-scoped transactional rollback — is the canonical "fast database test" pattern. Without the rollback layer each test would need a fresh database, multiplying the test-run time by N.

Real-world recipes

Parameterised matrix with descriptive IDs

import pytest
from datetime import date

@pytest.mark.parametrize(
    "birth, today, expected",
    [
        pytest.param(date(1990, 5, 1), date(2026, 5, 1), 36, id="exact-birthday"),
        pytest.param(date(1990, 5, 2), date(2026, 5, 1), 35, id="day-before"),
        pytest.param(date(2000, 2, 29), date(2026, 3, 1), 26, id="leap-baby"),
    ],
)
def test_age(birth, today, expected):
    assert age(birth, today) == expected

Custom id= strings appear in pytest -v output and -k filters — pytest -k leap-baby runs just the leap-day case.

Indirect parametrisation through a fixture

@pytest.fixture
def user(request):
    return User(role=request.param, active=True)

@pytest.mark.parametrize("user", ["admin", "guest", "viewer"], indirect=True)
def test_dashboard_access(user, client):
    response = client.get("/dashboard", auth=user)
    assert response.status_code == (200 if user.role != "guest" else 403)

indirect=True routes the parameter through the user fixture, which builds the full object. Use this when test inputs need post-processing (DB lookup, factory composition).

Async test with shared transport

import pytest
import httpx

@pytest.fixture(scope="module")
async def client():
    async with httpx.AsyncClient(base_url="https://api.example.com") as c:
        yield c

@pytest.mark.asyncio
async def test_list_users(client):
    r = await client.get("/users")
    assert r.status_code == 200

pytest-asyncio in mode = "auto" removes the @pytest.mark.asyncio decorator. Module-scoped async fixtures need pytest-asyncio>=0.23; older versions require function scope for async fixtures.

Marker-gated slow tests

# pyproject.toml
[tool.pytest.ini_options]
markers = [
    "slow: integration tests that hit network or DB",
    "smoke: minimal critical-path test set",
]
addopts = "-m 'not slow'"

Local runs skip slow tests by default; CI flips the gate with pytest -m "" (no filter) or pytest -m slow for the dedicated slow-suite stage.

Modifying collection from conftest.py

# conftest.py
def pytest_collection_modifyitems(config, items):
    skip_slow = pytest.mark.skip(reason="slow tests off in PR runs")
    if config.getoption("--no-slow", default=False):
        for item in items:
            if "slow" in item.keywords:
                item.add_marker(skip_slow)

Add a --no-slow flag via pytest_addoption and dynamically skip during collection. Cleaner than scattering @pytest.mark.skipif(...) decorators.

Performance tuning

Test runs grow as projects do; a 30-second loop is fine, a 30-minute loop kills developer velocity. The levers, in order of biggest impact:

• Parallelise with pytest-xdist. pytest -n auto divides tests across CPU cores. Three-way trade-off: session-scoped fixtures run once per worker, so heavy fixtures multiply cost; --dist loadgroup plus @pytest.mark.xdist_group("name") pins related tests to one worker.
• Right-size fixture scopes. A database connection at scope="function" is wasteful; at scope="session" it's shared across all tests. Default is function — bump deliberately, and add transactional rollback at function scope on top.
• Collect only what you need. pytest tests/unit -k "not slow" runs a subset. pytest --lf re-runs only the last-failed tests during a fix loop.
• Profile collection itself. pytest --collect-only -q shows the test list. If collection takes seconds, you have import-time work in test modules — move it inside fixtures or test bodies.
• Disable plugins you don't use. -p no:cacheprovider -p no:doctest shaves startup time. List active plugins with pytest --trace-config.
• pytest --durations=10 prints the ten slowest tests. Optimise the top ones — most suites follow a Pareto distribution.
• Avoid global autouse=True fixtures with heavy setup. Every test pays the cost whether it uses the fixture or not.

The pytest-testmon plugin tracks which lines each test exercises and skips unaffected tests on the next run. Effective on large suites; setup cost is real.

Troubleshooting common errors

For deep collection issues, pytest --collect-only -q is the surgical tool — it lists every test pytest would run without executing any code.

Ecosystem integrations

pytest sits in the middle of a wider test orchestration layer:

• tox / nox — environment matrices. Each tox env creates an isolated venv with a specific Python version and dep set, then invokes pytest. nox is the newer, Python-config equivalent.
• hatch — modern project manager with hatch test baked in (invokes pytest under the hood). Bundles environment management.
• coverage.py — used by pytest-cov. Combine subprocess coverage with coverage combine before reporting.
• pytest-randomly + hypothesis — both reseed each run; co-existing requires care so pytest-randomly doesn't reseed after hypothesis has chosen inputs.
• pytest-watch / ptw — re-runs tests on file changes. Useful in tight TDD loops but no longer maintained; pytest-watcher is the maintained successor.
• CI runners — GitHub Actions, GitLab CI, Buildkite all invoke pytest the same way. The --junitxml=report.xml flag generates a JUnit-format report that CI dashboards parse.
• IDE integration — PyCharm and VS Code (Python extension) auto-discover pytest tests and show pass/fail in the gutter. Both invoke pytest with --rootdir and --collect-only under the hood.

Version migration guide

pytest's major-version bumps are conservative — most projects upgrade by bumping the pin and re-running the suite. The deprecations that catch people out:

pytest 7 → 8 (2024)

• nose-compatibility methods (setup_method/teardown_method via nose) removed. Use explicit fixtures.
• pytest.warns() now requires a match= pattern when checking for a specific warning class; the bare form is deprecated for stricter assertions.
• --strict (no suffix) removed in favour of --strict-markers and --strict-config. CI configs that pass --strict fail with unrecognised option.
• The pytest.ini_options.minversion field is now enforced strictly — set it to your floor.

pytest 6 → 7 (2022)

• tmpdir (the legacy py.path.local fixture) deprecated in favour of tmp_path (returns pathlib.Path). Both still exist as of 8.x but tmpdir will be removed.
• pytest.skip() outside a test function (e.g. at module top level) now requires allow_module_level=True.

Plugin pinning — when bumping pytest, bump plugins in the same PR. pytest-asyncio<0.21 is incompatible with pytest 8; pytest-django<4.5 likewise. The plugin's setup.py declares pytest>=X — but transitive constraints are usually too loose, so explicit pins win.

A safe upgrade pattern: pin pytest==X.Y.Z exactly in pyproject.toml, run the suite, fix deprecations one warning class at a time using -W error::PytestDeprecationWarning to turn deprecation warnings into hard failures during the upgrade pass, then relax the pin to pytest>=X.Y once green.

CI integration

Minimal GitHub Actions workflow that runs pytest with coverage and uploads results:

name: tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -e ".[test]"
      - run: pytest --cov=mypkg --cov-report=xml -n auto --junitxml=report.xml
      - uses: codecov/codecov-action@v4
        if: matrix.python-version == '3.12'

Key choices in this template:

• Matrix on Python version — catches stdlib-behaviour drift early. Drop the oldest version one release after upstream EOL.
• -n auto — parallelises across the runner's CPUs (GitHub's ubuntu-latest gives 4).
• JUnit XML — every CI provider renders this format. Failures get a dedicated panel with the failing test list.
• Coverage uploaded once — uploading from every matrix cell duplicates lines. Pick the canonical version.

For monorepos, split test stages: a fast smoke job (pytest -m smoke, 30 s) gates the rest of the pipeline; a full job (pytest, 5 min) runs in parallel with build/deploy.

When NOT to use this

pytest is the right default, but it's not free. Cases where the stdlib or no framework wins:

• Single-file scripts. A 50-line CLI doesn't need a test framework — python -c 'import script; assert script.foo() == 42' may be enough. Add pytest when the test count crosses ~10.
• Library tests that ship to end-users. If a library bundles its own tests for downstream verification, unittest (stdlib) means no extra install. Reach for pytest only if the test suite is private.
• Embedded / restricted runtimes. MicroPython, AWS Lambda layers, anything where every megabyte matters — pytest plus plugins is 5+ MB. Stdlib unittest is free.
• Heavy mocking of import-time side effects. pytest's assertion rewriting happens at import. Tests that must manipulate sys.modules before pytest's importer runs may be easier under unittest.main() invoked directly.
• Doctest-heavy projects. python -m doctest works fine without pytest. The pytest --doctest-modules integration exists, but if doctests are your only tests, the framework overhead is mostly wasted.

For everything else — services, CLIs, libraries beyond ~500 LOC, projects with multiple contributors — pytest is the right tool.

See also

• Python: pytest — full API, fixtures, parametrize, recipes
• Concept: API — public-surface stability and test contracts
• Packages: pip-mypy — type-check alongside testing
• Packages: pip-ruff — lint your tests too