cheat sheet
pytest
Write and run Python tests with pytest. Covers test discovery, assertions, fixtures, parametrize, conftest, and common patterns.
pytest — Testing Framework
What it is
pytest is the de-facto standard Python testing framework. It discovers and runs tests automatically, provides rich assertion introspection, and extends via a large plugin ecosystem (pytest-asyncio, pytest-cov, pytest-mock, etc.).
Install
pip install pytest
pip install pytest-cov # coverage reports
pip install pytest-asyncio # async test support
Output: (none — exits 0 on success)
Quick example
# test_math.py
def add(a: int, b: int) -> int:
return a + b
def test_add():
assert add(2, 3) == 5
assert add(-1, 1) == 0
assert add(0, 0) == 0
pytest test_math.py -v
Output:
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.3.5
collected 1 item
test_math.py::test_add PASSED [100%]
============================== 1 passed in 0.05s ===============================
When / why to use it over unittest
- Simpler syntax: plain
assertstatements give clear failure messages. - Fixture dependency injection is more composable than
setUp/tearDown. - Built-in parametrize replaces manual test table loops.
- Enormous plugin ecosystem.
- Can run
unittest-style tests too.
Common pitfalls
Test file naming — pytest discovers files named
test_*.pyor*_test.pyand functions/methods namedtest_*. Files not matching this pattern are silently ignored.
Sharing mutable fixtures — fixtures with
scope="module"orscope="session"are shared. Mutating them in one test affects others. Usescope="function"(the default) unless you know what you're doing.
pytest -k "add"runs only tests whose name contains "add".pytest -xstops on first failure.pytest -sshowsprint()output in real time.
Parametrize
@pytest.mark.parametrize runs the same test function multiple times with different input/expected pairs. It eliminates copy-pasted test functions and surfaces all failing cases in a single run rather than stopping at the first one.
import pytest
@pytest.mark.parametrize("a, b, expected", [
(1, 2, 3),
(0, 0, 0),
(-1, 1, 0),
(100, -100, 0),
])
def test_add_param(a, b, expected):
assert a + b == expected
pytest test_math.py::test_add_param -v
Output:
test_math.py::test_add_param[1-2-3] PASSED
test_math.py::test_add_param[0-0-0] PASSED
test_math.py::test_add_param[-1-1-0] PASSED
test_math.py::test_add_param[100--100-0] PASSED
4 passed in 0.04s
Richer example — fixtures and tmp files
# test_fixtures.py
import pytest
from pathlib import Path
@pytest.fixture
def sample_data() -> list[dict]:
return [{"id": i, "value": i * 10} for i in range(1, 4)]
def test_data_length(sample_data):
assert len(sample_data) == 3
def test_data_values(sample_data):
assert sample_data[0]["value"] == 10
assert sample_data[-1]["value"] == 30
def test_file_roundtrip(tmp_path: Path):
"""tmp_path is a built-in pytest fixture: a temporary directory unique per test."""
f = tmp_path / "data.txt"
f.write_text("hello\nworld")
lines = f.read_text().splitlines()
assert lines == ["hello", "world"]
pytest test_fixtures.py -v
Output:
test_fixtures.py::test_data_length PASSED
test_fixtures.py::test_data_values PASSED
test_fixtures.py::test_file_roundtrip PASSED
3 passed in 0.06s
conftest.py — shared fixtures
Place fixtures used across multiple test files in conftest.py at the root (or any directory):
# conftest.py
import pytest
@pytest.fixture(scope="module")
def db_connection():
"""Module-scoped fixture: one connection per test module."""
conn = create_test_db()
yield conn
conn.close()
Pytest auto-discovers conftest.py — no import needed.
Coverage
pytest --cov=src --cov-report=term-missing
Output:
---------- coverage: platform linux, python 3.12.3 ----------
Name Stmts Miss Cover Missing
-----------------------------------------------
src/math_utils.py 10 1 90% 42
-----------------------------------------------
TOTAL 10 1 90%
Useful markers
Built-in markers let you skip tests conditionally, flag known failures, or filter test runs. skip excludes a test entirely; xfail runs it but expects it to fail (and flags unexpectedly passing tests); custom marks (registered in pyproject.toml) let you select subsets with -m.
@pytest.mark.skip(reason="not implemented yet")
def test_future():
...
@pytest.mark.xfail(reason="known bug #123")
def test_known_failure():
assert 1 == 2
@pytest.mark.slow
def test_large_dataset():
...
Run only fast tests: pytest -m "not slow"
Fixtures in depth
A fixture is a function decorated with @pytest.fixture whose return value is injected into any test that names it as a parameter. Fixtures replace the setUp/tearDown pattern with a composable, dependency-injection model — a test asks for db_connection by name, and pytest builds the dependency graph automatically. Fixtures can depend on other fixtures, be parameterised, and ship in conftest.py for sharing across modules.
Scope — how often a fixture is built
The scope parameter controls when pytest creates and destroys the fixture. function (default) builds a fresh instance for every test; session builds once for the entire run. Use the broadest scope that keeps tests isolated.
| Scope | Created | Destroyed | Use for |
|---|---|---|---|
function (default) | Before each test | After each test | Most data, anything mutable |
class | First test in class | After last test in class | Class-level shared state |
module | First test in file | After last test in file | Module-scoped expensive setup |
package | First test in package | After last test in package | Package fixtures (rare) |
session | Once per pytest run | After all tests | DB engines, HTTP servers, browser instances |
import pytest
@pytest.fixture(scope="session")
def db_engine():
"""One engine for the entire test run."""
engine = create_engine("sqlite:///:memory:")
yield engine
engine.dispose()
@pytest.fixture(scope="function")
def db_session(db_engine):
"""Fresh transactional session per test — rolled back after."""
connection = db_engine.connect()
txn = connection.begin()
session = Session(bind=connection)
yield session
session.close()
txn.rollback()
connection.close()
yield — teardown without try/finally
A fixture with yield runs the code before yield as setup, returns the yielded value, then runs everything after yield as teardown. The teardown runs even if the test fails — same semantics as a context manager, but no try/finally boilerplate.
@pytest.fixture
def temp_dir(tmp_path):
"""Set up a temp dir with seed files, then clean up."""
(tmp_path / "input.txt").write_text("hello")
(tmp_path / "data").mkdir()
yield tmp_path
# Teardown — runs even if the test raises
for p in tmp_path.iterdir():
if p.is_file():
p.chmod(0o644) # ensure removable
Parametrize a fixture
Add params=[...] to a @pytest.fixture and every test using the fixture runs once per param. Useful for "this test should pass against SQLite, Postgres, and MySQL" matrices.
@pytest.fixture(params=["sqlite", "postgres", "mysql"])
def db_backend(request):
return request.param
def test_insert(db_backend):
"""Runs three times — once per backend."""
assert db_backend in {"sqlite", "postgres", "mysql"}
pytest test_db.py -v
Output:
test_db.py::test_insert[sqlite] PASSED
test_db.py::test_insert[postgres] PASSED
test_db.py::test_insert[mysql] PASSED
Indirect parametrization
pytest.mark.parametrize can pass values through a fixture instead of as direct test arguments. Use indirect=True to route the param into the fixture rather than the test.
@pytest.fixture
def client(request):
"""Build a client configured per the parametrized URL."""
return APIClient(base_url=request.param)
@pytest.mark.parametrize("client", ["http://api1", "http://api2"], indirect=True)
def test_health(client):
assert client.get("/health").status_code == 200
Built-in fixtures pytest ships
pytest exposes a small library of pre-built fixtures. The most useful:
| Fixture | What it gives you |
|---|---|
tmp_path | pathlib.Path — unique temporary directory per test |
tmp_path_factory | Session-scoped tmp directory factory |
tmpdir | Older py.path.local version of tmp_path (prefer tmp_path) |
monkeypatch | Patch env vars, attributes, sys.path — auto-reverts |
capsys / capfd | Capture stdout/stderr (capsys Python-level, capfd file-descriptor-level) |
caplog | Capture log records from stdlib logging |
request | Introspect the current test (name, params, node id) |
recwarn | Capture warnings emitted during the test |
pytestconfig | Access pytest config values (CLI args, ini settings) |
cache | Persistent cross-run cache (--cache-clear, pytest_cache/) |
def test_env(monkeypatch):
monkeypatch.setenv("API_KEY", "test-key")
monkeypatch.delenv("LOG_LEVEL", raising=False)
monkeypatch.setattr("myapp.config.RETRIES", 0)
# All changes reverted after the test
def test_output(capsys):
print("hello")
captured = capsys.readouterr()
assert captured.out == "hello\n"
def test_warning(recwarn):
import warnings
warnings.warn("deprecated", DeprecationWarning)
assert len(recwarn) == 1
assert recwarn[0].category is DeprecationWarning
autouse — fixtures that always run
autouse=True makes a fixture apply to every test in its scope without being requested. Use sparingly — autouse fixtures are invisible at the test site, which hides dependencies. Common legitimate uses: reset a global, seed an RNG.
@pytest.fixture(autouse=True)
def reset_globals():
"""Every test gets a clean state."""
myapp.GLOBAL_REGISTRY.clear()
yield
myapp.GLOBAL_REGISTRY.clear()
Markers — categorising and gating tests
A marker tags a test with metadata. Built-in markers handle skipping, expected failure, and parametrization; custom markers let you run subsets of the suite (pytest -m slow). Register custom markers in pyproject.toml to silence "unknown marker" warnings.
Built-in markers
import pytest
import sys
@pytest.mark.skip(reason="not implemented yet")
def test_future():
...
@pytest.mark.skipif(sys.platform == "win32", reason="Linux/macOS only")
def test_unix_socket():
...
@pytest.mark.xfail(reason="known bug #123", strict=False)
def test_known_failure():
assert 1 == 2
@pytest.mark.xfail(raises=ValueError, strict=True)
def test_must_raise():
raise ValueError("expected")
# Custom markers
@pytest.mark.slow
def test_large_dataset():
...
@pytest.mark.integration
def test_with_real_db():
...
| Marker | Behaviour |
|---|---|
skip | Don't run; report as skipped |
skipif(condition) | Skip if condition truthy |
xfail | Run; expected to fail (unexpectedly-passing tests are flagged unless strict=False) |
parametrize | Run once per parameter set |
usefixtures | Apply fixtures by marker instead of arg list |
filterwarnings | Apply warnings.filterwarnings for this test only |
Registering custom markers
[tool.pytest.ini_options]
markers = [
"slow: tests that take more than a second",
"integration: requires external services",
"unit: fast pure-Python tests",
"smoke: minimum-viable subset that runs first",
]
# Run only fast unit tests
pytest -m "unit and not slow"
# Run everything except integration
pytest -m "not integration"
# Combine with -k for finer slicing
pytest -m slow -k "checkout"
Output:
collected 247 items / 12 deselected / 235 selected
tests/test_checkout.py::test_total_with_tax[slow] PASSED
tests/test_checkout.py::test_discount_application[slow] PASSED
2 passed, 12 deselected in 1.42s
conftest.py — sharing fixtures and hooks
conftest.py is pytest's plugin discovery mechanism inside your own repo. Any fixture, marker, hook, or plugin defined in a conftest.py is available to every test in that directory and its descendants. Place project-wide fixtures at the repo root; place narrower fixtures deeper in the tree.
my_project/
├── pyproject.toml
├── src/
│ └── myapp/
└── tests/
├── conftest.py # project-wide fixtures
├── unit/
│ ├── conftest.py # unit-test-specific fixtures
│ └── test_models.py
└── integration/
├── conftest.py # integration-specific fixtures (real DB)
└── test_api.py
# tests/conftest.py
import pytest
@pytest.fixture(scope="session")
def event_loop_policy():
import asyncio
return asyncio.DefaultEventLoopPolicy()
@pytest.fixture
def alice_user():
return {"id": 42, "name": "Alice Dev", "email": "alice@example.com"}
# tests/integration/conftest.py
import pytest
@pytest.fixture(scope="session")
def real_db():
"""Only available to tests under tests/integration/."""
engine = create_engine(os.environ["TEST_DB_URL"])
yield engine
engine.dispose()
Hooks — extend pytest itself
conftest.py can define pytest hook functions to alter collection, reporting, and lifecycle. The most commonly used:
# conftest.py
def pytest_collection_modifyitems(config, items):
"""Auto-mark anything under tests/integration as slow."""
for item in items:
if "integration" in item.nodeid:
item.add_marker("slow")
def pytest_configure(config):
"""Run once per pytest startup."""
config.addinivalue_line("markers", "smoke: minimum subset")
def pytest_sessionfinish(session, exitstatus):
"""Run once at end of session."""
print(f"\n[done] {session.testscollected} tests, exit {exitstatus}")
Configuration in pyproject.toml
pytest reads its config from one of pyproject.toml, pytest.ini, pyproject.toml's [tool.pytest.ini_options], tox.ini, or setup.cfg. Prefer pyproject.toml for new projects.
[tool.pytest.ini_options]
minversion = "8.0"
testpaths = ["tests", "src/tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"-ra", # show short test summary for all but passed
"--strict-markers", # error on unknown markers
"--strict-config", # error on unknown config keys
"--showlocals", # show local vars in tracebacks
"--tb=short", # short tracebacks
"--durations=10", # show 10 slowest tests
]
markers = [
"slow: tests that take more than a second",
"integration: requires external services",
"smoke: minimum subset",
]
filterwarnings = [
"error", # promote warnings to errors
"ignore::DeprecationWarning:third_party", # except this one
]
asyncio_mode = "auto" # pytest-asyncio: auto-detect async tests
log_cli = false # disable live log output (use caplog)
log_cli_level = "INFO"
Useful flags
| Flag | What it does |
|---|---|
-v / -vv | Verbose / very verbose output |
-q | Quiet — one char per test |
-s | Disable output capture (show print() live) |
-x / --exitfirst | Stop on first failure |
--maxfail=N | Stop after N failures |
-k EXPR | Run tests whose name matches expression ("add or sub") |
-m MARKEXPR | Run tests matching marker expression ("slow and not flaky") |
--lf / --last-failed | Re-run only the tests that failed last time |
--ff / --failed-first | Run last-failed first, then the rest |
--nf / --new-first | Run new tests first |
--pdb | Drop into pdb on first failure |
--trace | Drop into pdb at the start of every test |
--cache-clear | Clear the pytest cache directory |
--collect-only | List tests without running them |
-p no:cacheprovider | Disable cache plugin (for clean comparisons) |
--durations=N | Print the N slowest tests at the end |
-n NUM | Run in parallel (requires pytest-xdist) |
--co | Alias for --collect-only |
# Iterate fast: stop on first failure, drop into pdb
pytest -x --pdb -s
# Reproduce a flaky failure: re-run only failed, stop on first
pytest --lf -x
# Profile: find the slow tests
pytest --durations=20
# Parallel run on 8 cores (pytest-xdist required)
pytest -n 8
# Run only tests matching a pattern, in one file
pytest tests/test_orders.py -k "checkout"
Output:
============================= test session starts ==============================
collected 124 items / 121 deselected / 3 selected
tests/test_orders.py::test_checkout_basic PASSED [ 33%]
tests/test_orders.py::test_checkout_discount PASSED [ 66%]
tests/test_orders.py::test_checkout_tax PASSED [100%]
========================= 3 passed, 121 deselected in 0.42s ====================
Plugin ecosystem
pytest's strength is its plugin ecosystem. Below are the plugins that earn their keep on most Python projects. All install with pip install <pkg> and integrate without code changes (some need a config line).
pytest-cov — coverage
Measures which lines of source code your tests execute. Backed by coverage.py. Run alongside the tests with --cov=PACKAGE.
pip install pytest-cov
pytest --cov=src --cov-report=term-missing --cov-report=html --cov-fail-under=80
Output:
---------- coverage: platform linux, python 3.12.3 ----------
Name Stmts Miss Cover Missing
---------------------------------------------------
src/api.py 42 3 93% 58-60
src/models.py 89 0 100%
src/utils.py 24 1 96% 17
---------------------------------------------------
TOTAL 155 4 97%
[tool.coverage.run]
source = ["src"]
omit = ["src/**/__init__.py", "src/migrations/*"]
branch = true
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"raise NotImplementedError",
"if TYPE_CHECKING:",
"if __name__ == .__main__.:",
]
fail_under = 80
show_missing = true
pytest-xdist — parallel test execution
Distributes tests across processes (-n auto uses every core). Critical for large suites — a 5-minute serial run becomes 45 seconds. Tests must be isolated; shared state across processes will fail.
pip install pytest-xdist
pytest -n auto # one worker per CPU
pytest -n 4 # exactly 4 workers
pytest -n auto --dist=loadfile # all tests in a file go to the same worker
Output:
============================= test session starts ==============================
plugins: xdist-3.6.1
8 workers [124 items]
............................................................ [ 48%]
................................................................ [100%]
============================== 124 passed in 6.41s =============================
pytest-asyncio — testing coroutines
pytest by default can't run async def tests. pytest-asyncio makes the event loop available as a fixture and dispatches coroutines for you.
pip install pytest-asyncio
Output: (none — exits 0 on success)
[tool.pytest.ini_options]
asyncio_mode = "auto" # any async def test runs as a coroutine
import pytest
import asyncio
@pytest.mark.asyncio # explicit mode
async def test_fetch():
result = await fetch_data("https://api.example.com")
assert result["status"] == 200
# With asyncio_mode = "auto" the decorator is unnecessary
async def test_concurrent():
a, b = await asyncio.gather(fetch_data("/a"), fetch_data("/b"))
assert a and b
pytest-mock — friendlier mocking
Wraps unittest.mock in a mocker fixture so you don't need with patch(...) context managers. Mocks auto-undo at test end.
pip install pytest-mock
Output: (none — exits 0 on success)
def test_send_email(mocker):
"""Patch smtplib.SMTP for the duration of this test."""
mock_smtp = mocker.patch("myapp.email.smtplib.SMTP")
send_email("alice@example.com", "subject", "body")
mock_smtp.return_value.sendmail.assert_called_once()
def test_patch_attr(mocker):
mocker.patch("myapp.config.RETRIES", 5)
assert myapp.config.RETRIES == 5
hypothesis — property-based testing
Generates randomised inputs to surface edge cases your handwritten cases miss. Tells you the minimum input that breaks a property.
pip install hypothesis
Output: (none — exits 0 on success)
from hypothesis import given, strategies as st
@given(st.lists(st.integers()))
def test_sort_is_idempotent(xs):
assert sorted(sorted(xs)) == sorted(xs)
@given(st.text())
def test_roundtrip(s):
assert s.encode().decode() == s
When hypothesis finds a failure, it shrinks the input to the simplest reproducer:
Falsifying example: test_roundtrip(s='\ud800')
pytest-benchmark — performance regression detection
Pins each benchmark's baseline timing; failures surface when a change makes a benchmark slower.
pip install pytest-benchmark
Output: (none — exits 0 on success)
def test_parse_speed(benchmark):
result = benchmark(parse, "x" * 10000)
assert len(result) == 10000
Other notable plugins
| Plugin | Purpose |
|---|---|
pytest-django | Django integration: client, db, admin_user fixtures |
pytest-flask | Flask app fixtures |
pytest-playwright | Browser automation via Playwright |
pytest-randomly | Randomise test order — surfaces test-order dependencies |
pytest-sugar | Prettier output: per-test progress bar, instant failures |
pytest-rerunfailures | Retry flaky tests (--reruns 3) |
pytest-timeout | Per-test timeout via @pytest.mark.timeout(30) |
pytest-env | Set env vars in pyproject.toml |
pytest-html | HTML report of test results |
pytest-clarity | Better diffs in assertion failures |
Mocking patterns
The stdlib unittest.mock is the foundation; pytest-mock wraps it in a fixture. The three patterns you'll use 95% of the time:
Patch an attribute on a module
def test_with_mock(mocker):
"""Replace the SMTP class for the duration of the test."""
mock_smtp = mocker.patch("myapp.email.smtplib.SMTP")
instance = mock_smtp.return_value
instance.sendmail.return_value = {}
send_email("alice@example.com", "subject", "body")
mock_smtp.assert_called_once_with("smtp.example.com", 587)
instance.sendmail.assert_called_once()
instance.quit.assert_called_once()
Patch a method on a specific instance
def test_instance_patch(mocker):
"""Patch one method on one instance, not the class globally."""
client = APIClient(base_url="https://api.example.com")
mocker.patch.object(client, "get", return_value={"status": 200})
assert client.get("/") == {"status": 200}
Patch with side effects
def test_side_effect(mocker):
"""Each call returns the next value; iterables also work."""
mock_fetch = mocker.patch("myapp.api.fetch")
mock_fetch.side_effect = [{"a": 1}, {"a": 2}, ConnectionError]
assert myapp.api.fetch()["a"] == 1
assert myapp.api.fetch()["a"] == 2
with pytest.raises(ConnectionError):
myapp.api.fetch()
MagicMock and spec
A MagicMock is a callable object that accepts any method call and returns another MagicMock. Use spec= to constrain it to a real class's interface — calls to nonexistent methods raise AttributeError instead of silently returning a fresh mock.
from unittest.mock import MagicMock
def test_strict_mock(mocker):
real_session = mocker.MagicMock(spec=requests.Session)
real_session.get.return_value.status_code = 200
real_session.nonexistent() # AttributeError — real Session has no such method
Debugging tests with pdb
Pair pytest with the debugger to land inside a failing test with the full call stack and fixture values. The two essential flags are --pdb (drop into pdb on first failure) and -s (disable output capture so the prompt is visible). See sections/python/pdb for the debugger command reference.
# Drop into pdb on first failure
pytest --pdb -s
# Drop into pdb at the start of every test (not on failure)
pytest --trace -s
# Re-run only the failed tests, drop into pdb
pytest --lf -x --pdb -s
# Use ipdb instead of stock pdb
PYTHONBREAKPOINT=ipdb.set_trace pytest -s
Output:
============================= test session starts ==============================
tests/test_orders.py::test_checkout_discount FAILED [100%]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/alice/code/myapp/src/orders.py(58)apply_discount()
-> assert pct <= 1.0
(Pdb)
def test_with_breakpoint():
"""An explicit breakpoint inside the test — pytest -s required."""
x = compute()
breakpoint()
assert x == expected
Always pass
-swith--pdb. Without it pytest captures stdout/stderr and the(Pdb)prompt is invisible — you'll think the tests hung.
Async tests
Modern pytest plus pytest-asyncio handles async def tests natively. With asyncio_mode = "auto" in your config, any async def test_* is dispatched on an event loop automatically. The event loop is rebuilt per-test by default; use loop_scope="session" for shared-loop tests.
# test_async.py
import pytest
import asyncio
async def test_concurrent_fetches():
results = await asyncio.gather(
fetch("/a"),
fetch("/b"),
fetch("/c"),
)
assert len(results) == 3
@pytest.mark.asyncio(loop_scope="session")
async def test_with_session_loop():
"""This test reuses the session-wide event loop."""
...
For libraries that aren't asyncio-native (Trio, AnyIO), pytest-trio and pytest-anyio provide equivalent integration.
Running tests programmatically
Embed pytest invocation in scripts via pytest.main(). Returns the exit code. Useful for self-test sub-commands and CLI test runners.
# scripts/run_tests.py
import sys
import pytest
def main() -> int:
return pytest.main(["-v", "--cov=src", "tests/"])
if __name__ == "__main__":
sys.exit(main())
Common pitfalls
- Test file naming — pytest only discovers
test_*.pyor*_test.py. Rename or setpython_filesin config. - Mutable shared fixtures —
scope="module"orscope="session"fixtures are shared. Mutating them in one test affects others. Default tofunctionscope unless you specifically want sharing. -sis needed with--pdb— without it, the(Pdb)prompt is captured and invisible.autousefixtures hide dependencies — they apply silently. Test bodies don't show they depend on the fixture. Use sparingly.conftest.pylocation — fixtures intests/integration/conftest.pyare only visible to tests under that directory. Hoist to the rootconftest.pyto share project-wide.asyncio_mode = "auto"vs"strict"— strict requires@pytest.mark.asyncioon every async test. Auto picks them up by signature. Strict is safer in mixed sync/async repos.- Test-order dependence — tests passing in order but failing under
pytest-randomlyindicates shared state. Fix the shared state, don't disable randomisation. pytest.fixture(scope="session")evaluated even when no test uses it — actually, fixtures are lazy; they only build on first reference. But once built, they live until session end.mocker.patch("x.y")patches the use site, not the definition — patch the module whereyis imported into, not where it's defined.myapp.email.smtplib.SMTP(used in email.py), notsmtplib.SMTP.--covslows tests substantially — coverage instrumentation adds 10–30% overhead. Run coverage in CI only, not in dev loop.xfail(strict=True)flips on success — a test that "unexpectedly" passes is reported as a failure under strict xfail. Usestrict=Falseif you're hunting for the fix and don't want CI to flap.- Test discovery via
__init__.py— pytest's "rootdir" detection differs based on__init__.pyplacement. If discovery is misbehaving, setpythonpathandtestpathsinpyproject.toml.
Real-world recipes
Bootstrap pytest on a new project
cd ~/code/myproject
pip install pytest pytest-cov pytest-mock pytest-asyncio
mkdir -p tests
cat >> pyproject.toml <<'EOF'
[tool.pytest.ini_options]
minversion = "8.0"
testpaths = ["tests"]
addopts = ["-ra", "--strict-markers", "--showlocals"]
asyncio_mode = "auto"
markers = [
"slow: tests that take more than a second",
"integration: requires external services",
]
[tool.coverage.run]
source = ["src"]
branch = true
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"if TYPE_CHECKING:",
"raise NotImplementedError",
]
EOF
cat > tests/test_smoke.py <<'EOF'
def test_truth():
assert 1 + 1 == 2
EOF
pytest
Output:
============================= test session starts ==============================
collected 1 item
tests/test_smoke.py::test_truth PASSED [100%]
============================== 1 passed in 0.02s ===============================
Re-run only failed tests, debug interactively
The fastest debug loop for a failing PR: re-run failures, stop on first, drop into pdb.
pytest # full run; some fail
pytest --lf -x --pdb -s # only failed, stop on first, into pdb
# fix the bug
pytest --lf # confirm; should be green
pytest # full re-run before push
Output:
=========================== short test summary info ============================
FAILED tests/test_orders.py::test_checkout_total - AssertionError
================== 1 failed, 234 passed, 12 skipped in 4.21s ===================
Coverage gating in CI
Fail the build below 80% line coverage. Pair with --cov-branch for branch coverage too.
- run: pip install pytest pytest-cov -r requirements.txt
- run: pytest --cov=src --cov-report=xml --cov-report=term --cov-fail-under=80
- uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
Output:
TOTAL 155 4 97%
Required test coverage of 80% reached. Total coverage: 97.42%
Test against a matrix of Python versions
GitHub Actions matrix to run tests on every supported Python.
strategy:
fail-fast: false
matrix:
python: ["3.10", "3.11", "3.12", "3.13"]
os: [ubuntu-latest, macos-latest]
jobs:
test:
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "${{ matrix.python }}" }
- run: pip install -e ".[dev]"
- run: pytest -n auto --cov=src
Parametrized test for many invariants
When a single function has many input/output pairs to verify, use a single parametrized test rather than copy-pasting test functions. The pytest.param wrapper attaches an id and optional marks.
import pytest
from decimal import Decimal
@pytest.mark.parametrize(
"items, expected_total",
[
pytest.param([], Decimal("0.00"), id="empty"),
pytest.param([("apple", 1.50)], Decimal("1.50"), id="one"),
pytest.param([("apple", 1.50), ("pear", 0.75)], Decimal("2.25"), id="two"),
pytest.param(
[("apple", 1.50)] * 1000,
Decimal("1500.00"),
id="thousand",
marks=pytest.mark.slow,
),
],
)
def test_total(items, expected_total):
assert compute_total(items) == expected_total
pytest -v
Output:
test_total.py::test_total[empty] PASSED
test_total.py::test_total[one] PASSED
test_total.py::test_total[two] PASSED
test_total.py::test_total[thousand] PASSED
Mock external HTTP in a FastAPI test
Combine FastAPI's TestClient with pytest-mock to test handlers without hitting real services.
from fastapi.testclient import TestClient
from myapp import app
def test_user_endpoint(mocker):
mock_db = mocker.patch("myapp.api.users.fetch_user")
mock_db.return_value = {"id": 42, "name": "Alice Dev"}
client = TestClient(app)
response = client.get("/users/42")
assert response.status_code == 200
assert response.json() == {"id": 42, "name": "Alice Dev"}
mock_db.assert_called_once_with(42)
Property-based fuzzing of a parser
Hypothesis generates random inputs and shrinks failures. Pair with @settings to bound the run time.
from hypothesis import given, settings, strategies as st
@given(st.text(min_size=1, max_size=100))
@settings(max_examples=200, deadline=500)
def test_parse_never_crashes(s):
"""The parser must handle arbitrary text without raising."""
try:
parse(s)
except ParseError:
pass # expected for invalid input
except Exception as e:
pytest.fail(f"unexpected error on {s!r}: {e}")
Speed up CI with xdist
Parallel test runs cut suite time near-linearly. Use --dist=loadfile for test classes/files with shared expensive fixtures so they stay in the same worker.
pip install pytest-xdist
pytest -n auto --dist=loadfile --cov=src --cov-report=xml
Output:
============================= test session starts ==============================
plugins: xdist-3.6.1, cov-5.0.0
8 workers [248 items]
............................................................ [ 24%]
............................................................ [ 48%]
............................................................ [ 72%]
..................................................................... [100%]
---------- coverage: platform linux, python 3.12.3 ----------
Coverage XML written to file coverage.xml
============================== 248 passed in 9.74s =============================
Capture log output and assert on it
caplog captures stdlib logging records. With propagate_logs=True (default in recent pytest), loguru-based applications also surface here when configured to route to stdlib (see sections/python/loguru for the bridge).
import logging
def test_warns_on_retry(caplog):
with caplog.at_level(logging.WARNING):
result = retry_op(max_attempts=3)
warnings = [r for r in caplog.records if r.levelname == "WARNING"]
assert len(warnings) == 2 # two retries before success
assert "retry 1" in caplog.text
Custom marker that requires a flag
A test marked @pytest.mark.expensive should not run unless --run-expensive is passed. Skip it by default; opt in explicitly.
# conftest.py
def pytest_addoption(parser):
parser.addoption("--run-expensive", action="store_true", default=False,
help="run expensive tests")
def pytest_collection_modifyitems(config, items):
if config.getoption("--run-expensive"):
return
skip_expensive = pytest.mark.skip(reason="need --run-expensive option to run")
for item in items:
if "expensive" in item.keywords:
item.add_marker(skip_expensive)
pytest # skips expensive
pytest --run-expensive # includes them
Output:
============================= test session starts ==============================
collected 42 items
tests/test_unit.py ...................................... [ 95%]
tests/test_expensive.py ss [100%]
========================= 40 passed, 2 skipped in 0.84s ========================
collected 42 items
tests/test_unit.py ...................................... [ 95%]
tests/test_expensive.py .. [100%]
============================== 42 passed in 14.21s =============================
See also
sections/python/pdb— full debugger command reference; pair withpytest --pdb -s.sections/python/loguru— structured logging;caplogcan capture loguru output when bridged to stdlib.sections/python/mypy— type-check tests too via[[tool.mypy.overrides]] module = "tests.*".sections/python/pre-commit— wire a fast pytest subset as apre-pushhook.sections/python/pyproject-toml— the home of[tool.pytest.ini_options].