pytest-benchmark

Visit Website

Introduction

pytest-benchmark adds a benchmark fixture to pytest. The fixture is a callable that benchmarks any function passed to it, grouping the work into rounds calibrated to the timer’s precision, with sensible defaults for micro-benchmarks.

It integrates tightly with pytest, so you should know pytest basics first. It benchmarks functions only, can store results and compare against previous runs, and offers benchmark.pedantic for precise control over rounds, iterations and setup. Results can be exported as JSON for CI dashboards.

Whatever tool you choose, benchmark results are only useful if they are stable. Pin the hardware or runner, run enough iterations to make variance visible, and treat a single noisy result as a reason to rerun, not to block a merge.

The source code is at ionelmc/pytest-benchmark on GitHub under the BSD-2-Clause licence, with about 1,500 stars; the most recent commit was on 1 November 2026.

Features

1. benchmark fixture: Benchmark any callable.
2. Automatic calibration: Rounds sized to timer precision.
3. Pedantic mode: Control rounds and setup.
4. Comparisons: Compare saved runs.
5. Regression failure: Fail on slowdowns.
6. JSON export: For CI tools.

Pros

1. Free to use: Open source under the BSD-2-Clause licence, so the core tool costs nothing to use.
2. Actively maintained: The repository had commits as recent as 1 November 2026.

Cons

1. Function-level only: Not for end-to-end timing.
2. Noisy on shared CI: Needs stable runners.

Usage and Links (Example)

pip install pytest-benchmark

def test_my_stuff(benchmark):
    result = benchmark(sum, range(1000))
    assert result == 499500

pytest --benchmark-compare   # compare with a previously saved run

Resources:

• pytest-benchmark website
• GitHub repository

Pricing

pytest-benchmark is free and open source under the BSD-2-Clause licence.

Recommended for?

1. Python developers adding performance checks to existing pytest suites.

Conclusion

pytest-benchmark is the easiest way to benchmark Python code.

Store baselines and compare on every change.

On CheckOps, pytest-benchmark is filed under the Performance & load and Unit testing stages. Related tools listed here include pytest, pytest-xdist, hyperfine, Locust, Coverage.py and Hypothesis. A short trial on a real project, with your own code, data and pipeline, is usually the fastest way to choose between them, because fit matters more than feature lists. Facts here come from the official site, documentation and repository, checked on 11 November 2026.