Bencher

Visit Website

Introduction

Bencher is a continuous benchmarking platform that helps engineers and agents catch performance regressions before they merge. Its argument starts from what most teams do: run benchmarks on GitHub Actions with a comparison script. That breaks down because shared CI runners are noisy: shared CPU, memory contention and scheduler jitter mean two identical runs can differ wildly, and when benchmarks cry wolf on every PR, engineers stop looking.

The examples it gives of costly regressions are concrete: a database regression on a hot path paging someone, a compiler regression silently slowing every downstream build, a browser engine shipping a 10% paint regression, or a crypto library adding a microsecond to a handshake and breaking an SLA. Local benchmarks are not reproducible either, because each check means pulling the baseline branch and waiting.

Bencher’s workflow has three steps. Run: execute benchmarks locally or in CI on the exact same bare-metal runners, using your favourite benchmarking tools; the bencher CLI orchestrates the run and stores results. Track: monitor, query and graph results over time in the web console by branch, testbed, benchmark and measure. Catch: detect regressions with customisable statistical thresholds before they merge, locally or in CI, on the same hardware.

Benchmark harness adapters parse the output of popular tools, such as Rust’s Criterion (rust_criterion), JSON output and a “magic” adapter that auto-detects formats. PR comments show each benchmark’s result, its change, the threshold boundary and whether an alert fired, with links to plots and alerts.

Bencher is open source and can be self-hosted, including a self-hosted GitHub App and OAuth options, or used as Bencher Cloud. The project’s own analogy is useful: for the same reason you run unit tests to prevent feature regressions, you should run benchmarks to prevent performance regressions.

Bare Metal is what separates Bencher from a simple results dashboard: if your current CI benchmarks are too noisy to trust, moving them to dedicated, identical machines is often a bigger improvement than any statistical trick. Whatever tool you choose, benchmark results are only useful if they are stable. Pin the hardware or runner, run enough iterations to make variance visible, and treat a single noisy result as a reason to rerun, not to block a merge.

The source code is at bencherdev/bencher on GitHub under the Apache-2.0 or MIT (outside its “plus” directories) licence, with about 929 stars; the most recent commit was on 10 November 2026.

Bencher’s public project pages are also a transparency tool: open-source maintainers can show users a history of performance across releases, which builds trust and makes regressions visible to contributors.

Thresholds deserve care. Start with generous boundaries, watch how much results vary on your chosen testbed, and tighten them gradually so alerts stay rare and meaningful.

For teams without dedicated hardware, the cloud service is the easiest way to see whether bare-metal runs make results noticeably more stable than shared CI runners.

Features

1. Run, track, catch: End-to-end benchmarking workflow.
2. Bare-metal runners: Identical hardware locally, in sandboxes and in CI.
3. Harness adapters: Criterion, JSON and auto-detecting adapters.
4. Statistical thresholds: Customisable regression detection.
5. PR reports and alerts: Results with boundaries in PRs.
6. Web console: Plots by branch, testbed and measure.
7. CLI: bencher run orchestration.
8. Self-hosting: Self-hosted server and GitHub App.

Pros

1. Language-agnostic: Works with existing benchmark tools.
2. Noise solution: Bare metal tackles noisy runners.
3. Free to use: Open source under the Apache-2.0 or MIT (outside its “plus” directories) licence, so the core tool costs nothing to use.
4. Actively maintained: The repository had commits as recent as 10 November 2026.

Cons

1. Bare Metal is paid: Dedicated hardware costs money.
2. Mixed licensing: Some “plus” features are under a separate licence.
3. Setup effort: Thresholds need tuning.

Usage and Links (Example)

bencher run \
  --project my-project \
  --branch main \
  --testbed intel-v1 \
  --spec intel-v1 \
  --adapter rust_criterion \
  --error-on-alert \
  "cargo bench"

In GitHub Actions, install the CLI and run bencher run with your benchmark command and API token.

Resources:

• Bencher website
• GitHub repository

Pricing

Bencher is open source and can be self-hosted. Bencher Cloud and Bare Metal runners are offered as paid services; see bencher.dev/pricing for current plans.

Recommended for?

1. Teams whose CI benchmarks are too noisy to trust.
2. Open-source projects wanting public performance history.
3. Organisations needing self-hosted benchmarking.

Conclusion

Bencher is the leading open-source continuous benchmarking platform, and Bare Metal addresses noise at the root.

Start with the free tier, then move critical benchmarks to bare metal.

On CheckOps, Bencher is filed under the Performance & load, Release & CI gates and Unit testing stages. Related tools listed here include BenchmarkDotNet, hyperfine, JMH, K6, Lighthouse CI and cargo-nextest. Before standardising on one, compare recent maintenance activity, the pricing model as your usage grows, and the integrations you rely on today. Information on this page was verified against the official website, documentation and repository on 11 November 2026.