CodSpeed

Visit Website

Introduction

CodSpeed brings continuous performance testing and autonomous performance optimisation into pull requests. Instead of engineers profiling by hand or discovering slowdowns after release, benchmarks run in CI on every change and results appear on the PR, with automated performance gating that can block merges which degrade execution speed.

Its core technical claim is measurement quality. Benchmarks on shared CI runners are notoriously noisy, and noise trains engineers to ignore alerts. CodSpeed’s instruments collect high-precision tracing data and filter out system jitter, automatically isolating relevant performance data to bring benchmarking variance below 1%, so results do not need manual re-runs or second-guessing. A wall-time instrument and macro runners (dedicated runners billed by the minute) cover workloads where real elapsed time matters.

When something does get slower, AI-enhanced differential flamegraphs show exactly which lines of code got slower and what changed, so you can jump straight to the fix rather than digging through logs. The newest piece is the Optimization Agent: it takes the root cause found, applies a fix, benchmarks the result and opens a pull request for review. CodSpeed’s homepage shows an agent-opened PR reporting a 31% speed-up.

CodSpeed supports Rust, Python, Node.js and more, plugging into the benchmark frameworks those ecosystems already use, so you write ordinary benchmarks and CodSpeed handles running and comparing them. Testimonials on the site include Astral, which says CodSpeed has become a key part of the development loop for its toolchain (Ruff, uv and ty).

Where does it fit? Unit tests stop functional regressions; CodSpeed does the same for performance at the code level. It complements load-testing tools such as k6 or Gatling, which test whole systems under traffic, and browser tools like Lighthouse, which measure page experience. For libraries, compilers, parsers, data pipelines and hot server paths, PR-level benchmarking catches regressions when they are cheapest to fix.

Adoption is usually incremental: wrap a handful of critical-path functions in benchmarks, enable the CI integration, and let CodSpeed build a baseline on the main branch before you turn on merge gating. Then widen coverage as the team learns which benchmarks are meaningful. Whatever tool you choose, benchmark results are only useful if they are stable. Pin the hardware or runner, run enough iterations to make variance visible, and treat a single noisy result as a reason to rerun, not to block a merge.

Choose benchmarks that represent real work, such as parsing a typical file or handling a typical request, rather than trivial loops. Benchmarks that match production paths make regressions meaningful and make the Optimization Agent’s suggestions worth reviewing.

Review the flamegraph diff even when a PR improves performance: it confirms that the speed-up came from the intended change rather than from skipped work.

Teams new to continuous benchmarking can learn a lot from simply watching the baseline for a week before acting on any alerts.

Features

1. Continuous benchmarking: Benchmarks on every PR.
2. Low variance: Measurement variance below 1%.
3. Differential flamegraphs: AI-enhanced, line-level diffs.
4. Performance gating: Block regressing merges.
5. Optimization Agent: Fixes, benchmarks and opens PRs.
6. Wall-time instrument: Real elapsed-time measurements.
7. Macro runners: Dedicated runners for larger workloads.
8. Multi-language: Rust, Python, Node.js and more.
9. PR reports: Results where reviewers look.

Pros

1. Free plan: Free tier with wall-time runner minutes.
2. Trusted by major OSS: Used by the Astral toolchain.

Cons

1. Benchmarks still need writing: You must define what to measure.
2. Runner minutes: Macro runner usage is metered.
3. Code-level focus: Not a load-testing tool.

Usage and Links (Example)

1. Write benchmarks with a framework CodSpeed supports for your language (see its docs for Rust, Python, Node.js and others).
2. Add the CodSpeed step to your CI workflow and connect the repository.
3. Let the main branch build a baseline.
4. Turn on performance gating and review differential flamegraphs on PRs.

Resources:

• CodSpeed website

Pricing

CodSpeed has a $0 per user per month plan that includes 600 wall-time macro runner minutes a month, and a paid plan from $15 per user per month; macro runner usage beyond 600 minutes is $0.032 a minute. Enterprise plans are available.

Recommended for?

1. Library and tooling authors where speed is a feature.
2. Backend teams protecting hot paths.
3. Teams experimenting with AI optimisation agents.

Conclusion

CodSpeed is the most polished way to make performance a PR-level check, and its low-variance measurement solves the main problem with CI benchmarks.

Start with a few critical benchmarks and gate only once the baseline is stable.

On CheckOps, CodSpeed is filed under the Performance & load, Release & CI gates and Unit testing stages. Related tools listed here include BenchmarkDotNet, hyperfine, JMH, K6, Lighthouse CI and SiteSpeed.io. Before standardising on one, compare recent maintenance activity, the pricing model as your usage grows, and the integrations you rely on today. Information on this page was verified against the official website, documentation and repository on 11 November 2026.