BenchmarkDotNet

Visit Website

Introduction

BenchmarkDotNet is a .NET library that helps you transform methods into benchmarks, track their performance and share reproducible measurement experiments. The README says it is “no harder than writing unit tests”. It lives at dotnet/BenchmarkDotNet under the MIT licence.

Micro-benchmarking is easy to get wrong: JIT warm-up, dead-code elimination, garbage collection and noisy machines all distort timings. BenchmarkDotNet handles warm-up, pilot runs, iteration counts and outlier removal for you, and the README credits the perfolizer and pragmastat statistical engines for reliable, precise results.

It is used widely across the .NET ecosystem, and the project is part of the .NET Foundation organisation on GitHub.

In the quality loop, BenchmarkDotNet belongs next to unit tests. A unit test proves a method is correct; a benchmark proves it is fast enough and does not allocate more than it should. Teams often keep a benchmarks project alongside their test projects and run it before merging performance-sensitive changes, or on a schedule on dedicated hardware to track trends.

Features

1. Attribute-driven: Mark methods with [Benchmark] and run them with BenchmarkRunner.Run<T>().
2. Statistics: Automatic warm-up, iteration selection, outlier detection and summary statistics.
3. Parameters: [Params] and argument sources run the same benchmark over several inputs.
4. Baselines: Mark one method as the baseline to see relative ratios.
5. Jobs and runtimes: Compare across .NET runtimes, JIT settings and platforms in one run.
6. Diagnosers: Memory allocation, threading and hardware-counter diagnosers add context to timings.
7. Exporters: Markdown, HTML, CSV and other report formats for sharing.
8. Protection from mistakes: Guards against common benchmarking errors, as the README states.

Pros

1. De facto standard for .NET: Familiar to most .NET performance engineers.
2. Reproducible: Reports capture environment details for comparison.
3. MIT licence: Free for any use.
4. Active: Commits on the day of review; about 11,500 stars.
5. Low ceremony: Feels like writing a unit test.

Cons

1. Micro, not load: It measures code paths, not system throughput under concurrent users.
2. Slow runs: Accurate measurement takes time; full suites can run for many minutes.
3. Needs release builds: Results are only meaningful in optimised builds on quiet machines.
4. CI noise: Shared CI runners add variance; trend comparisons need care.
5. .NET only: Other platforms need other harnesses, such as JMH for the JVM.

Usage and Links (Example)

A minimal benchmark:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

public class Md5VsSha256 {
    [Benchmark(Baseline = true)] public byte[] Md5() => /* ... */ null;
    [Benchmark] public byte[] Sha256() => /* ... */ null;
}

BenchmarkRunner.Run<Md5VsSha256>();

Run it with dotnet run -c Release and read the summary table and exported reports.

Useful options when you go beyond the basics:

1. Add [MemoryDiagnoser] to the class to report allocated bytes and GC collections per operation.
2. Use [Params(10, 100, 1000)] on a field to see how cost scales with input size.
3. Use [SimpleJob(RuntimeMoniker.Net80)] and similar attributes to compare runtimes side by side.
4. Export results (for example GitHub-flavoured Markdown) and attach them to the pull request so reviewers see the before and after numbers.

Keep benchmarks small and focused, avoid I/O inside measured code, and return values from benchmark methods so the JIT cannot remove the work.

Resources:

• BenchmarkDotNet website
• GitHub repository

Pricing

BenchmarkDotNet is free and open source under MIT.

Recommended for?

1. .NET library and runtime developers comparing implementations.
2. Teams guarding hot paths against performance regressions.
3. Engineers writing performance-focused pull requests that need evidence.
4. Not a fit for load testing services; use NBomber, k6 or JMeter for that.

Conclusion

BenchmarkDotNet is the standard micro-benchmarking harness for .NET, with sound statistics and rich diagnostics.

Run it on stable hardware in release mode, and use load-testing tools for end-to-end throughput.