Commit Graph
2 Commits
Author SHA1 Message Date
Chandler CarruthandDana Jansens c50e21695f Allow detecting benchmark dry runs and reduce benchmark test costs (#8013)
We use the dry-run mode for running benchmarks in a test. This is
important, but doesn't need to use large batches of files. Doing so
would cause the compilation benchmark to become excessively slow,
especially on GitHub runners.

Sadly, Google Benchmark doesn't expose its dry-run mode, but we already
have our own `main` function that injects custom logic for our
benchmarks. Extend that to detect the dry run flags and environment
variables so that we can customize the benchmark behavior in that mode.
Then we use that detection to reduce the batch size.

Also cleans up the counters computation to both be a bit better
organized (I think), and to avoid keeping the tokenized buffers we build
to count tokens around during the timed running of the benchmark.

Also cleans up a bug in some clamping code spotted while fixing this.

Assisted-by: Antigravity with Gemini

---------

Co-authored-by: Dana Jansens <danakj@orodu.net>
2026-10-22 18:00:21 +00:00
Chandler Carruth 632a81fcb6 Keep benchmark values passed to DoNotOptimize in registers (#7870)
Clang implements the `"+r,m"` constraint in `benchmark::DoNotOptimize`
by always choosing memory, so each call stores the value to the stack
and loads it back. Most of our benchmarks call it on a loop counter or
another value carried to the next iteration, which puts that store and
reload on the loop's critical path.

On an Apple M1, the cost of that round trip depends on which register
the compiler uses to address the stack slot, and unrelated code changes
move it. Changing only that register from `x29` to `sp`, with the same
address, made `BM_SetLookupHitPtr<Set<int>>` 41.5% to 49.1% slower.

Add `Carbon::Testing::DoNotOptimize` in
`//testing/base:benchmark_helpers`, and switch every benchmark to it. It
only accepts types it can keep in registers: integers, enums, and
pointers go into a `"+r"` constraint with a `"memory"` clobber, and
containers pass their `data()` pointer to that form, so the compiler
must assume the call reads and writes their contents. Any other type is
a compile error. The container form doesn't block optimizing based on a
container's size; passing the container's address does.

Benchmarks that pass a loop counter or carried value no longer measure a
store and reload per iteration, so their numbers aren't comparable with
earlier runs. For example, the hashing latency benchmarks no longer
include one in each hash's latency.

Assisted-by: Claude Code
2026-10-02 00:15:21 +00:00