Files
carbon-lang/toolchain
Chandler Carruthandjosh11b 9319bb0369 Add more complex function definition generation and an impl-file benchmark (#7998)
Previously, we only had the dense API declarations benchmark. This adds
a benchmark that aims to emulate what a realistic implementation file in
Carbon would look like, based on what `.cpp` files look like in Carbon,
Clang, and LLVM. It's still a little artificial in order to provide
stability, but it should give us coverage of compile time performance of
statements and function definitions at more realistic scales.

In order to generate more interesting and longer function definitions,
this introduces a plan-based approach to guide how each feature of a
definition is produced. Initially, it identifies these as roughly:

- local variables
- updates to variables
- `if`, `else`, `while` blocks
- "guard" statements -- conditions with a return, break, or continue
- read of a parameter

The plan also manages the nesting depth of these constructs.

Because the plan establishes how many of these constructs exist in a
function definition, and at what nesting depth, the emission can
randomize the order and arrangement without changing the size of the
function. This follows the pattern in all our benchmark source
generation of getting byte-identical stability while avoiding
predictable structures. Technically, I think the current approach goes a
bit too far because function definitions I suspect are somewhat
predictable in their structure in practice. But a conservative benchmark
I think is a good starting point. =]

The target distributions for definition sizes, maximum depths, and
statement mixes come from a very rough Claude-powered analysis of 10,697
function definitions in 533 Carbon toolchain, LLVM, and Clang
implementation files. These were analyzed and classified by Claude using
a one-off Python script and libclang. I haven't reviewed any of the
analysis carefully, the resulting distributions look plausible enough as
a starting point. Comparing real code against generated C++ from the
plans, each number per-definition average:

| statements | definitions | max depth   | ifs         | loops       |
|------------|-------------|-------------|-------------|-------------|
| 7-9        | 10.9 / 11%  | 1.25 / 1.23 | 1.74 / 1.86 | 0.23 / 0.15 |
| 14-19      | 7.3 / 7%    | 1.89 / 1.88 | 3.53 / 3.71 | 0.61 / 0.42 |
| 28-39      | 3.3 / 3%    | 2.76 / 2.75 | 7.75 / 8.46 | 1.26 / 1.27 |
| 60+        | 1.5 / 2%    | 4.03 / 3.80 | 25.0 / 27.1 | 3.72 / 4.04 |

To maintain stability, every definition has an accumulator, a statement
reading its parameters, and a `return`, so real definitions of one to
three statements become three.

Note that updates are currently also used instead of calls to other
functions -- a later PR will switch those to calls so that we form more
plausible call graphs. In general, subsequent PRs will refine and expand
the plan to generate increasingly more realistic function definitions.

There are probably lots of improvements that could be made here, both to
generate higher quality definitions and to improve the generation logic
to be more efficient, etc. But already this has allowed _much_ more
extensive benchmarking of Carbon's compile-time performance so it seems
likely a reasonable increment.

Assisted-by: Claude Code and Antigravity with Gemini

---------

Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
2026-10-21 23:02:26 +00:00
..
2026-10-15 20:15:28 +00:00
2026-10-20 23:21:00 +00:00
…