Avoid passing -unwindlib on macOS where it isn't needed (#7825)

Removes some warnings from our links on macs when the flag is ignored.

.agents/skills/agent_tools/SKILL.md

@@ -0,0 +1,45 @@
+---
+name: Agent tools
+description:
+    Guidelines and restrictions on shell commands and file manipulation tools
+    for AI assistants.
+---
+
+# Agent tools
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+AI assistants working on the Carbon repository **MUST NOT** use legacy or
+generic UNIX shell search/edit commands when specialized environment tools
+exist.
+
+## Command line tools restrictions
+
+-   **DO NOT USE**: `cat`, `less`, `grep`, `sed`, or other shell utilities for
+    viewing, searching, or modifying files.
+-   **DO NOT USE**: `patch` to write and apply patch files.
+-   **DO NOT USE**: Writing custom scripts in other languages to circumvent this
+    limitation.
+-   **DO USE**: High-fidelity semantic API tools:
+    -   **Viewing**: Use `view_file` instead of `cat` / `less`.
+    -   **Searching**: Use `grep_search` / `find_by_name` instead of `grep` /
+        `find`.
+    -   **Modifying**: Use `replace_file_content`, `multi_replace_file_content`,
+        or `write_to_file` instead of `sed` / `patch` / `python` edits.
+
+You may only write and run temporary programs to modify source code if no
+semantic tool is applicable or when performing complex, systematic transforms
+across many codebase directories simultaneously.
+
+## Temporary files management
+
+Temporary files and scratchpad test scripts created by the assistant during
+analysis, experiments, or debugging:
+
+-   **MUST** reside within the `tmp/` subdirectory under the workspace root.
+-   **MUST** be periodically cleaned out and deleted before ending your turn to
+    preserve a clean git workspace.

.agents/skills/bazel/SKILL.md

@@ -1,8 +1,8 @@
 ---
 name: Bazel usage
 description:
-    Instructions for using Bazel or Bazelisk to build, test, and debug in the
-    Carbon repository.
+    Instructions that **MUST** be followed when using Bazel or Bazelisk to
+    build, test, and debug in the Carbon repository.
 ---
 
 # Bazel usage
@@ -62,7 +62,8 @@ project uses Bazelisk.
 
 You can run the Carbon driver or command line directly via Bazel:
 
--   `bazelisk run //toolchain -- compile --phase=parse toolchain/parse/testdata/basics/empty.carbon`
+-   `bazelisk run //toolchain -- compile --phase=parse
+    toolchain/parse/testdata/basics/empty.carbon`
 
 ## Advanced configurations
 

.agents/skills/builtins/SKILL.md

@@ -0,0 +1,276 @@
+---
+name: Builtin functions
+description:
+    Instructions for registering, mapping, constant evaluating, and lowering
+    builtin functions in the Carbon toolchain.
+---
+
+# Builtin Functions in the Carbon Toolchain
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+Builtin functions are compiler-recognized primitives mapping directly from
+Carbon code expressions (via standard prelude bindings) to optimized backend
+execution. This document defines the complete structural workflow, C++ patterns,
+constant evaluation logic, machine lowering mechanics, library bindings, and
+validation strategies required to implement builtin functions in the Carbon
+compiler.
+
+---
+
+## Technical Flow & Lifecycle
+
+```mermaid
+graph TD
+  Src[Carbon Source Code] -->|Prelude Map| Sem[Semantic Analysis / SemIR]
+  Sem -->|Signature Constraint| Sig[builtin_function_kind.cpp]
+  Sem -->|Phase Evaluation| Eval[eval.cpp Constant Interpreter]
+  Sem -->|Machine Codegen| Lower[handle_call.cpp LLVM Lowering]
+  Eval -->|Diagnostics| Diag[diagnostics/kind.def]
+  Lower -->|Native Instructions| LLVM[LLVM IR Generation]
+```
+
+Adding a builtin function involves a 5-step integration:
+
+1.  **Define the Builtin Kind**: Register the enum in
+    [builtin_function_kind.def](../../../toolchain/sem_ir/builtin_function_kind.def).
+2.  **Signature & Compile-Time Registry**: Declare the mapping name, parameter
+    constraints, and compile-time evaluation residency in
+    [builtin_function_kind.cpp](../../../toolchain/sem_ir/builtin_function_kind.cpp).
+3.  **Compile-Time Interpreter Support**: Wire constant evaluation hooks and
+    bounds/exception diagnostics in
+    [eval.cpp](../../../toolchain/check/eval.cpp).
+4.  **LLVM IR Lowering Support**: Connect target machine generation in
+    [handle_call.cpp](../../../toolchain/lower/handle_call.cpp).
+5.  **Prelude Library Mapping**: Bind primitive interfaces to named builtins
+    under [core/prelude/](../../../core/prelude/).
+
+---
+
+## Detailed Step-by-Step Implementation Guide
+
+### Step 1: Kind Definition & Registration
+
+Register your builtin function name using the X-macro in
+[builtin_function_kind.def](../../../toolchain/sem_ir/builtin_function_kind.def):
+
+```cpp
+// toolchain/sem_ir/builtin_function_kind.def
+
+// Converts an integer type to a floating-point type.
+CARBON_SEM_IR_BUILTIN_FUNCTION_KIND(IntConvertFloat)
+```
+
+### Step 2: Signature Validation & Compile-Time Residence
+
+Inside
+[builtin_function_kind.cpp](../../../toolchain/sem_ir/builtin_function_kind.cpp):
+
+1.  **Define Parameter Constraints**: If the parameter requires novel
+    constraints (e.g. "must be a float type"), define a template constraint
+    struct checking the matching `SemIR` type instruction (such as `FloatType`
+    or `FloatLiteralType`). Use pre-established semantic helpers:
+
+    -   `TypeParam<I, T>`: Ensures different parameters resolve to identical
+        type structures (e.g., generic constraint matching).
+    -   `AnyInt`, `AnyFloat`, `AnySizedInt`, `AnySizedFloat`, `CharCompatible`,
+        `StdInitializerList`, `NoReturn`.
+
+2.  **Map Literal Name & Register Constraint Signature**: Declare a
+    `BuiltinInfo` constant inside `namespace BuiltinFunctionInfo` matching the
+    macro-defined name:
+
+    ```cpp
+    // toolchain/sem_ir/builtin_function_kind.cpp
+
+    constexpr BuiltinInfo IntConvertFloat = {
+        "int.convert_float", ValidateSignature<auto(AnyInt)->AnyFloat>};
+    ```
+
+3.  **Establish Compile-Time Residency Status**: Update
+    `BuiltinFunctionKind::IsCompTimeOnly` to determine if a call requires
+    compile-time evaluation:
+    -   **Checked/Diagnostics Primitives**: Return `true` immediately. Runtime
+        lowering of these is illegal (e.g. `IntConvertFloatChecked`).
+    -   **Runtime Primitives**: Return
+        `AnyLiteralTypes(sem_ir, arg_ids, return_type_id)` to enforce that
+        expressions involving unsized literal values (like `IntLiteral` or
+        `FloatLiteral`) are evaluated exclusively at compile-time (as they lack
+        runtime representation).
+
+---
+
+### Step 3: Constant Evaluation Support
+
+Wire the interpreter inside [eval.cpp](../../../toolchain/check/eval.cpp) to
+execute compile-time computations:
+
+1.  **Implement Constant Evaluation Logic**:
+
+    -   Handle the builtin case inside `MakeConstantForBuiltinCall` (which
+        processes the compile-time execution of the call).
+    -   Confirm type validation phase is `Phase::Concrete` to reject incomplete
+        bindings:
+
+        ```cpp
+        case SemIR::BuiltinFunctionKind::IntConvertFloat: {
+          if (phase != Phase::Concrete) {
+            return MakeConstantResult(context, call, phase);
+          }
+          return PerformIntToFloatConvert(context, loc_id, arg_ids[0], call.type_id,
+                                          /*require_exact=*/false);
+        }
+        ```
+
+    -   Extract inputs safely from local value stores (e.g.
+        `context.ints().Get(arg.int_id)` or
+        `context.floats().Get(arg.float_id)`).
+    -   Leverage high-precision LLVM mathematical structures (`llvm::APInt`,
+        `llvm::APFloat`, `llvm::APSInt`) to handle custom bits and signedness
+        safely.
+
+2.  **Diagnose Invalid Parameters or Exceptions**:
+
+    -   Define compile-time diagnostics inside
+        [kind.def](../../../toolchain/diagnostics/kind.def):
+
+        ```cpp
+        // toolchain/diagnostics/kind.def
+        CARBON_DIAGNOSTIC_KIND(IntTooLargeForFloatType)
+        ```
+
+    -   Emplace localized diagnostic formatting messages where they are caught
+        in `eval.cpp`:
+
+        ```cpp
+        CARBON_DIAGNOSTIC(IntTooLargeForFloatType, Error,
+                          "integer value {0} too large for floating-point type {1}",
+                          llvm::APSInt, SemIR::TypeId);
+        context.emitter().Emit(loc_id, IntTooLargeForFloatType, val, dest_type_id);
+        ```
+
+    -   Return `SemIR::ErrorInst::ConstantId` to gracefully abort invalid
+        constant generation rather than crashing the compiler.
+
+3.  **Fast-Path Range Limits**:
+    -   Before evaluating expensive math operations on giant exponents (e.g.
+        `1.0e1000000`), executing range limits check against `dest_width + 64`
+        (sized) or `IntStore::MaxIntWidth` (unsized) is mandatory to prevent
+        out-of-bounds calculations and compile-time memory exhaustion.
+
+---
+
+### Step 4: Machine Code Generation (LLVM Lowering)
+
+Inside [handle_call.cpp](../../../toolchain/lower/handle_call.cpp):
+
+1.  **Map to Native LLVM Instructions**: For runtime-eligible builtins, map the
+    call inside `HandleBuiltinCall` to native LLVM IR builder methods:
+
+    ```cpp
+    case SemIR::BuiltinFunctionKind::IntConvertFloat: {
+      auto* operand = context.GetValue(arg_ids[0]);
+      auto* dest_type = context.GetTypeOfInst(inst_id);
+      bool is_signed = IsSignedInt(context, arg_ids[0]);
+      context.SetLocal(
+          inst_id, is_signed
+                       ? context.builder().CreateSIToFP(operand, dest_type)
+                       : context.builder().CreateUIToFP(operand, dest_type));
+      return;
+    }
+    ```
+
+2.  **Assert on Compile-Time-Only Builtins**: Throw a hard assertion on
+    lowering-cases for checked validator builtins that should never hit code
+    generation:
+
+    ```cpp
+    case SemIR::BuiltinFunctionKind::IntConvertFloatChecked: {
+      CARBON_CHECK(builtin_kind.IsCompTimeOnly(
+          context.sem_ir(), arg_ids,
+          context.sem_ir().insts().Get(inst_id).type_id()));
+      CARBON_FATAL("Missing constant value for call to comptime-only function");
+    }
+    ```
+
+---
+
+### Step 5: Standard Library Prelude Integration
+
+Map the standard library primitive interfaces to your newly minted named
+builtins under [core/prelude/](../../../core/prelude/):
+
+-   **Primitive Mappings**: Bind Carbon methods directly to string-literal
+    builtin equivalents:
+
+    ```carbon
+    fn Convert[self: Self]() -> Float(To) = "int.convert_float";
+    ```
+
+-   **Strict Orphan Rule Compliance**: Carbon's orphan rules prohibit
+    implementing interfaces where neither the type nor the interface is locally
+    defined in the backing source module.
+    -   **Literal Conversions**: Literal types (like `FloatLiteral`,
+        `IntLiteral`) do not have backing Carbon source files. Therefore, an
+        `impl` of `UnsafeAs` (which is defined in `as.carbon`) between two
+        literal types must reside inside `as.carbon` itself.
+    -   **Sized Conversions**: Implementations targeting sized primitives (e.g.
+        `Int(N)`, `Float(N)`) must reside in their respective type source files
+        (such as [int.carbon](../../../core/prelude/types/int.carbon) or
+        [float.carbon](../../../core/prelude/types/float.carbon)) where the
+        backing target type resides to prevent duplicate symbols and structural
+        recursion loops.
+
+---
+
+## High-Fidelity Validation & Test Authoring
+
+Follow the [Toolchain tests](../toolchain_tests/SKILL.md) skill with specialized
+patterns for builtins:
+
+### 1. Checker Builtin File Splits
+
+Create validation splits under
+[toolchain/check/testdata/builtins/](../../../toolchain/check/testdata/builtins/):
+
+-   **Test Naming Convention**: All tests under
+    [toolchain/check/testdata/builtins/](../../../toolchain/check/testdata/builtins/)
+    must be named after the builtin they are testing, replacing `.` characters
+    in the builtin name with `/` (directories). For example, a test for the
+    builtin `"char_literal.convert"` must be located at
+    `toolchain/check/testdata/builtins/char_literal/convert.carbon`.
+-   **Minimal Prelude & Direct Call Isolation**: Builtin tests must **not** test
+    the prelude library or operators. They must use the minimal primitive
+    prelude (`// INCLUDE-FILE:
+    toolchain/testing/testdata/min_prelude/primitives.carbon`) or a smaller
+    prelude, and explicitly declare and call the builtin functions under test
+    directly (e.g., `fn Add(a: f64, b: f64) -> f64 = "float.add";`). This
+    isolates the testing of compiler builtins from the library prelude.
+-   **Min-Prelude Limitations**: Standard operators (like `+`, `-`, `/`, `<`,
+    etc.) are **not** available in minimized preludes because the core operators
+    library isn't imported. To write tests with a minimal footprint, call
+    primitive builtins directly (e.g. `float.negate`, `float.div`) inside your
+    test code to build expressions.
+-   **Canonicalized Float Comparison**: In SemIR, real literal representations
+    with identical mathematical values can result in mismatched `RealId` objects
+    based on spelling variations. Verify compile-time constant conversions using
+    canonicalized comparison functions (e.g. passing converted results through
+    `Expect(X as f64)`) to completely avoid spelling mismatches in expected
+    outputs.
+-   **Locals Bypass**: If validating generic implicit conversions, compile-time
+    arguments cannot take local runtime variable parameters. Validate
+    compile-time conversions by passing literal constants directly, and sized
+    variable implicit conversions at runtime.
+
+### 2. Machine Codegen Lowering Splits
+
+Create testing splits under
+[toolchain/lower/testdata/builtins/](../../../toolchain/lower/testdata/builtins/):
+
+-   Emplace a simple carbon binding to the tested builtin.
+-   Confirm matching LLVM metadata target definitions are mapped precisely
+    (e.g., matching `sitofp i32 %a to float`, `fptosi float %a to i32`).

.agents/skills/code_style/SKILL.md

@@ -33,6 +33,87 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 -   **Python**: Use `pre-commit run black --files <file.py>` to format Python
     files.
 
+## Comments
+
+-   **Describe the code that is there.** A comment should explain what the
+    current code does or why it is the way it is, not narrate what the code used
+    to be. Do not add a comment to justify a deletion or explain that something
+    is no longer necessary; a reader of the new code has no idea what is being
+    contrasted against, and the comment rots as soon as the old shape is
+    forgotten. Put that reasoning in the commit message instead.
+-   **Do not introduce a local variable just to host a comment.** If a call
+    argument reads clearly on its own, inline it rather than naming it so that a
+    comment has somewhere to attach.
+-   **Lead with a plain summary.** Start a doc comment with a direct statement
+    of what the function does or returns, such as "Returns true if the InstKind
+    is a singleton." Put nuance in a following sentence rather than qualifying
+    the summary into something harder to read.
+-   **Re-read a comment before updating it.** When a symbol is renamed or
+    changes meaning, a comment that mentions it is not automatically stale. Work
+    out what the comment actually asserts first: it may be describing what the
+    code _uses_, which is still true, and rewriting it loses information.
+
+## Documentation
+
+This covers standalone prose: `/docs`, `README.md` files, and the skill files
+under `.agents/skills`.
+
+-   **Be true of the tree it lands in.** Documentation shipping in the same
+    commit as the code it describes should name that code freely. Documentation
+    that lands separately must not: a reader who greps for an identifier from an
+    unlanded change finds nothing, and a claim that only holds after that change
+    is false until it lands. This is easy to get wrong when writing up a lesson
+    while the change that taught it is still in flight.
+-   **Use placeholders for illustrations.** When an example only needs to show a
+    shape, name it `Foo` rather than reaching for a real symbol. Save real
+    identifiers for documenting that identifier, where going stale is at least
+    detectable.
+-   **Make each point stand alone.** A reader has none of the discussion that
+    produced it. State the rule, and enough of the reason to apply it, without
+    assuming knowledge of the change that motivated it.
+
+## Naming
+
+-   **A name is a claim, so keep it true.** When a change invalidates the
+    invariant a name describes, rename it in the same change. A factory called
+    `MakeSingletonFooId` has to be renamed once `Foo` is no longer a singleton,
+    even though nothing forces you to.
+-   **Types in a signature are part of the claim.** A function returning the id
+    of a namespace should return `InstId`, not `TypeInstId`, because a namespace
+    is not a type. Do not let a convenient wider or narrower id type imply
+    something false.
+-   **Delete a predicate whose name stops distinguishing anything.** If a change
+    widens a test so that it no longer says what its name implies, remove it and
+    let callers use the underlying test, rather than keeping a wrapper that
+    sounds meaningful. What callers actually want is often the inverse, and is
+    worth adding under its own name.
+-   **Name a variable for its role, not its representation.** If the role a
+    value plays is unchanged, keep its name even when a change means it is now
+    held or spelled differently.
+
+## Commit descriptions
+
+-   **Say what the change is, not how you made it.** Describe the resulting
+    code. Only describe process when the process is more interesting than the
+    change, as with a large mechanical transformation.
+-   **Do not narrate your own work.** Statements like "each such site was
+    audited" or "we investigated every caller" describe effort, not the change.
+-   **Omit routine mechanical steps.** Do not mention running the testdata
+    autoupdater or the formatter. Every change is expected to include those.
+-   **Do not enumerate each edit.** Describe the change as a whole rather than
+    listing every function or file touched; the diff already lists them.
+-   **Use the project's terms precisely.** Reach for the term of art the
+    codebase uses. Calling something a "type alias" or a "namespace" when it is
+    a named scope misleads, and is worse than a vaguer but accurate word.
+-   **Scope claims to what changed.** Say the specific thing that is now true,
+    rather than a broader statement that happens to contain it.
+
+## Design
+
+-   **Do not distort the data model to improve output.** If printed or golden
+    output is undesirable, change the printer, not the data structures that
+    feed it.
+
 ## Style Guides
 
 -   **C++ style**: Follow the

.agents/skills/diagnostics/SKILL.md

@@ -0,0 +1,261 @@
+---
+name: Diagnostics
+description:
+    Instructions for declaring, formatting, emitting, testing, and styling
+    diagnostic messages (errors, warnings, notes) in the Carbon toolchain.
+---
+
+# Diagnostics in the Carbon Toolchain
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+The Carbon compiler features a highly-engineered, context-aware diagnostics
+framework designed to deliver precise, readable, and highly targetable
+diagnostic output (errors, warnings, notes). This document establishes strict
+rules for declaring, formatting, emitting, testing, and styling compiler
+diagnostics.
+
+---
+
+## Architecture Overview
+
+```mermaid
+graph TD
+  Kind[kind.def Registry] -->|Registration| Enum[Kind Enum ID]
+  Enum -->|Build/Emit| Emitter[Emitter LocT]
+  Emitter -->|ConvertLoc| Loc[Converted Physical Loc]
+  Emitter -->|formatv serialization| Formatting[format_providers.h / Custom Types]
+  Emitter -->|Emit Messages| Consumer[Console / Sorting Consumer]
+  Consumer -->|stable sort| StdErr[Compiler Standard Error]
+```
+
+Diagnostics are handled via three decoupled core components:
+
+1.  **Registry**: Globally enumerated kinds inside
+    [kind.def](../../../toolchain/diagnostics/kind.def).
+2.  **Emitters**: Specialized formatting pipelines (parameterized on custom
+    phase location types `LocT` like `Token` or `LocId`) that convert raw tokens
+    to standardized physical source locations (file, line, column, and text
+    snippet).
+3.  **Consumers**: Pipelines that process, track, filter, and sort diagnostics.
+    The default `SortingConsumer` buffers and stable-sorts diagnostics based on
+    their `last_byte_offset` matching compiler traversal order to ensure perfect
+    causal ordering.
+
+---
+
+## 1. Declaring and Registering Diagnostics
+
+All diagnostic types must pass structural uniqueness and coverage verifications.
+
+### The Diagnostic Registry
+
+Every diagnostic kind must be registered globally as an enum option under
+[kind.def](../../../toolchain/diagnostics/kind.def):
+
+```cpp
+// toolchain/diagnostics/kind.def
+CARBON_DIAGNOSTIC_KIND(RealLiteralTooLargeForUnsizedInt)
+```
+
+### The Uniqueness Rule
+
+To ensure optimal compile-time and analysis integrity, every diagnostic kind
+declared in `kind.def` **MUST** be mapped to **one and only one** C++ macro
+declaration (`CARBON_DIAGNOSTIC` or `CARBON_DIAGNOSTIC_ON_SCOPE`).
+
+-   **DO NOT** duplicate diagnostic definitions across different locations.
+-   The C++ representation of the diagnostic is a static/global constant of type
+    `DiagnosticBase<Args...>`.
+-   **Local Scope (Recommended)**: If the diagnostic is unique to a single
+    block/function body, declare it **locally** inside the function body
+    adjacent to its `Emit` trigger:
+
+    ```cpp
+    void ConvertFloatValueToInt(...) {
+      CARBON_DIAGNOSTIC(FloatNaNConvertedToInt, Error,
+                        "cannot convert NaN to integer type {0}", SemIR::TypeId);
+      context.emitter().Emit(loc_id, FloatNaNConvertedToInt, dest_type_id);
+    }
+    ```
+
+-   **File Scope**: If the diagnostic is shared among multiple functions inside
+    the _same_ file, declare it at **file scope** inside the anonymous namespace
+    of the `.cpp` file.
+-   **Global Scope**: If a diagnostic (such as a shared helper note) is reused
+    _across different physical files_, define it in a shared header (e.g.
+    context/check helpers) and mark it `extern` where applicable, ensuring the
+    macro is only invoked once.
+
+---
+
+## 2. Formatting Diagnostic Arguments
+
+Carbon diagnostics leverage LLVM's `formatv` engine. Parameters must be passed
+using strongly-typed arguments to preserve translation capability.
+
+### String Lifetimes & Pitfalls
+
+-   **`llvm::StringRef` is DISALLOWED**: Do not pass `StringRef` as a parameter
+    type to `CARBON_DIAGNOSTIC` due to unsafe lifetime and buffer-allocation
+    boundaries.
+-   **`llvm::StringLiteral` is DISALLOWED**: Do not use literal types as
+    arguments as they prevent future diagnostic localization and translations.
+-   **Use `std::string`**: If string formatting or custom allocations are
+    required, declare the parameter storage type as `std::string`.
+
+### Format Selectors (`format_providers.h`)
+
+Use specialized formatting wrappers under
+[format_providers.h](../../../toolchain/diagnostics/format_providers.h) to
+express clean inline options in format strings:
+
+| Wrapper                    | Target Format Style             | Example Usage                  | Output                                                              |
+| :------------------------- | :------------------------------ | :----------------------------- | :------------------------------------------------------------------ |
+| **`BoolAsSelect`**         | `{Index:true\|false}`           | `"{0:is signed\|is unsigned}"` | Maps bool to selection string.                                      |
+| **`IntAsSelect`**          | `{Index:=Val:String\|:Default}` | `"{0:=1:is\|:are}"`            | Matches exact options.                                              |
+| **`IntAsSelect` (Plural)** | `{Index:s}`                     | `"{0} argument{0:s}"`          | Prints `"s"` if value != 1 (e.g., `"1 argument"`, `"3 arguments"`). |
+
+### Custom Toolchain Type Mappings
+
+Custom structures can define how they serialize inside diagnostics using the
+`DiagnosticType` tag mapping to `Diagnostics::TypeInfo<StorageType>`:
+
+-   **Identifiers & Names** (declared in `check/diagnostic_helpers.h`):
+    -   `NameId`: Formats raw identifier spelling, safely escaping keyword
+        conflicts under backticks automatically.
+    -   `LibraryNameId`: Formats custom library descriptors cleanly (e.g.
+        `default library` or `library "foo"`).
+-   **Sized Primitives**:
+    -   `TypedInt`: Formats an `APInt` constant exactly, extracting target
+        signedness representation automatically from its bound type
+        representation.
+-   **Type Formatter Hierarchy**: When choosing parameter types to print
+    compiler type representations, follow this priority list:
+    1.  **`TypeOfInstId` (Preferred)**: Resolves the backing type of an
+        `InstId`, preserving programmatic aliasing, constraints, and source
+        spelling context. Enclosed under backticks automatically.
+    2.  **`InstIdAsType`**: Converts an `InstId` for a type expression, printing
+        custom type layouts under backticks.
+    3.  **`TypeId` (Fallback)**: Canonical description of the type. **Avoid when
+        possible** because type canonicalization loses intermediate source
+        program spelling and aliasing metadata.
+    4.  **`*AsRawType` (e.g. `InstIdAsRawType`, `TypeIdAsRawType`)**: Formats
+        the type layout exactly like their counter-structures above, but
+        **omits** enclosing backticks (useful when inserting types inside larger
+        code snippets).
+
+---
+
+## 3. Fluent Emission Builders & RAII Scopes
+
+### Fluent Builder Pattern
+
+For compound diagnostics requiring multiple sub-notes, carets, or custom code
+overrides, use `Build` to chain actions fluently:
+
+```cpp
+context.emitter()
+    .Build(second_node, ModifierRepeated, context.token_kind(second_node))
+    .Note(first_node, ModifierPrevious, context.token_kind(first_node))
+    .OverrideSnippet("custom snippet...")
+    .Emit();
+```
+
+> [!SAFETY] Emitter builders are marked `[[nodiscard]]`. To prevent a developer
+> from creating a builder but failing to terminal-chain `.Emit()`, the builder
+> uses an rvalue overload `Emit() &&` that triggers a compile-time
+> `static_assert(false)`. You must save the builder to an lvalue or execute the
+> chain exactly as `emitter.Build(...).Note(...).Emit()`.
+
+### RAII Context & Annotation Scopes
+
+Manage large checking structures requiring blanket note context using RAII block
+scopes:
+
+-   `ContextScope`: Automatically converts any diagnostics emitted within its
+    scope into sub-notes under a high-level operation descriptor:
+
+    ```cpp
+    ContextScope context_scope(&context.emitter(), [&](ContextBuilder& builder) {
+      builder.Context(eval_loc, InCallToEvalFn);
+    });
+    // any checker error emitted here will automatically append the 'InCallToEvalFn' note
+    ```
+
+-   `AnnotationScope`: RAII block scope that automatically attaches blanket note
+    annotations to all scoped diagnostics.
+
+---
+
+## 4. Diagnostics Wording Style Guide
+
+Refer to the official
+[Diagnostic message style guide](../../../toolchain/docs/diagnostics.md#diagnostic-message-style-guide)
+for complete details.
+
+To maintain message consistency and integrate cleanly with Clang diagnostics in
+interoperable code, adhere strictly to these rules:
+
+-   **Start with lowercase and omit periods**: Start diagnostic messages with a
+    lowercase letter or quoted code, and do **not** end them with a period
+    (e.g., `"cannot convert..."` or ``"`self` declared..."``).
+-   **Use backticks for quoted code**: Enclose identifiers, code constructs, and
+    types inside standard backticks (e.g., ``"`{0}` is bad"``).
+-   **Phrase as bullet points without articles**: Phrase diagnostics as
+    descriptive bullet points or sentence fragments rather than full sentences.
+    Leave out standard articles (`a`, `an`, `the`) unless necessary for logical
+    clarity. Semicolons can be used to separate fragments within a message.
+-   **Describe the situation and language rule**: Diagnostics should describe
+    the exact situation the toolchain observed. The language rule violated can
+    be mentioned if it wouldn't otherwise be clear:
+    -   _Situation-only_: `"redeclaration of X"` (implies that redeclaration is
+        not permitted).
+    -   _Rule-inclusion_: ``"`self` declared in invalid context; can only be
+        declared in implicit parameter list"``.
+-   **Wording Choice ("cannot" vs "allowed")**: Explicitly avoid `"allowed"`,
+    `"legal"`, `"permitted"`, `"valid"`, and related passive wording. You may
+    use `"cannot"` if needed, but try to use phrasing that does not require it:
+    -   _Correct_: ``"`export` in `impl` file"`` (Avoids `"allowed"`)
+    -   _Incorrect_: ``"`export` is only allowed in API files"``
+    -   _Correct_: ``"`extern library` specifies current library"`` (Avoids
+        `"cannot"`)
+    -   _Incorrect_: ``"`extern library` cannot specify the current library"``
+-   **Developer Intent Hints**: It is acceptable for a diagnostic to guess at
+    the developer's intent and provide a hint _after_ explaining the situation
+    and the rule, but never as a substitute for that:
+    -   _Correct_: ``"cannot implicitly convert `i32` to `String`; add `as
+        String` for explicit conversion"``
+    -   _Incorrect_: ``"add `as String` to convert `i32` to `String`"`` (Lacks
+        the core violation message).
+-   **Structure for Tooling API**: Try to structure diagnostics such that
+    parameter inputs can be programmatically extracted without string parsing
+    (prefer strongly-typed parameters over format placeholders where possible).
+
+---
+
+## 5. Diagnostics Testing & Coverage Verification
+
+Carbon strictly enforces testing coverage at build-time.
+
+1.  **Tag Verification Requirement**: Every diagnostic kind declared in
+    `kind.def` (which is not blacklisted in the `UntestedKinds` array under
+    [coverage_test.cpp](../../../toolchain/diagnostics/coverage_test.cpp))
+    **MUST** be verified by at least one testcase file inside
+    `toolchain/*/testdata/`.
+2.  **Stderr Checklist Matchers**: The testcase split verifying the diagnostic
+    must catch it using standard CHECK matchers, explicitly tracking the
+    matching enum tag in standard error comments:
+
+    ```carbon
+    // CHECK:STDERR: fail_bounds.carbon:[[@LINE+1]]:15: error: cannot convert NaN to integer type `i32` [FloatNaNConvertedToInt]
+    let a: i32 = Convert(nan_val);
+    ```
+
+3.  **Build Enforcement**: Failing to provide a diagnostic test check matcher
+    triggers a build compilation error on the target test
+    `//toolchain/diagnostics:coverage_test`.

.agents/skills/jj/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: Jujutsu (jj) usage
+description:
+    Instructions for using Jujutsu (jj) for version control in the Carbon
+    repository.
+---
+
+# Jujutsu (jj) usage
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+[Jujutsu](https://github.com/jj-vcs/jj) is a Git-compatible version control
+system that may be used in Carbon checkouts.
+
+> [!IMPORTANT] You can detect if Jujutsu is in use by checking for a `.jj`
+> directory in the repository root. If present, you **must** use `jj` and **must
+> not** use `git`. If absent, you **must not** use `jj`.
+
+## General usage
+
+Always use the `--no-pager` flag when invoking `jj` to prevent the command from
+blocking or waiting for terminal paging.
+
+## Common commands
+
+### Syncing with remote
+
+-   **Fetch from remote**: `jj --no-pager git fetch`
+-   **Create a new change on top of trunk**: `jj --no-pager new trunk`
+-   **Show repository status**: `jj --no-pager status`
+-   **Show commit history**: `jj --no-pager log`
+
+### Managing changes
+
+-   **View diff of current changes**: `jj --no-pager diff`
+-   **Commit changes**: `jj --no-pager commit`
+    -   _Note_: Prefer using `jj commit` over the combination of `jj describe`
+        and `jj new`.
+-   **Abandon/discard current changes**: `jj --no-pager abandon`
+-   **Rebase current change onto trunk**: `jj --no-pager rebase -o trunk`
+
+### Working with a stack of changes
+
+A change is often built as a stack of commits sent up as a single pull request.
+The stack is not necessarily based on `trunk`; it may be based on another change
+that is itself still in flight.
+
+> [!WARNING] **Never rewrite the history of a change that has been submitted as
+> a pull request.** Reviewers track a PR by its commits, and squashing,
+> reordering, or abandoning them discards review that is already in progress.
+> This cannot be undone from their side.
+>
+> Before rewriting history in any other case, propose the exact command and wait
+> for confirmation. This applies to `squash`, `rebase`, `abandon`, and
+> `describe` on an existing change.
+
+#### Finding the base of the stack
+
+Bookmarks delimit the stack. List the bookmarks that are ancestors of the
+working copy, nearest first:
+
+```bash
+jj --no-pager log -r '::@ & bookmarks()'
+```
+
+Reading the result takes care, because two situations produce similar output:
+
+-   **Editing an existing change.** The nearest bookmark names the change being
+    worked on, and the bookmark below it is the base.
+-   **Starting a new change.** The commits above the nearest bookmark have no
+    bookmark of their own yet, so the nearest bookmark is itself the base.
+
+`trunk` is only ever a base. Finding `trunk` nearest means new work is being
+built on top of it, never that `trunk` itself is being worked on.
+
+The graph does not distinguish the two cases: an unbookmarked or empty commit
+above a bookmark may be the next commit of that change or the start of a new
+one. Ask which it is when it is not clear, and ask before choosing where a fix
+should land rather than after. Guessing wrong means squashing into a change that
+may already be under review.
+
+Once the base is known, use it to scope commands to the current stack:
+
+```bash
+jj --no-pager log -r '<base-bookmark>..@'
+```
+
+#### Managing the stack
+
+-   **Fold a fix into an earlier change**:
+    `jj --no-pager squash --into <change-id> [path]`. Follow-up fixes and
+    formatter reflows belong in the change that introduced the code, not in a
+    trailing "fixes" commit, unless that change has already been submitted.
+    Naming a path squashes only that part of the working copy, leaving unrelated
+    work in place.
+-   **Descriptions**: only one change in the stack needs a long description, the
+    one used as the pull request description. Every other change gets a short
+    one-line summary. Do not repeat the long text across the stack.

.agents/skills/language_server/SKILL.md

@@ -0,0 +1,197 @@
+---
+name: Language server
+description:
+    Instructions for working on Carbon's LSP language server, including its
+    architecture, its file_test-based tests, and the VS Code extension.
+---
+
+# Language server
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+## Introduction
+
+This skill covers [`toolchain/language_server/`](/toolchain/language_server/),
+which implements `carbon language-server`, and
+[`utils/vscode/`](/utils/vscode/), the VS Code extension that launches it.
+
+## Architecture
+
+The server is built on clangd's LSP transport (`clang::clangd`), not on a
+Carbon-specific one. That means clangd's `Protocol.h` types (`Position`,
+`Range`, `Location`, `Hover`, `MarkupContent`) are the interface currency.
+
+-   `server.cpp` / `incoming_messages.cpp`: message dispatch. A handler must be
+    registered in `incoming_messages.cpp` before it can be called.
+-   `handle_*.cpp`: one file per request family, each declaring its entry point
+    in `handle.h`.
+-   `handle_initialize.cpp`: the advertised capabilities. **Adding a capability
+    changes `Content-Length` in every test that calls `initialize`**, so expect
+    a large autoupdate diff.
+-   `context.h` / `context.cpp`: `Context::File` per open document, plus the
+    compile driver. `Context::File::unit()` has a `CARBON_CHECK` on the compile
+    driver, so any handler that reaches for the parse tree must first rule out
+    documents that were never compiled.
+-   `position.h`, `sem_ir_index.h`: mapping source positions to SemIR
+    instructions for real Carbon files.
+-   `sem_ir_text.h`, `handle_sem_ir_text.h`: navigation within the *formatted
+    SemIR* in a test file's `// CHECK:STDOUT:` lines. This is a heuristic text
+    index, deliberately independent of the real SemIR data structures. See
+    [the SemIR text reader](#the-semir-text-reader).
+
+### Document kinds
+
+The server handles two kinds of document, distinguished by the `languageId`
+from `textDocument/didOpen`, with a content sniff as a fallback:
+
+-   `carbon`: a real Carbon file. Compiled; diagnostics published.
+-   `carbon-testdata`: a test file. **Not compiled**, because we lack logic to
+    split it into one file per `// ---` split marker.
+
+> [!IMPORTANT]
+> A handler that assumes every file was compiled will crash on a test file.
+> When adding one, give the test-file path an explicit early return.
+
+## Tests
+
+There is **no language-server-specific test target**.
+`toolchain/language_server/BUILD` only declares
+`filegroup(name = "testdata")`, which is pulled into
+`//toolchain/testing:all_testdata` and run by `//toolchain/testing:file_test`.
+
+```bash
+# Run just the language server tests (or any subset).
+bazelisk test //toolchain/testing:file_test \
+    --test_arg=--file_tests=toolchain/language_server/testdata/position/hover_and_goto.carbon
+
+# See the raw output, which is much easier to read than a test failure.
+bazelisk run //toolchain/testing:file_test -- --dump_output \
+    --file_tests=toolchain/language_server/testdata/position/hover_and_goto.carbon
+
+# Update expectations. Never hand-write CHECK lines.
+./toolchain/autoupdate_testdata.py toolchain/language_server/testdata/...
+```
+
+These tests run serially, because clangd's logging is a global singleton.
+
+### Test file shape
+
+The request stream is a `// --- STDIN` split written with the `[[@LSP-*]]`
+keywords, and the responses land in a trailing `// --- AUTOUPDATE-SPLIT`.
+Documents come from other splits by way of `"text": "FROM_FILE_SPLIT"`, which
+is substituted with the content of the split whose name matches the `uri`.
+
+```carbon
+// --- position.carbon
+fn Abs(n: i32) -> i32 { return n; }
+
+// --- STDIN
+[[@LSP-CALL:initialize:"capabilities": {}]]
+[[@LSP-NOTIFY:textDocument/didOpen:
+  "textDocument": {
+    "uri": "file:/position.carbon",
+    "languageId": "carbon",
+    "text": "FROM_FILE_SPLIT"
+  }
+]]
+[[@LSP-CALL:textDocument/hover:
+  "textDocument": {"uri": "file:/position.carbon"},
+  "position": {"line": 0, "character": 3}
+]]
+[[@LSP-CALL:shutdown]]
+[[@LSP-NOTIFY:exit]]
+
+// --- AUTOUPDATE-SPLIT
+```
+
+Full keyword documentation is in
+[`testing/file_test/README.md`](/testing/file_test/README.md).
+
+### Traps
+
+> [!WARNING]
+> **A blank line inside the `STDIN` split breaks the JSON transport.** It
+> terminates a header block, so clangd logs a timestamped
+> `Warning: Missing Content-Length header, or zero-length message.` The
+> timestamp makes the test unreproducible, so it fails on the next run.
+> Comment lines between messages are fine; blank lines are not. A single blank
+> line immediately before `// --- AUTOUPDATE-SPLIT` is also fine.
+
+Other things worth knowing:
+
+-   **`positionEncoding` is UTF-16.** A `character` is a UTF-16 code unit
+    offset, not a byte offset.
+-   **Line and character numbers in requests are 0-based**, while the `locN_M`
+    suffixes in SemIR output are 1-based. Off-by-ones here are silent: the
+    request succeeds and returns the wrong thing.
+-   **A split can hold a document that itself contains `// CHECK:STDOUT:`
+    lines**, because `CHECK` lines only form expectations inside the
+    `AUTOUPDATE-SPLIT`. Such a document still can't contain a literal `// ---`
+    line, which would split the enclosing test file; write it as
+    `[[@0x2f]]/ --- name.carbon`.
+
+## The SemIR text reader
+
+`sem_ir_text.cpp` indexes the formatted SemIR inside a test file's
+`// CHECK:STDOUT:` lines so that hover and go-to-definition work on operand
+names. It is a heuristic reader, not a parser, and its correctness rests on
+facts about `toolchain/sem_ir/formatter.cpp` and
+`toolchain/sem_ir/inst_namer.cpp`. Re-check these if the formatter changes:
+
+-   There are exactly four scope keywords: `file`, `generated`, `imports`, and
+    `constants` (`InstNamer::GetScopeName`). Everything else is `@entityname`.
+-   A reference is `%name` within its own scope and `scope.%name` otherwise
+    (`InstNamer::GetNameFor`). Names may contain `.` and may _start_ with one,
+    as in `%.Self.frozen`.
+-   **Type annotations are printed in the `constants` scope.**
+    `Formatter::FormatTypeOfInst` does
+    `llvm::SaveAndRestore file_scope(scope_, InstNamer::ScopeId::Constants)`,
+    so in `%x: %foo = ...` a bare `%foo` means `constants.%foo`. This does not
+    apply to ordinary operands or to `[concrete = ...]` annotations.
+-   **`*_decl` braces hold the declared entity's scope.** The braces of
+    `%F.decl: ... = fn_decl @F [...] { ... } { ... }` are lexically inside
+    `file { }`, but their names belong to `@F`.
+-   **`!with Self:` switches scope without a brace**, until `!members:`
+    switches it back. Brace counting alone cannot see this.
+-   A `specific @F(args) { }` block uses `@F`'s scope and defines nothing; each
+    `%name => value` row references an instruction of the generic.
+
+### Validating a change to the reader
+
+The unit tests only cover a handful of cases. To check a change against the
+real corpus, drive the server over a sample of check testdata, hovering on
+every `%name`, and compare the resolved fraction before and after. Roughly 99%
+of names resolve; the residue are names the formatter references but never
+emits a definition line for, such as `%I.WithSelf.F`, which only ever appears
+inside a `[symbolic = ...]` annotation.
+
+Two things to get right in such a harness:
+
+-   Feed the request stream from a **file**, not a pipe. The server reads stdin
+    as a file and reports `error: Input/output error` on a pipe.
+-   Read the output as **bytes**. Python's `text=True` rewrites the `\r\n`
+    framing, and `Content-Length` counts bytes.
+
+## VS Code extension
+
+[`utils/vscode/`](/utils/vscode/) declares three languages in `package.json`:
+
+| Language id      | Applies to                        |
+| ---------------- | --------------------------------- |
+| `carbon`         | `*.carbon`                        |
+| `carbon-testdata`| `**/testdata/**/*.carbon`         |
+| `semir`          | `*.semir`                         |
+
+> [!NOTE]
+> The TextMate _scope_ for SemIR is `source.carbon-semir`, but the
+> _language id_ is `semir`. Markdown code fences in hover text resolve language
+> ids, so a fence must say ` ```semir `.
+
+`extension.ts` launches the server over stdio, using the `carbonPath` setting
+(default `./bazel-bin/toolchain/carbon`). Its `documentSelector` controls which
+files are sent to the server at all; a new document kind has to be added there
+as well as in the server.

.agents/skills/prek/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: Prek
+description:
+    Instructions for running prek, the Carbon pre-submit/style/lint checker,
+    that *MUST* be run before submitting an change.
+---
+
+# Prek
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+`prek` is the Carbon pre-submit, style, and lint checker. Running it is
+mandatory before submitting any changes.
+
+## Running prek
+
+To run `prek` on all files:
+
+```bash
+prek run -a
+```
+
+To validate a specific list of files:
+
+```bash
+prek run --files <files>
+```
+
+## Running prek in a Jujutsu (jj) workspace
+
+If you are working in a Jujutsu workspace, running `prek` directly will fail
+because it expects a standard Git repository structure. Instead, use the helper
+script:
+
+```bash
+./scripts/jj_prek.sh
+```
+
+This script runs `prek` on all files that have changed between `trunk` and your
+current Jujutsu `@` change.
+
+Note that the script always compares against `trunk`. If your change is based on
+another bookmark rather than on `trunk`, the script also checks the files
+changed by that underlying change, so a reported failure may not be in your own
+work.
+
+## Hooks that rewrite files
+
+Some hooks, notably `clang-format` and `rumdl`, fix problems in place rather
+than only reporting them. When they do, `prek` reports a failure and exits
+non-zero even though the tree is now correct.
+
+Re-run `prek` after any failure that modified files, and treat the second, clean
+run as the result. Review what it changed: a reflow is expected, but a content
+change may not be what you intended.
+
+## Prek dependency errors
+
+> [!TIP] If `prek` fails with an error about resolving dependencies or security
+> policy, you may be running in a restricted environment where the
+> special-purpose `gpkg` tool is required. Prefix the command with `gpkg`, for
+> example: `gpkg prek run -a` or `gpkg ./scripts/jj_prek.sh`.

.agents/skills/proposals/SKILL.md

@@ -0,0 +1,135 @@
+---
+name: Proposals
+description:
+    Instructions for writing, submitting, and managing Carbon evolution
+    proposals.
+---
+
+# Proposals
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+## Overview
+
+This skill provides instructions and best practices for working with proposals.
+Only create a proposal when explicitly directed as part of the task.
+
+Make sure to confirm the desired title for the proposal as that will govern the
+filename.
+
+## Create a new proposal
+
+1.  **Use the helper script**: Run `./proposals/scripts/new_proposal.py "Title"`
+    to create a templated file and instructions for setting up the PR.
+2.  **Proposal file**: The file will be named `proposals/p######-title.md`,
+    where `######` is the 6-digit GitHub pull request number and `title` is a
+    slugified version of the proposal title.
+3.  **Template**: Follow the structure in `proposals/scripts/template.md`,
+    noting the specific `TODO` instructions in each section for the content that
+    should be included there.
+4.  **PR description**: Update the pull request description to match the
+    abstract section in the proposal document.
+
+> [!IMPORTANT] Do _not_ mark the PR ready for review. The user must have the
+> opportunity to review the proposal produced before asking for any review.
+
+## Writing style and best practices
+
+-   **Skimmable**: Use
+    [BLUF](<https://en.wikipedia.org/wiki/BLUF_(communication)>) (Bottom Line Up
+    Front) or
+    [Inverted Pyramid](<https://en.wikipedia.org/wiki/Inverted_pyramid_(journalism)>)
+    style. Keep it brief, focused, and technical.
+-   **Match existing proposal style**: Review [existing proposals](/proposals)
+    (preferring more recent ones with higher numbers) to understand the expected
+    style, wording, and nature of content to include.
+-   **Connect to goals**: In the Rationale section, link to specific goals in
+    [`/docs/project/goals.md`](/docs/project/goals.md) and principles in
+    [`/docs/project/principles`](/docs/project/principles) (e.g.,
+    `error_handling.md`, `one_way.md`).
+-   **Living design**: If the proposal updates design documentation, include
+    those changes in the PR if possible. If deferred, add "TODO" comments
+    pointing to the proposal (e.g., `> **TODO:** Document ... adopted in
+    [p######](/proposals/p######-title.md)`). For pervasive changes, file a
+    GitHub issue instead of adding many TODOs.
+
+## Alternatives considered and leads decisions
+
+There are always alternatives to a proposal, and the proposal should carefully
+include sections describing all of them and the rationale for not selecting
+them. Any living design document updates should focus on fully describing the
+end-state design, and the key motivating aspects of that design. The main
+proposal should focus on _what is changing_ and _why it is changing_, and should
+leave detailed description of the resulting design to the living design
+document, and _why not_ rationale to the description of each alternative.
+
+-   **Cover all the alternatives**: Make sure to describe any alternatives
+    considered, even if minor or rejected early.
+-   **Be specific**: Don't be vague about any of the alternatives, or the
+    rationale for not choosing them.
+-   **Connect to goals or principles**: In addition to the rationale section,
+    one of the best rationale structures for rejecting an alternative connects
+    that choice back to the goals or principles relevant.
+-   **Always frame as a tradeoff**: Selecting the proposed direction instead of
+    an alternative is _always_ a tradeoff, with both advantages and
+    disadvantages.
+-   **Where relevant, cite the leads issue** that decides against an
+    alternative, in addition to summarizing the key points, tradeoffs, and
+    rationale for the decision.
+
+> [!IMPORTANT] Don't just list the alternatives, create a sub-section for each
+> alternative and carefully describe the alternative, the advantages,
+> disadvantages and what the core of the decision is to reject each alternative.
+
+> [!IMPORTANT] Carefully research each alternative in the leads issue in order
+> to provide this clear and comprehensive explanation.
+
+## Building from a leads issue
+
+Sometimes a proposal is specifically documenting and formalizing a decided leads
+issue. When this is the case, carefully research that leads issue, reading the
+original issue text and every comment on the issue. Also read any linked Google
+documents, linked issues, examples, gists, or other supplemental information
+cited.
+
+-   **Summarize the leads issue**: Ensure you provide a high level summary of
+    the _decided_ direction of the leads issue as the proposal.
+-   **Capture and document** every key aspect of the decision made and factor
+    that led to the decision. It is important that the proposal stands alone,
+    and the leads issue is merely cited for context and history.
+-   **Stay grounded**: Only include alternatives, rationale, and arguments based
+    on what you find in the issue and related documents. No new information
+    should be in the proposal.
+
+Use the `gh` command line tool to query leads issues in order to carefully
+examine all of the comments. Follow any mentioned links to gather more data.
+Refer to the [GitHub CLI usage skill](/.agents/skills/github_cli/SKILL.md) for
+detailed instructions on using the `gh` tool.
+
+Ask the user to clarify any aspects of the leads issue that are unclear rather
+than continuing to edit the proposal. If there are questions that you don't find
+an answer to in the issue, ask this to the user and let them provide an answer
+that you use as the basis of what to include.
+
+## Examples are golden
+
+Heavily leverage examples to illustrate both the specifics of the design being
+proposed, and the nature of the change being proposed. More examples to
+illustrate more aspects, corner cases, or provide a more complete understanding
+are almost always good. Comments in example code should focus on what that part
+of the example illustrates from the proposal.
+
+## Keep the PR description in sync with the abstract
+
+Whenever you edit the abstract or notice differences from the PR description,
+you should update the PR description to match the abstract. The only exception
+is to retain any "Assisted-by" or other tags at the end of the description that
+are only needed there and not in the abstract.
+
+If you are creating or updating a proposal, make sure the PR description in
+question contains an `Assisted-by:` tag that is appropriate for describing which
+AI tool is being used.

.agents/skills/review_testdata_changes/SKILL.md

@@ -0,0 +1,366 @@
+---
+name: Review testdata changes
+description:
+    Instructions for judging whether changes to file test output
+    (`// CHECK:STDOUT:` and `// CHECK:STDERR:` lines) are correct, which
+    changes are acceptable churn, and which are regressions in disguise.
+---
+
+# Review testdata changes
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+## Introduction
+
+Most toolchain work moves file test output. `./toolchain/autoupdate_testdata.py`
+rewrites the `// CHECK:STDOUT:` and `// CHECK:STDERR:` lines in
+`toolchain/*/testdata/` to match current behavior, so after running it the tests
+pass again whether or not the new behavior is right. Deciding that the new
+output is the output you wanted is a separate, manual step, and it is the step
+this skill covers.
+
+Three skills divide the work:
+
+-   [Toolchain tests](../toolchain_tests/SKILL.md): how to _author_ tests and
+    generate their output.
+-   This skill: how to _judge_ an output diff.
+-   [Summarize testdata changes](../summarize_testdata_changes/SKILL.md): how to
+    _report_ an output diff once you believe it is correct.
+
+> [!IMPORTANT] Never hand-edit `// CHECK:STDOUT:` or `// CHECK:STDERR:` lines.
+> Everything below is about changing the _code_ until the generated output is
+> right, never about editing the output to match the code.
+
+## The rule that generates all the others
+
+**Every line of testdata churn must have a cause you can name.** Not "it is
+similar to the other changes", not "the tests pass now" — an actual sentence
+saying which code change produced it and why that is the intended result.
+
+A diff you cannot narrate is a diff you have not reviewed. Changes you cannot
+explain are where regressions hide, because a regression and an intended change
+look exactly alike once the autoupdater has written them down.
+
+> [!CAUTION] Autoupdating is destructive to your evidence. Once the autoupdater
+> has run, the previous expectations are gone from the working copy. Read the
+> diff after _every_ autoupdate run, and if something changed for a reason you
+> cannot name, fix the code before autoupdating again. Recovering the old
+> expectations later means reverting and re-running.
+
+## STDERR and STDOUT are different kinds of evidence
+
+Treat them separately; they have different standards of proof.
+
+**STDERR is user-visible behavior.** These are the diagnostics a Carbon
+programmer sees. A change here is a change to the language implementation as
+users experience it, so each one needs an individual justification. For a
+refactoring, the expected STDERR diff is empty.
+
+**STDOUT is internal representation.** SemIR dumps, parse trees, LLVM IR. Users
+never see it. Churn here is normal and often unavoidable, so the standard is not
+"no change" but "no change I cannot account for".
+
+For a change that is supposed to preserve behavior, write the list of accepted
+STDERR changes down _before_ you autoupdate, and keep it current. Order matters
+more than form: written first, the list is a prediction the diff can falsify;
+written afterwards, it is a description of whatever happened, and describes a
+regression exactly as well as an intended change.
+
+The list does not have to be a deliverable. Scratch notes you never publish do
+the job, because the work is in committing to the list, not in presenting it.
+What a reader needs is not the list but its exceptions: diagnostics that changed
+without being predicted, and predictions that did not occur. Both are findings.
+The matches are not, and reporting them buries the two entries that matter.
+
+## Judging STDOUT churn
+
+Sort each STDOUT change into mechanical or structural.
+
+### Mechanical churn
+
+Expected, and cheap to accept in bulk once you have confirmed the pattern:
+
+-   **Renaming.** An instruction, type, or scope prints under a new name.
+-   **Positional name renumbering.** Names like `%x.loc18_46.3` embed a line,
+    column, and disambiguating index. Two different events move them:
+    -   Adding or removing an instruction at a location renumbers the rest, so a
+        _single_ removed instruction can show up as many changed lines in the
+        same block. Confirm the cascade is a cascade before accepting it as one.
+    -   Adding or removing a _diagnostic_ moves the source lines themselves, so
+        the line component changes everywhere below it in the file. See
+        [Autoupdate to a fixed point](#autoupdate-to-a-fixed-point).
+-   **Fingerprint-derived names.** Mangled names and some scope names are
+    derived from a hash of their inputs. If you changed a hashed input, these
+    move. Confirm that each such difference is _only_ the fingerprint, and not a
+    fingerprint difference concealing a structural one.
+
+> [!TIP] Mechanical churn is usually wide and shallow: the same substitution,
+> repeated across many files. If a "mechanical" pattern needs a different
+> explanation in each file, it is not mechanical.
+
+### Structural churn
+
+Each of these needs its own explanation:
+
+-   Instructions appearing or disappearing.
+-   Changed `[concrete = ...]`, `[symbolic = ...]`, or other constant-value
+    annotations.
+-   Changed types on existing instructions.
+-   Changed control flow: new or removed blocks, changed branch targets.
+-   Raw instruction ids renumbering in `--dump-raw-sem-ir` output when you did
+    not intend to change the id layout.
+
+## Fewer instructions is not automatically better
+
+A diff that removes instructions looks like an optimization. Whether it is one
+depends on what the changed function owes its caller, and two functions can
+produce the same shrinking diff for opposite reasons:
+
+-   A function that only needs a constant value, but built instructions on the
+    way to it, was doing wasted work. Dropping them and using the constant
+    directly is correct, and the shorter output reflects that.
+-   A function whose caller needs a **non-canonical instruction** can be made
+    shorter the same way, by using the constant instead of creating an
+    instruction of its own — and that is wrong. The instruction carries location
+    information, and symbolic constant substitution operates on it; the constant
+    value alone loses both.
+
+The instruction count is identical evidence in both cases, so it cannot be the
+thing you judge. Decide what the function is required to produce; the count
+follows from that.
+
+The same reasoning runs in reverse: a diff that _adds_ instructions is not
+automatically a regression.
+
+## Judging STDERR churn
+
+### A diagnostic's authority depends on the file's prefix
+
+The `fail_` and `todo_` prefixes (see
+[Toolchain tests](../toolchain_tests/SKILL.md)) say how much the recorded
+diagnostics are worth:
+
+-   **`fail_...`** — the test should and does produce errors. The recorded
+    diagnostic **is the specification**. Changing it is a user-visible behavior
+    change and needs justification on its own merits.
+-   **`fail_todo_...`** — the test produces errors (or crashes) but shouldn't,
+    or produces the wrong errors. The recorded diagnostic is explicitly **not**
+    the specification; the file exists to record that today's behavior is wrong.
+    Changing it replaces one wrong answer with another. That is acceptable when
+    you can say why the new message follows from your change and why it is no
+    further from the intended eventual behavior — which, for many such files, is
+    no diagnostic at all.
+-   **`todo_fail_...`** — the test should produce errors but does not. Gaining a
+    diagnostic here may be _progress_, not a regression. Either way the file
+    must be renamed, since the framework requires a `fail_` prefix on any file
+    that errors: `fail_...` if it now produces the right error, `fail_todo_...`
+    if the error is the wrong one. Both renames make the test pass, so record
+    which case it is instead of letting the rename settle it.
+-   **`todo_...`** — behavior is wrong but produces no errors, and shouldn't.
+    Gaining a diagnostic here is a regression unless you can argue otherwise.
+
+> [!IMPORTANT] This is a reason to look at the _filename_ before judging a
+> diagnostic change, not a license to ignore `todo_` files. "It was already
+> broken" does not excuse making it differently broken for no reason.
+
+### Reclassifying tests
+
+If your change moves a test between the states above, the prefix must move with
+it: when the test is fixed, when it starts failing, and when it starts failing
+differently. The correspondence between the `fail_` prefix and whether
+compilation actually failed is enforced by the test framework, not by the
+autoupdater, so it surfaces when you run `bazelisk test` and not when you
+autoupdate. **Autoupdating is not a substitute for running the tests.**
+
+Only that half of the name is checked. Nothing enforces `todo_`, so a file whose
+behavior you have just fixed can keep its `todo_` prefix indefinitely and still
+pass. A missing `fail_` stops the build; a stale `todo_` is silent, and is yours
+to catch.
+
+When a fix drops a file's prefix, also check that the file still belongs where
+it is and that its comments do not still describe the old broken behavior.
+
+### Vaguer diagnostics are a signal, not a verdict
+
+When a diagnostic becomes less specific — a general "unsupported" message
+replacing one that named the problem — that usually means a code path stopped
+finding information it previously had. Sometimes that is correct: the
+information was misleading, and the old message was confidently wrong.
+
+Do not accept it silently and do not reject it reflexively. Say which direction
+it moved and whether the new message is closer to or further from the eventual
+intended behavior.
+
+## Signals in the shape of the diff
+
+### Zero churn is a result
+
+If you removed something you believed was doing work and _no_ testdata moved,
+that is not a missing test run — it is the proof that the thing was a no-op.
+Say so explicitly; it is one of the strongest pieces of evidence a refactoring
+can produce.
+
+The converse is also informative. If you expected a path to churn and it didn't,
+either your model of the code is wrong or that path is untested. Find out which,
+and consider adding a test before continuing.
+
+### Churn should be proportional to the change
+
+-   **Wide churn from a narrow change** means your model of the code is wrong.
+    Do not autoupdate over it. Find the structural mistake first.
+-   **Narrow churn from a sweeping change** means the affected paths are
+    probably untested.
+
+Set a rough expectation for the size of the diff before you run the autoupdater,
+and treat a large mismatch in either direction as a finding.
+
+### Never make the diff smaller by weakening the test
+
+Editing test _input_ (the Carbon source, not the CHECK lines) to make a diff
+look better is a behavior change in disguise. Deleting a test that now produces
+awkward output is worse. If a test's input has to change, that is a separate,
+explicitly-justified change, not diff cleanup.
+
+## What the autoupdater will not fix for you
+
+-   **`NOAUTOUPDATE` files.** Their expectations are maintained by hand. They
+    fail under `bazelisk test` rather than being silently rewritten.
+-   **Hand-written C++ expectations**, for example golden output asserted in a
+    `_test.cpp`. When several assertions in one of these break together, often
+    only the first failure is reported, so fixing it can reveal another. Re-run
+    until clean rather than assuming one fix was the whole repair.
+-   **Golden files outside `testdata/`**, and documentation that quotes compiler
+    output.
+
+## Review loop
+
+### Run the autoupdater
+
+Autoupdate everything, then read the diff a directory at a time. Passing no
+paths is the default and updates every file test in the toolchain:
+
+```bash
+./toolchain/autoupdate_testdata.py
+```
+
+Narrow the scope only while iterating on one subdirectory you know you are not
+done with, where each round would otherwise regenerate output you have already
+read:
+
+```bash
+./toolchain/autoupdate_testdata.py toolchain/check/testdata/SUBDIR/**/*
+```
+
+The globs are expanded by the shell; the script filters its arguments to
+`.carbon` files under a `testdata/` directory.
+
+> [!TIP] If intermediate states crash on `CARBON_CHECK` failures, pass
+> `--non-fatal-checks` so you can see the full set of downstream damage in one
+> run instead of one crash at a time.
+
+> [!IMPORTANT] Return to the full scope before judging the diff. Whether churn
+> is proportional to the change, and whether a change to one phase moved another
+> phase's testdata, are only visible across everything.
+
+### Autoupdate to a fixed point
+
+One pass is not always enough, because the autoupdater's output is part of its
+own input. `// CHECK:STDERR:` lines sit inline, immediately above the source
+line they describe, and the compiler reads them as comments in the file.
+Gaining or losing a diagnostic therefore moves every source line below it.
+
+Within a single run, the compiler has already read the file as it was, so the
+two kinds of output end up in different states:
+
+-   **STDERR is correct after one pass.** These lines locate themselves
+    relatively, as `[[@LINE+N]]`, and the autoupdater recomputes `N` as it
+    places them.
+-   **STDOUT is stale after one pass.** SemIR names like `%x.loc18_46.3` embed
+    an absolute line and column with no filename attached, and the autoupdater
+    only remaps `file.carbon:18`-style references. Nothing rewrites the `loc`,
+    so it still describes where the instruction was _before_ the diagnostic
+    lines moved it.
+
+Running again compiles the shifted file and the names catch up. The diagnostic
+set does not change this time, so nothing shifts again and a third run is a
+no-op. Keep running the autoupdater until it stops changing files: one pass when
+the diagnostics held still, two when they didn't.
+
+> [!WARNING] The intermediate state is self-inconsistent, not just unfinished:
+> its `loc` names describe a file layout that no longer exists. Keep reading the
+> diff after every run — that rule does not change — but do not chase positional
+> churn to a cause until the file has converged, and do not present the diff
+> until then either.
+
+The converging pass should be positional renumbering and nothing else. If it
+moves an instruction, a type, or a constant value, then something other than a
+`loc` name is sensitive to where lines fall in the file. Find out what before
+accepting it.
+
+The file tests do catch a file left unconverged, since each test re-runs the
+autoupdate in memory and fails with
+`Autoupdate would make changes to the file content` when the result differs. But
+that arrives at `bazelisk test` time, after you have already read a diff that
+was describing a file which had moved out from under it.
+
+### Inspect the diff
+
+Inspect the diagnostics first, since that is the acceptance criterion:
+
+```bash
+# STDERR-only view, with jj.
+jj --no-pager diff --git 'glob:toolchain/*/testdata/**' \
+  | grep -E '^[-+].*CHECK:STDERR'
+
+# The same, with git.
+git diff -- 'toolchain/*/testdata/*' \
+  | grep -E '^[-+].*CHECK:STDERR'
+```
+
+For a structured view separating test input, STDERR, and STDOUT changes, use the
+helper from the
+[Summarize testdata changes](../summarize_testdata_changes/SKILL.md) skill:
+
+```bash
+jj --no-pager diff --git 'glob:toolchain/*/testdata/**' \
+  | python3 .agents/skills/summarize_testdata_changes/scripts/parse_diff.py
+
+git diff -- 'toolchain/*/testdata/*' \
+  | python3 .agents/skills/summarize_testdata_changes/scripts/parse_diff.py
+```
+
+The argument after the tool name is not interchangeable: `glob:...` is a jj
+fileset, `-- ...` is a git pathspec. `--git` and `--no-pager` are jj flags; git
+already emits this format and skips the pager when piped.
+
+### Run the tests
+
+Then run the tests, which is what catches prefix mismatches and non-autoupdated
+expectations:
+
+```bash
+bazelisk test //toolchain/...
+```
+
+See the [Bazel usage](../bazel/SKILL.md) skill.
+
+## Checklist
+
+Before presenting a testdata diff as finished:
+
+-   [ ] The autoupdater was run until it made no further changes, so no `loc`
+        name describes a stale line numbering.
+-   [ ] The list of accepted STDERR changes was written before autoupdating, and
+        every change either matches it or is reported as an exception.
+-   [ ] Every STDOUT change is either an instance of a named mechanical pattern
+        or has its own explanation.
+-   [ ] Instructions that appeared or disappeared are justified by what the
+        changed function owes its caller, not by the instruction count.
+-   [ ] Files whose prefix no longer matches their behavior have been renamed.
+-   [ ] No test input was changed, and no test was deleted, to make the diff
+        smaller.
+-   [ ] The size of the diff is proportional to the size of the change.

.agents/skills/summarize_testdata_changes/SKILL.md

@@ -17,6 +17,10 @@ This skill provides instructions for creating a comprehensive report summarizing
 changes to Carbon testdata files (`toolchain/*/testdata`) and associating them
 with related code changes.
 
+This skill is about _reporting_ a diff. For deciding whether the diff is correct
+in the first place, see the
+[Review testdata changes](../review_testdata_changes/SKILL.md) skill.
+
 ## Goals
 
 Produce a report that:
@@ -40,10 +44,11 @@ input changes.
 
 #### For Git Users:
 
--   **Summarize code changes**: `git diff --stat -- ':!toolchain/*/testdata'`
+-   **Summarize code changes**: `git diff --stat -- ':!toolchain/*/testdata/*'`
     -   To see content of non-testdata changes:
-        `git diff -- ':!toolchain/*/testdata'`
--   **Identify testdata changes**: `git diff --name-only 'toolchain/*/testdata'`
+        `git diff -- ':!toolchain/*/testdata/*'`
+-   **Identify testdata changes**:
+    `git diff --name-only 'toolchain/*/testdata/*'`
 
 #### For Jujutsu (jj) Users:
 
@@ -80,7 +85,7 @@ STDOUT changes. This script reads a unified diff from stdin.
 
 ```bash
 # For Git:
-git diff -- 'toolchain/*/testdata' | python3 .agents/skills/summarize_testdata_changes/scripts/parse_diff.py
+git diff -- 'toolchain/*/testdata/*' | python3 .agents/skills/summarize_testdata_changes/scripts/parse_diff.py
 
 # For Jujutsu (jj):
 jj diff --git 'toolchain/*/testdata' | python3 .agents/skills/summarize_testdata_changes/scripts/parse_diff.py
@@ -110,7 +115,8 @@ gh pr diff 1234 | python3 .agents/skills/summarize_testdata_changes/scripts/pars
         `// CHECK`), along with diagnostic output changes where relevant
     -   Diagnostic Changes: Changes to diagnostic output (lines prefixed with
         `// CHECK:STDERR`) with no corresponding changes to test inputs
-    -   [Output Type] Changes: Changes to STDOUT (lines prefixed with `// CHECK:STDOUT`)
+    -   [Output Type] Changes: Changes to STDOUT (lines prefixed with `//
+        CHECK:STDOUT`)
         -   Create one section for each relevant kind of test. For example,
             parser tests should typically be in a "Parse Tree Changes" section,
             check tests should typically be in a "SemIR Changes" section, and

.agents/skills/summarize_testdata_changes/scripts/parse_diff.py

@@ -6,7 +6,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 import sys
 from collections import defaultdict
-from typing import TextIO, Dict, List
+from typing import Dict, List, TextIO
 
 
 def parse_diff(stream: TextIO) -> None:

.agents/skills/tool_usage/SKILL.md

@@ -1,34 +0,0 @@
----
-name: Tool usage
-description:
-    Instructions for AI assistants on what tools to use in the carbon-lang
-    project.
----
-
-# Tool usage
-
-<!--
-Part of the Carbon Language project, under the Apache License v2.0 with LLVM
-Exceptions. See /LICENSE for license information.
-SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
--->
-
-## Bazelisk and Bazel
-
-We use `bazelisk` for build and test.
-
-**IMPORTANT**: AI assistants use `bazelisk` instead of `bazel`.
-
-## Pre-commit
-
-Running `pre-commit` is mandatory. To run it on all files:
-
-```bash
-pre-commit run -a
-```
-
-To validate a specific list of files:
-
-```bash
-pre-commit run --files <files>
-```

.agents/skills/toolchain_development/SKILL.md

@@ -31,6 +31,14 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
     -   Refer to [Toolchain Idioms](/toolchain/docs/idioms.md) for a
         comprehensive list of patterns (for example, `ValueStore`, formatting
         `.def` files, struct reflection) used throughout the implementation.
+-   **Builtin Functions**: Refer to the **Builtin functions** skill
+    ([SKILL.md](../builtins/SKILL.md)) for guidelines on registering, mapping,
+    constant evaluating, and lowering compiler builtin primitives (e.g.
+    `"int.convert_float"`).
+-   **Language server**: Refer to the **Language server** skill
+    ([SKILL.md](../language_server/SKILL.md)) before working on
+    `toolchain/language_server/` or `utils/vscode/`. Neither follows the
+    patterns described here.
 -   **Phases**: Lex -> Parse -> Check -> Lower.
 -   **Definitions**: Many kinds (tokens, parse nodes, SemIR instructions) are
     defined in `.def` files and expanded by way of macros.
@@ -45,8 +53,8 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 -   **Test everything**: `bazelisk test //...`
 -   **Test specific target**: `bazelisk test //toolchain/testing:file_test`
--   **Test specific file**:
-    `bazelisk test //toolchain/testing:file_test --test_arg=--file_tests=<path_to_carbon_file>`
+-   **Test specific file**: `bazelisk test //toolchain/testing:file_test
+    --test_arg=--file_tests=<path_to_carbon_file>`
 -   **Build toolchain**: `bazelisk build //toolchain/...`
 
 ### Updating test data
@@ -68,6 +76,10 @@ script:
 
 ## Debugging and diagnostics
 
+-   **Compiler Diagnostics**: Refer to the **Diagnostics** skill
+    ([SKILL.md](../diagnostics/SKILL.md)) for strict rules on declaring,
+    formatting, emitting, testing, and styling compiler diagnostic messages
+    (errors, warnings, notes).
 -   **Printing to stderr**: Use `llvm::errs() << "debug info\n";`.
     -   Avoid `std::cout` (it may interfere with tool output).
 -   **SemIR Stringification**:
@@ -86,6 +98,20 @@ script:
 -   **`llvm::Expected<T>`**: Similar to `ErrorOr`, used when interfacing with
     LLVM.
 
+### Context-Aware Diagnostics
+
+When declaring and emitting errors, ensure semantic wording matches the exact
+context:
+
+-   **Semantic Precision**: Do not reference "types" when raising errors for
+    unsized expressions like `IntLiteral` or `FloatLiteral`. For example, use
+    `RealLiteralTooLargeForUnsizedInt` instead of a diagnostic referencing an
+    "integer type".
+-   **Wording Consistency**: Before declaring a new diagnostic in
+    [kind.def](../../../toolchain/diagnostics/kind.def), search for existing
+    diagnostics in the targeted implementation files (for example, other uses of
+    `MaxIntWidth`) to align message structures and parameter expectations.
+
 ### Casting (LLVM style)
 
 -   Use `llvm::cast<T>(obj)` (checked, asserts on failure).
@@ -93,6 +119,16 @@ script:
 -   Use `llvm::isa<T>(obj)` (boolean check).
 -   **Avoid** `dynamic_cast` and standard RTTI.
 
+### Leverage LLVM APIs
+
+Before implementing custom algorithms for mathematical, logical, or bitwise
+operations, inspect target LLVM ADT class APIs:
+
+-   **Builtin APIs**: Verify if LLVM classes (such as `APInt`, `APFloat`, or
+    `APSInt`) already offer native equivalents (for example, `.pow()`,
+    `ilogb()`, `.changeSign()`, `convertFromAPInt()`). Avoid duplicate, naive,
+    or inefficient custom loops.
+
 ### Data structures
 
 -   Prefer APIs in `common/` and `toolchain/base/` over LLVM ADTs. For example,
@@ -113,3 +149,21 @@ script:
     `clang-format`).
 5.  **Parse node order**: Semantics processes parse nodes in post-order; ensure
     your parser transitions support this.
+6.  **Builtin implementation gaps**: If adding a primitive builtin function,
+    make sure you address all phases of the lifecycle: macro definition
+    registration, signature validation, compile-time constant evaluation
+    (interpreter), LLVM IR lowering, and prelude modular implementation bindings
+    (avoiding orphan rules). Refer to the **Builtin functions** skill
+    ([SKILL.md](../builtins/SKILL.md)) for details.
+7.  **Premature helper abstraction**: Avoid extracting tiny helper functions
+    that are called from exactly one place and do not significantly modularize
+    complex code. Prefer inlining directly to keep the implementation compact,
+    readable, and localized.
+8.  **Redundant bounds calculations**: Avoid repeating calculations of complex
+    boundary estimations (such as lower and upper bound estimations). Refactor
+    the logic to calculate unified values once, preserving compactness.
+9.  **Trusting stale `clangd` diagnostics**: In-editor diagnostics are only as
+    good as `compile_commands.json`. If it predates a newly added file, `clangd`
+    falls back to a default command and reports nonsense, such as missing
+    standard headers or "no member named `None`". Regenerate it with
+    `./scripts/create_compdb.py`, which only takes a few seconds.

.agents/skills/toolchain_tests/SKILL.md

@@ -23,6 +23,11 @@ Toolchain tests evaluate Carbon source files through Lexing, Parsing, Checking,
 and optionally Lowering. Output (for example SemIR dumps, Clang errors) is
 captured and validated using inline CHECK records.
 
+Language server tests also use `file_test`, but with quite different
+conventions (an LSP message stream, `AUTOUPDATE-SPLIT`, `FROM_FILE_SPLIT`).
+Refer to the **Language server** skill
+([SKILL.md](../language_server/SKILL.md)) for those.
+
 ## Structure and Authoring
 
 ### File Layout and Headers
@@ -53,6 +58,12 @@ prelude file using `// INCLUDE-FILE`. Usually, include
 `primitives.carbon`. This significantly speeds up execution and minimizes STDOUT
 noise.
 
+-   **Builtin Primitive Testing**: Standard operators (such as `+`, `-`, `/`,
+    `<`, etc.) are **not** imported or available inside minimized preludes. To
+    write tests with a minimal prelude footprint, call primitive builtins
+    directly (e.g., `float.negate`, `float.div`) inside your test code to build
+    expressions.
+
 ### Split Tests and `[[@TEST_NAME]]`
 
 A single physical file can test multiple scenarios using split constraints:
@@ -68,9 +79,10 @@ library "[[@TEST_NAME]]";
 // ...
 ```
 
--   Use `library "[[@TEST_NAME]]";` in each split when necessary to prevent name
-    conflicts or redefining the default library.
--   Exactly `[[@TEST_NAME]]` (including the brackets) should be used. The test
+-   **Always** use `library "[[@TEST_NAME]]";` in each split rather than
+    hardcoding the library name. This prevents name conflicts, avoids redefining
+    the default library, and keeps the test code clean and templateable.
+-   Exactly `[[@TEST_NAME]]` (including the brackets) must be used. The test
     infrastructure automatically replaces it with the split's filename minus
     `todo_` and `fail_` prefixes.
 -   **Do not put code that is expected to pass and code that is expected to fail
@@ -97,6 +109,47 @@ may omit `fail_` if it contains a least one split that has a `fail_` prefix.
 Both the `fail_` and `todo_` prefixes are stripped from filename properties like
 `[[@TEST_NAME]]`.
 
+### Constant Evaluation Validation
+
+When testing constant evaluation in semantic checker tests, follow these
+conventions to ensure diagnostic stability and accuracy:
+
+-   **Literal Spelling Canonicalization**: In Semantic IR, real literals
+    (floating-point constants) with identical mathematical values can be
+    assigned distinct internal representation identifiers based on spelling
+    variations in source code. To completely prevent literal spelling mismatches
+    in expected output checks, validation tests must be performed using
+    canonical comparison methods (for example, passing converted values through
+    an `Expect(X as f64)` function).
+-   **Generic Parameters Validation**: To bypass compile-time constraints where
+    local runtime variables are rejected as generic function arguments, test
+    generic type conversions at runtime, and validate compile-time conversions
+    by passing static literal values directly into primitive builtin calls.
+-   **Exhaustive Edge Case Verification**: For complex mathematical algorithms
+    (such as floating-point to integer truncation and rounding), map and execute
+    test constraints covering every code branch, conditional exit, and fallback
+    evaluation path.
+-   **Rounding Threshold Boundaries**: Test cases that land extremely close to
+    mathematical boundaries (for example, floating-point literals representing a
+    tiny fraction above 1.0, such as $2^{30} \times 2^{-30}$ or
+    $10^{10} \times 10^{-10}$, verifying correct exact truncation down to 1 or
+    0).
+-   **Precise Float Literal Spelling**: Spell floating-point literals in test
+    code with exact mathematical precision targeting target thresholds. For
+    example, if testing the smallest fractional increment above 1.0, use the
+    exact hex fractional representation (e.g. `0x1.0000000000001p0`) or a highly
+    precise decimal fractional spelling (e.g. `1.0000000000000001`) instead of
+    coarse fractions like `1.1` to ensure correct boundary assertions.
+-   **Representation Capacity Boundaries**: Explicitly target edge cases near
+    representation limits of target types. Test combinations of mantissas and
+    exponents that yield values exactly on, just below, or just above the
+    capacity limits of fixed-size destination types (e.g. signed/unsigned
+    targets like `i32` or `u32`).
+-   **Zero-Value Sizing Bounds**: Verify boundary inputs of `0` and `0.0`
+    explicitly. Assert that zero inputs are sized and simplified correctly
+    without triggering calculation underflows, division-by-zero errors, or
+    underestimating required bit allocations.
+
 ### Test Code Comments
 
 -   **No agent thinking:** Do not include comments describing your reasoning or
@@ -134,3 +187,7 @@ updater:
 Review the updated test outputs (for example, by way of `git diff`). Ensure
 logic paths are correctly tested rather than producing massive boilerplate
 blocks.
+
+Autoupdating makes the tests pass whether or not the new behavior is correct, so
+deciding that the new output is the output you wanted is a separate step. See
+the [Review testdata changes](../review_testdata_changes/SKILL.md) skill.

.bazelrc

@@ -45,7 +45,11 @@ build --use_target_config_carbon_rules
 # Default to using a disk cache to minimize re-building LLVM and Clang which we
 # try to avoid updating too frequently to minimize rebuild cost. The location
 # here can be overridden in the user configuration where needed.
-common --disk_cache=~/.cache/carbon-lang-build-cache
+#
+# We avoid the disk cache on MacOS because it breaks debugging. When the cache
+# is used, the separate debug symbol files are not perserved.
+common:linux --disk_cache=~/.cache/carbon-lang-build-cache
+common:windows --disk_cache=~/.cache/carbon-lang-build-cache
 # If you'd like a different disk cache size, override it by copying this
 # line to `user.bazelrc` in the repository root and modify the number there.
 common --experimental_disk_cache_gc_max_size=100G
@@ -176,7 +180,6 @@ common --incompatible_disallow_empty_glob
 common --incompatible_disallow_legacy_py_provider
 common --incompatible_disallow_sdk_frameworks_attributes
 common --incompatible_disallow_struct_provider_syntax
-common --incompatible_do_not_split_linking_cmdline
 common --incompatible_dont_enable_host_nonhost_crosstool_features
 common --incompatible_dont_use_javasourceinfoprovider
 common --incompatible_enable_apple_toolchain_resolution

.clang-tidy

@@ -38,14 +38,32 @@ Checks:
   - '-readability-trailing-comma'
   - '-readability-use-anyofallof'
 
+  # These are copies of older google- prefixed rules that have been moved
+  # out of that prefix, but the google- prefix names still exist as aliases to
+  # these. We enable the google- prefix rules and use those in our NOLINT
+  # expressions, so we disable the newer aliased rules.
+  #
+  # Alias for google-explicit-constructor.
+  - '-misc-explicit-constructor'
+  # Alias for google-readability-casting.
+  - '-modernize-avoid-c-style-cast'
+
   # Warns when we have multiple empty cases in switches, which we do for comment
   # reasons.
   - '-bugprone-branch-clone'
+  # We use CRTP inheritence widely and across distant areas of the codebase,
+  # which makes maintaining friend lists for the constructors frustrating.
+  - '-bugprone-crtp-constructor-accessibility'
+  # We shadow methods with CRTP, instead of using virtual, such as for Print().
+  - '-bugprone-derived-method-shadowing-base-method'
   # Frequently warns on multiple parameters of the same type.
   - '-bugprone-easily-swappable-parameters'
   # Finds issues like out-of-memory in main(). We don't use exceptions, so it's
   # unlikely to find real issues.
   - '-bugprone-exception-escape'
+  # We have File class types in different namespaces and we forward declare it,
+  # but don't find this to be problematic.
+  - '-bugprone-forward-declaration-namespace'
   # Doesn't respect `[[clang::enum_extensibility(open)]]`.
   - '-bugprone-invalid-enum-default-initialization'
   # Has false positives in places such as using an argument to declare a name,
@@ -54,6 +72,8 @@ Checks:
   - '-bugprone-macro-parentheses'
   # Conflicts with integer type C++ style.
   - '-bugprone-narrowing-conversions'
+  # We return const references from value stores.
+  - '-bugprone-return-const-ref-from-parameter'
   # Complains about reasonable code like `1 << 20` and would push us away from
   # our integer type C++ style rules.
   - '-bugprone-signed-bitwise'
@@ -73,6 +93,8 @@ Checks:
   # Extremely slow. TODO: Re-enable once
   # https://github.com/llvm/llvm-project/issues/128797 is fixed.
   - '-misc-confusable-identifiers'
+  # We use multiple inheritence without virtual extensively.
+  - '-misc-multiple-inheritance'
   # Overlaps with `-Wno-missing-prototypes`.
   - '-misc-use-internal-linkage'
   # Suggests `std::array`, which we could migrate to, but conflicts with the
@@ -95,17 +117,32 @@ Checks:
   - '-readability-enum-initial-value'
   # Warns too frequently.
   - '-readability-function-cognitive-complexity'
+  # Allows naming styles we don't use, and has errors on our use of `_1`, `_2`
+  # to have multiple unnamed vars in a destructuring declaration.
+  - '-readability-identifier-naming'
+  # Warns on use of CARBON_KIND() and can't use NOLINT effectively inside a
+  # macro.
+  - '-readability-inconsistent-ifelse-braces'
   # Warns in reasonably documented situations.
   - '-readability-magic-numbers'
   # Warns on `= {}` which is also used to indicate which fields do not need to
   # be explicitly initialized in aggregate initialization.
   - '-readability-redundant-member-init'
+  # We generally do want to collapse if statements, and ask for it in review.
+  # But this check ignores when ifs are nested to place comments above/below
+  # the nested if block. And when the outer if block is also initializing a
+  # variable. There are more than a handful of cases where we want to do this,
+  # especially working with LLVM apis like dyn_cast.
+  - '-readability-redundant-nested-if'
   # Broken, wants to remove parens from `*(p + 1)` and `("Foo" + s).str()`.
   # TODO: Re-enable once https://github.com/llvm/llvm-project/issues/192435 and
   # related bugs are fixed.
   - '-readability-redundant-parentheses'
   # Warns when callers use similar names as different parameters.
   - '-readability-suspicious-call-argument'
+  # Low value check, and it's a stylistic choice to use `#if defined(...)` when
+  # paired with `#elif defined(...)`.
+  - '-readability-use-concise-preprocessor-directives'
 
 CheckOptions:
   # Don't warn on structs; done by ignoring when there are only public members.
@@ -121,6 +158,10 @@ CheckOptions:
     value: CamelCase
   - key: readability-identifier-naming.NamespaceCase
     value: CamelCase
+  # Headers re-open LLVM and Clang namespaces to forward-declare their types,
+  # which is much cheaper to compile than including their headers.
+  - key: readability-identifier-naming.NamespaceIgnoredRegexp
+    value: '^(clang|llvm)$'
   - key: readability-identifier-naming.StructCase
     value: CamelCase
   - key: readability-identifier-naming.TemplateParameterCase

.clangd

@@ -6,12 +6,11 @@ CompileFlags:
   # Workaround for https://github.com/clangd/clangd/issues/1582
   Remove: [-march=*]
 Diagnostics:
-  # `unused-function`: has false positives due to not performing template
-  #   instantiation. We get a more reliable version of this warning from the
-  #   compiler.
-  # `unused-includes`: has false positives, reporting includes unused when
-  #   they are used. Probably the same root cause as unused-function.
-  Suppress: [unused-function, unused-includes]
+  # `unneeded-internal-declaration`, `unused-function`, `unused-includes`,
+  # `unused-template`: These all have false positives due to not performing
+  # template instantiation. We get a more reliable version of these warnings
+  # from the compiler.
+  Suppress: [unneeded-internal-declaration, unused-function, unused-includes, unused-template]
 
 ---
 

.codespell_ignore

@@ -3,6 +3,7 @@
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 AggregateT
+AnyOther
 ArchType
 atleast
 circularly
@@ -17,6 +18,7 @@ groupt
 indext
 inout
 isELF
+iterm
 parameteras
 pullrequest
 rightt

.github/actions/build-setup-common/action.yml

@@ -11,13 +11,12 @@ inputs:
 runs:
   using: composite
   steps:
-    # Setup Python and related tools.
-    - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+    # Setup Python and related tools with uv.
+    - name: Set up uv and Python
+      uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
       with:
-        # Match the min version listed in docs/project/contribution_tools.md
-        # or the oldest version available on the OS.
-        python-version:
-          ${{ inputs.matrix_runner == 'macos-14' && '3.11' || '3.10' }}
+        enable-cache: true
+        version: '0.11.15'
 
     - uses: ./.github/actions/build-setup-macos
       if: startsWith(inputs.matrix_runner, 'macos')
@@ -38,9 +37,10 @@ runs:
         bazelisk --version
         echo '*** run_bazel.py'
         ./scripts/run_bazel.py --version
-        echo '*** python'
-        which python
-        python --version
+        echo '*** uv'
+        which uv
+        uv --version
+        uv python list --only-installed
         echo '*** clang'
         which clang
         clang --version

.github/actions/build-setup-macos/action.yml

@@ -23,7 +23,7 @@ runs:
           xcrun simctl delete all
         sudo rm -rf ~/Library/Developer/CoreSimulator/Caches/*
 
-    # Install and cache LLVM 19 from Homebrew. Some runners may have LLVM 19,
+    # Install and cache LLVM 21 from Homebrew. Some runners may have LLVM 21,
     # but this is reliable (including with libc++), and gives us testing at the
     # minimum supported LLVM version.
     - name: Cache Homebrew
@@ -46,7 +46,7 @@ runs:
           }}
         # Note the key needs to include all the packages we're adding.
         key:
-          Homebrew-Cache-${{ inputs.matrix_runner }}-${{ runner.arch }}-llvm@19
+          Homebrew-Cache-${{ inputs.matrix_runner }}-${{ runner.arch }}-llvm@21
 
     - name: Install LLVM and Clang with Homebrew
       if: steps.cache-homebrew-macos.outputs.cache-hit != 'true'
@@ -60,11 +60,11 @@ runs:
         LEAVES=$(brew leaves | egrep -v '^(bazelisk|gh|git|git-lfs|gnu-tar|go@.*|jq|pipx|node@.*|openssl@.*|wget|yq|zlib)$')
         brew uninstall -f --ignore-dependencies $LEAVES
         echo '*** Installing LLVM deps'
-        brew install --force-bottle --only-dependencies llvm@19
+        brew install --force-bottle --only-dependencies llvm@21
         echo '*** Installing LLVM itself'
-        brew install --force-bottle --force --verbose llvm@19
-        echo '*** brew info llvm@19'
-        brew info llvm@19
+        brew install --force-bottle --force --verbose llvm@21
+        echo '*** brew info llvm@21'
+        brew info llvm@21
         echo '*** brew autoremove'
         brew autoremove
         echo '*** brew info'
@@ -77,7 +77,7 @@ runs:
     - name: Setup LLVM and Clang
       shell: bash
       run: |
-        LLVM_PATH="$(brew --prefix llvm@19)"
+        LLVM_PATH="$(brew --prefix llvm@21)"
         echo "Using ${LLVM_PATH}"
         echo "${LLVM_PATH}/bin" >> $GITHUB_PATH
         echo '*** ls "${LLVM_PATH}"'

.github/actions/build-setup-ubuntu/action.yml

@@ -22,6 +22,16 @@ runs:
         # to save time.
         large-packages: false
 
+    # Select the LLVM release - by the cache key and the download.
+    - name: Select LLVM release
+      shell: bash
+      run: |
+        if [[ "${{ runner.arch }}" == "ARM64" ]]; then
+          echo "LLVM_RELEASE=21.1.8" >> "$GITHUB_ENV"
+        else
+          echo "LLVM_RELEASE=21.1.8" >> "$GITHUB_ENV"
+        fi
+
     # Cache and install a recent version of LLVM. This uses the GitHub action
     # cache to avoid directly downloading on each iteration and improve
     # reliability.
@@ -30,15 +40,16 @@ runs:
       uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
       with:
         path: ~/llvm
-        key: LLVM-19.1.7-Cache-ubuntu-${{ runner.arch }}
+        key: LLVM-${{ env.LLVM_RELEASE }}-Cache-ubuntu-${{ runner.arch }}
 
     - name: Download LLVM and Clang installation
       if: steps.cache-llvm-ubuntu.outputs.cache-hit != 'true'
       shell: bash
       run: |
         cd ~
-        LLVM_RELEASE=19.1.7
-        LLVM_TARBALL_NAME=LLVM-$LLVM_RELEASE-Linux-X64
+        # `LLVM_RELEASE` comes from the "Select LLVM release" step; `runner.arch`
+        # (`X64`/`ARM64`) matches the package's arch suffix.
+        LLVM_TARBALL_NAME=LLVM-$LLVM_RELEASE-Linux-${{ runner.arch }}
         LLVM_PATH=~/llvm
         echo "*** Downloading $LLVM_RELEASE"
         wget --show-progress=off "https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_RELEASE/$LLVM_TARBALL_NAME.tar.xz"
@@ -50,10 +61,12 @@ runs:
         echo "*** Testing `clang++ --version`"
         $LLVM_PATH/bin/clang++ --version
         # The installation contains *huge* parts of LLVM we don't need for the
-        # toolchain. Prune them here to keep our cache small.
+        # toolchain. Prune them here to keep our cache small. x86-64 and
+        # AArch64 use different LLVM releases whose tool sets differ, so `-f`
+        # ignores entries that are absent from a given package.
         echo "*** Cleaning the 'llvm' directory"
-        rm $LLVM_PATH/lib/{*.a,*.so,*.so.*}
-        rm $LLVM_PATH/bin/{flang-*,mlir-*,clang-{scan-deps,check,repl},*-test,llvm-{lto*,reduce,bolt*,exegesis,jitlink},bugpoint,opt,llc}
+        rm -f $LLVM_PATH/lib/{*.a,*.so,*.so.*}
+        rm -f $LLVM_PATH/bin/{flang-*,mlir-*,clang-{scan-deps,check,repl},*-test,llvm-{lto*,reduce,bolt*,exegesis,jitlink},bugpoint,opt,llc}
         echo "*** Size of the 'llvm' directory"
         du -hs $LLVM_PATH
 

.github/workflows/README.md

@@ -19,7 +19,7 @@ Most jobs only have a few endpoints, but due to tools which do downloads, a few
 have significantly more. These are:
 
 -   clangd_tidy.yaml (Bazel)
--   pre_commit.yaml (Bazel, pre-commit)
+-   prek.yaml (Bazel, prek)
 -   nightly_release.yaml (Bazel)
 -   tests.yaml (Bazel)
 

.github/workflows/check_dependent_pr.yaml

@@ -27,23 +27,28 @@ jobs:
           egress-policy: block
           allowed-endpoints: >
             api.github.com:443 github.com:443 pypi.org:443
-            files.pythonhosted.org:443
+            files.pythonhosted.org:443 raw.githubusercontent.com:443
+            releases.astral.sh:443
 
       # Note: pull_request_target checks out the base branch by default.
       # This is safe as it avoids running untrusted code from the PR branch.
       - name: Checkout code
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
 
-      - name: Install dependencies
-        run: |
-          python3 -m pip install gql==2.0.0 requests
+      - name: Set up uv
+        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+        with:
+          enable-cache: true
+          version: '0.11.15'
 
       - name: Check Dependent PR
         run: |
           if [ "$EVENT_ACTION" = "closed" ]; then
-            python3 github_tools/check_dependent_pr.py --scan
+            ./github_tools/check_dependent_pr.py --scan
           else
-            python3 github_tools/check_dependent_pr.py --pr-number "${PR_NUMBER}"
+            ./github_tools/check_dependent_pr.py --pr-number "${PR_NUMBER}"
           fi
         env:
           GITHUB_ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/clangd_tidy.yaml

@@ -48,21 +48,27 @@ jobs:
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
             pypi.org:443
+            raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
+            releases.astral.sh:443
             releases.bazel.build:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
 
       - id: filter
         uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
+          predicate-quantifier: 'every'
           filters: |
             has_cpp:
               - added|modified: '{**/*.cpp,**/*.h}'
+              - '!**/*.tpl.h'
           list-files: 'shell'
 
       - uses: ./.github/actions/build-setup-common
@@ -75,13 +81,9 @@ jobs:
         if: steps.filter.outputs.has_cpp == 'true'
         run: ./scripts/create_compdb.py
 
-      - name: Install clangd-tidy
-        if: steps.filter.outputs.has_cpp == 'true'
-        run: pip install clangd-tidy==1.1.0.post2
-
       - name: Run clangd-tidy
         if: steps.filter.outputs.has_cpp == 'true'
         env:
           FILTER_FILES: ${{ steps.filter.outputs.has_cpp_files }}
         run: |
-          clangd-tidy -p . -j 10 $FILTER_FILES
+          uvx --with clangd-tidy==1.1.0.post2 clangd-tidy -p . -j 10 $FILTER_FILES

.github/workflows/gh_pages_ci.yaml

@@ -28,6 +28,15 @@ jobs:
 
       - name: Checkout
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+        with:
+          enable-cache: true
+          version: '0.11.15'
+
       - name: Prebuild actions
         run: ./website/prebuild.py
       - name: Setup Ruby

.github/workflows/gh_pages_deploy.yaml

@@ -18,15 +18,13 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
   cancel-in-progress: true
 
-# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages.
-permissions:
-  contents: read
-  pages: write
-  id-token: write
+permissions: {}
 
 jobs:
   build:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
     steps:
       - name: Harden Runner
         uses: step-security/harden-runner@58077d3c7e43986b6b15fba718e8ea69e387dfcc # v2.15.1
@@ -35,6 +33,15 @@ jobs:
 
       - name: Checkout
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+        with:
+          enable-cache: true
+          version: '0.11.15'
+
       - name: Prebuild actions
         run: ./website/prebuild.py
       - name: Setup Pages
@@ -49,11 +56,12 @@ jobs:
       - name: Build with Jekyll
         env:
           JEKYLL_ENV: production
+          STEPS_PAGES_OUTPUTS_BASE_PATH: ${{ steps.pages.outputs.base_path }}
         run: |
           bundle exec jekyll build --verbose \
             --source ./ \
             --destination ./_site \
-            --baseurl "${{ steps.pages.outputs.base_path }}"
+            --baseurl "${STEPS_PAGES_OUTPUTS_BASE_PATH}"
       - name: Upload artifact
         # Automatically uploads an artifact from the './_site' directory by
         # default.
@@ -65,6 +73,10 @@ jobs:
       url: ${{ steps.deployment.outputs.page_url }}
     runs-on: ubuntu-latest
     needs: build
+    # Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages.
+    permissions:
+      pages: write
+      id-token: write
     steps:
       - name: Harden Runner
         uses: step-security/harden-runner@58077d3c7e43986b6b15fba718e8ea69e387dfcc # v2.15.1

.github/workflows/nightly_release.yaml

@@ -59,6 +59,7 @@ jobs:
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
             pypi.org:443
+            raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
             releases.bazel.build:443
@@ -68,6 +69,8 @@ jobs:
 
       - name: Checkout branch
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
 
       - name: Set up remote cache access
         env:
@@ -91,7 +94,7 @@ jobs:
           ./scripts/run_bazel.py \
             --attempts=5 --jobs-on-last-attempt=4 \
             test -c opt --stamp --remote_download_toplevel \
-            --pre_release=nightly --nightly_date=${{ env.nightly_date }} \
+            --pre_release=nightly --nightly_date=${nightly_date} \
             //toolchain \
             //toolchain/install:carbon_toolchain_tar_gz \
             //toolchain/install:carbon_toolchain_tar_gz_test
@@ -112,8 +115,8 @@ jobs:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           gh release create \
-            --title "Nightly build ${{ env.nightly_date }}" \
+            --title "Nightly build ${nightly_date}" \
             --generate-notes \
             --prerelease \
-            v${{ env.release_version }} \
-            "bazel-bin/toolchain/install/carbon_toolchain-${{ env.release_version }}.tar.gz"
+            v${release_version} \
+            "bazel-bin/toolchain/install/carbon_toolchain-${release_version}.tar.gz"

.github/workflows/prek.yaml

@@ -2,7 +2,7 @@
 # Exceptions. See /LICENSE for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
-name: pre-commit
+name: prek
 
 on:
   pull_request:
@@ -14,7 +14,7 @@ permissions:
   contents: read # For actions/checkout.
 
 jobs:
-  pre-commit:
+  prek:
     runs-on: ubuntu-22.04
     steps:
       - name: Harden Runner
@@ -40,15 +40,18 @@ jobs:
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
             pypi.org:443
+            raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
+            releases.astral.sh:443
             releases.bazel.build:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
-      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          persist-credentials: false
 
       # Ensure LLVM is set up consistently.
       - uses: ./.github/actions/build-setup-common
@@ -56,22 +59,22 @@ jobs:
           matrix_runner: ubuntu-22.04
           remote_cache_upload: '--remote_upload_local_results=false'
 
-      - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
+      - uses: j178/prek-action@01345c78b7de7d79edf368729212760396ba9345 # v2
 
-      # We want to automatically create github suggestions for pre-commit file
+      # We want to automatically create github suggestions for prek file
       # changes for a pull request. But `pull_request` actions never have write
       # permissions to the repository, so we create the suggestions in a separate
-      # privileged `workflow_run` action in pre_commit_suggestions.yaml. Here,
+      # privileged `workflow_run` action in prek_suggestions.yaml. Here,
       # we upload the diffs and event configuration to an artifact for use by
       # that action.
-      - name: Collect pre-commit output
+      - name: Collect prek output
         if: failure()
         run: |
-          mkdir -p pre-commit-output
-          git diff > pre-commit-output/diff
-          cp $GITHUB_EVENT_PATH pre-commit-output/event
+          mkdir -p prek-output
+          git diff > prek-output/diff
+          cp $GITHUB_EVENT_PATH prek-output/event
       - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         if: failure()
         with:
-          name: pre-commit output
-          path: pre-commit-output/*
+          name: prek output
+          path: prek-output/*

.github/workflows/prek_suggestions.yaml

@@ -2,11 +2,11 @@
 # Exceptions. See /LICENSE for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
-# Create PR suggestions based on problems found by pre-commit action.
-name: 'Add pre-commit suggestions'
+# Create PR suggestions based on problems found by prek action.
+name: 'Add prek suggestions'
 
-# This action is run whenever the `pre-commit` action finishes. Because the
-# `pre-commit` action is an unprivileged action running on (for example) the
+# This action is run whenever the `prek` action finishes. Because the
+# `prek` action is an unprivileged action running on (for example) the
 # `pull_request` event, it's run without write permissions to the repository, so
 # we use a separate privileged `workflow_run` action here to pick up its results
 # and convert them into suggestion comments.
@@ -15,7 +15,7 @@ name: 'Add pre-commit suggestions'
 # this file will not take effect until they are merged to trunk.
 on:
   workflow_run:
-    workflows: [pre-commit]
+    workflows: [prek]
     types:
       - completed
 
@@ -25,7 +25,7 @@ permissions:
 
 jobs:
   pull-request-suggestions:
-    # Only generate suggestions if pre-commit for a PR failed.
+    # Only generate suggestions if prek for a PR failed.
     if: |
       github.event.workflow_run.conclusion == 'failure' &&
       github.event.workflow_run.event == 'pull_request'
@@ -48,16 +48,18 @@ jobs:
           reviewdog_version: latest
 
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
 
-      - name: Download pre-commit output
+      - name: Download prek output
         uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
         with:
-          name: pre-commit output
+          name: prek output
           github-token: ${{ secrets.GITHUB_TOKEN }}
           run-id: ${{ github.event.workflow_run.id }}
 
       # Use https://github.com/reviewdog/reviewdog to create PR suggestions
-      # matching the diff that pre-commit created.
+      # matching the diff that prek created.
       - name: Create suggestions
         env:
           REVIEWDOG_GITHUB_API_TOKEN:

.github/workflows/sync_repos.yaml

@@ -32,6 +32,8 @@ jobs:
       # Checkout our main repository.
       - name: Checkout the main repository
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
 
       # Run the sync script.
       - name: Sync to other repositories

.github/workflows/tests.yaml

@@ -27,8 +27,9 @@ jobs:
       matrix.config.name) || '' }} (${{ matrix.runner }})
     strategy:
       matrix:
-        # Test a recent version of each supported OS.
-        runner: ['ubuntu-22.04', 'macos-14']
+        # Test a recent version of each supported OS, covering both x86-64 and
+        # AArch64: Linux on x86-64 and AArch64, and macOS on AArch64.
+        runner: ['ubuntu-22.04', 'ubuntu-22.04-arm', 'macos-14']
         # Create a synthetic matrix dimension with the event name for filtering.
         event: ['${{ github.event_name }}']
         config:
@@ -70,14 +71,18 @@ jobs:
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
             pypi.org:443
+            raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
+            releases.astral.sh:443
             releases.bazel.build:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
 
       - id: test-setup
         uses: ./.github/actions/test-setup

.gitignore

@@ -14,7 +14,8 @@
 /examples/**/bazel-*
 /examples/**/MODULE.bazel.lock
 
-# Directories created by python.
+# Files and directories created by python.
+uv.lock
 **/__pycache__/
 
 # Ignore the user's VSCode settings and debug setup.
@@ -49,3 +50,10 @@
 
 # Ignore the .gdb_history that's created next to the project-specific .gdbinit
 .gdb_history
+
+# Generated by scripts/create_compdb.py
+/external
+
+# Linux perftools output
+perf.data
+perf.data.old

.pre-commit-config.yaml

@@ -11,8 +11,7 @@ default_language_version:
   python: python3 # Defaults to python2, so override it.
 
 repos:
-  - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # frozen: v6.0.0
+  - repo: builtin
     hooks:
       - id: check-added-large-files
       - id: check-case-conflict
@@ -29,6 +28,15 @@ repos:
         exclude: '^(.*/fuzzer_corpus/.*|.*\.svg)$'
       - id: trailing-whitespace
         exclude: '^(.*/fuzzer_corpus/.*|.*/testdata/.*\.golden|.*\.svg)$'
+
+  # Run markdown linting early so that doc style and table-of-contents see the
+  # linted state.
+  - repo: https://github.com/rvben/rumdl-pre-commit
+    rev: v0.2.58
+    hooks:
+      - id: rumdl
+        args: [--fix]
+
   - repo: https://github.com/google/pre-commit-tool-hooks
     rev: efaea7c61c774c0b1a9805fd999e754a2d19dbd1 # frozen: v1.2.5
     hooks:
@@ -39,6 +47,15 @@ repos:
             .*AGENTS.md
           )$
       - id: markdown-toc
+
+  # Re-run markdown linting to fix any issues caused by doc style and TOC. This
+  # is very fast, so it shouldn't be problematic to run twice.
+  - repo: https://github.com/rvben/rumdl-pre-commit
+    rev: v0.2.58
+    hooks:
+      - id: rumdl
+        args: [--fix]
+
   - repo: local
     hooks:
       - id: fix-cc-deps
@@ -49,10 +66,29 @@ repos:
         pass_filenames: false
 
   # Formatters should be run late so that they can re-format any prior changes.
-  - repo: https://github.com/psf/black
-    rev: c6755bb741b6481d6b3d3bb563c83fa060db96c9 # frozen: 26.3.1
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: 0c7b6c989466a93942def1f84baf36ddfcd60c83 # frozen: v0.15.14
     hooks:
-      - id: black
+      - id: ruff-check
+        args: [--fix]
+      - id: ruff-format
+
+  - repo: local
+    hooks:
+      - id: ty
+        name: ty
+        entry: ty check --no-progress
+        language: python
+        additional_dependencies:
+          - ty==0.0.46
+          - rich
+          - 'gql>=2.0.0,<3.0.0'
+          - PyGitHub
+          - types-requests
+          - requests
+        types: [python]
+        pass_filenames: false
+
   - repo: local
     hooks:
       - id: prettier
@@ -61,7 +97,7 @@ repos:
         # TODO: Not upgrading to/past 3.4.0 due to list indent changes that may
         # get fixed. See: https://github.com/prettier/prettier/issues/16929
         additional_dependencies: ['prettier@3.3.3']
-        types_or: [html, javascript, json, markdown, yaml]
+        types_or: [html, javascript, json, yaml]
         entry: npx prettier@3.3.3 --write --log-level=warn
   - repo: local
     hooks:
@@ -69,7 +105,7 @@ repos:
         name: Bazel buildifier
         entry: scripts/run_buildifier.py
         # Beyond just formatting, explicitly fix lint warnings.
-        args: ['--lint=fix', '--warnings=all', '-r', '.']
+        args: ['--lint=fix', '--warnings=all']
         language: python
         files: |
           (?x)^(
@@ -112,6 +148,23 @@ repos:
         entry: scripts/check_sha_filenames.py
         language: python
         files: ^.*/fuzzer_corpus/.*$
+      - id: check-proposal-names
+        name: Check proposal names
+        entry: proposals/scripts/check_proposal_names.py
+        language: python
+        files: ^proposals/p.*\.md$
+        # This edits files other than the ones passed to it, so we need each
+        # chunk of files to be run through the script serially.
+        require_serial: true
+        # This also renames files, invalidating the list of files provided to
+        # subsequent checks so we fail-fast if this makes changes.
+        fail_fast: true
+      - id: build-textmate-grammar
+        name: Build TextMate grammar
+        entry: scripts/update_tm_language.py
+        language: system
+        files: ^utils/vscode/carbon\.tmLanguage\.json$
+        pass_filenames: false
       - id: check-toolchain-diagnostics
         name: Check toolchain diagnostics
         entry: toolchain/diagnostics/check_diagnostics.py
@@ -134,36 +187,6 @@ repos:
         language: python
         files: ^.*/BUILD$
         pass_filenames: false
-  - repo: https://github.com/PyCQA/flake8
-    rev: d93590f5be797aabb60e3b09f2f52dddb02f349f # frozen: 7.3.0
-    hooks:
-      - id: flake8
-  - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: 'fc0f09a29bb495f4a91f00266155d6282d52485d' # frozen: v1.20.2
-    hooks:
-      - id: mypy
-        # Use setup.cfg to match the command line.
-        args:
-          - --config-file=setup.cfg
-        # This should match the requirements added in the WORKSPACE pip_install.
-        additional_dependencies:
-          - gql >= 2.0.0, < 3.0.0
-          - PyGitHub
-          - rich
-          - types-requests
-        # Exclusions are:
-        # - p#### scripts because they're not tested or maintained.
-        # - lit.cfg.py because it has multiple copies, breaking mypy.
-        # - `bazel_test_runner.py` which depends on Bazel-specific imports.
-        # - Unit tests because they sometimes violate typing, such as by
-        #   assigning a mock to a function.
-        exclude: |
-          (?x)^(
-              proposals/(?!scripts/).*|
-              .*/lit\.cfg\.py|
-              examples/bazel_test_runner\.py|
-              .*_test\.py
-          )$
   - repo: https://github.com/codespell-project/codespell
     rev: 2ccb47ff45ad361a21071a7eedda4c37e6ae8c5a # frozen: v2.4.2
     hooks:
@@ -203,7 +226,7 @@ repos:
           - ''
           - '*/'
           - --custom_format
-          - '\.(plist)$'
+          - '\.(plist|tmLanguage)$'
           - '<!--'
           - ''
           - '\-->'
@@ -220,13 +243,14 @@ repos:
           - --custom_format
           - '\.lua$'
           - ''
-          - '-- '
+          - '\-- '
           - ''
         exclude: |
           (?x)^(
               .bazelversion|
               .github/pull_request_template.md|
               .python-version|
+              LICENSE.*|
               compile_flags.txt|
               github_tools/requirements.txt|
               third_party/.*|
@@ -246,6 +270,7 @@ repos:
         name: Check build graph
         entry: scripts/check_build_graph.py
         language: python
+        pass_filenames: false
         files: |
           (?x)^(
             .*BUILD.*|
@@ -257,6 +282,7 @@ repos:
 # This excludes third-party code, and patches to third-party code.
 exclude: |
   (?x)^(
+      \.jj/.*|
       MODULE.bazel.lock|
       .*package-lock\.json|
       bazel/bazel_clang_tidy/.*\.patch|

.python-version

@@ -1 +1 @@
-3.10
+3.12

.rumdl.toml

@@ -0,0 +1,83 @@
+# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+# Exceptions. See /LICENSE for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+[global]
+exclude = [
+  ".clang-tidy",
+  ".git",
+  ".jj",
+  "CHANGELOG.md",
+  "LICENSE.md",
+  ".github/pull_request_template.md",
+]
+respect-gitignore = true
+
+# Disable rules that produce the most noise initially. Some of these might make
+# sense to re-enable.
+disable = [
+  "MD033", # Inline HTML - commonly used in real-world markdown
+  "MD036", # Emphasis used instead of heading
+  "MD040", # Code blocks should have a language specified
+  "MD014", # Commands in code blocks should show output
+  "MD034", # Bare URLs
+  "MD059", # Link text should be descriptive
+  "MD028", # Blank line inside blockquote
+]
+
+# Line wrapping
+[MD013]
+reflow = true
+# Note that we might want to use the "normalize" reflow mode to have more
+# consistent line wrapping, however this mode is currently deeply incompatible
+# with inline HTML that we use reasonably often. For now, we go with the default
+# mode that doesn't try to normalize wrapping.
+reflow-mode = "default"
+ignore-link-urls = false
+code-blocks = false
+code-spans = false
+atomic-spans = false
+headings = false
+stern = true
+
+# Heading style
+[MD003]
+style = "atx"
+
+# Narrow restriction on trailing punctuation in headings -- allows ':' and '!'.
+[MD026]
+punctuation = ".,;"
+
+# Unordered list marker style
+[MD004]
+style = "dash"
+
+# Ordered list numbering
+[MD029]
+style = "one-or-ordered"
+
+# Unordered list indentation
+[MD007]
+style = "fixed"
+indent = 4
+
+[MD077]
+style = "aligned"
+
+# Spaces after list markers
+[MD030]
+ul-single = 3
+ul-multi = 3
+ol-align-column = 4
+
+# Code block style
+[MD046]
+style = "fenced"
+
+# Emphasis style
+[MD049]
+style = "underscore"
+
+# Strong style
+[MD050]
+style = "asterisk"

.vscode/extensions.json

@@ -4,8 +4,9 @@
     "bierner.github-markdown-preview",
     "carbon-lang.carbon-vscode",
     "esbenp.prettier-vscode",
+    "rvben.rumdl",
     "llvm-vs-code-extensions.vscode-clangd",
-    "ms-python.black-formatter",
-    "ms-python.python"
+    "charliermarsh.ruff",
+    "astral-sh.ty"
   ]
 }

.vscode/lldb_launch.json

@@ -19,6 +19,24 @@
         "env TEST_TMPDIR=/tmp"
       ]
     },
+    {
+      "type": "lldb-dap",
+      "request": "launch",
+      "name": "file_test (all files) (lldb)",
+      "program": "bazel-bin/toolchain/testing/file_test",
+      "args": [],
+      "debuggerRoot": "${workspaceFolder}",
+      "initCommands": [
+        "command script import external/+llvm_project+llvm-project/llvm/utils/lldbDataFormatters.py",
+        "command script import scripts/lldbinit.py",
+        "settings append target.source-map \".\" \"${workspaceFolder}\"",
+        "settings append target.source-map \"/proc/self/cwd\" \"${workspaceFolder}\"",
+        "settings set escape-non-printables false",
+        "settings set target.max-string-summary-length 10000",
+        "env TEST_TARGET=//toolchain/testing:file_test",
+        "env TEST_TMPDIR=/tmp"
+      ]
+    },
     {
       "type": "lldb-dap",
       "request": "launch",

AGENTS.md

@@ -1,4 +1,4 @@
-# Gemini & AI assistant guide for Carbon
+# Gemini & AI Assistant Guide for Carbon
 
 <!--
 Part of the Carbon Language project, under the Apache License v2.0 with LLVM
@@ -6,40 +6,14 @@ Exceptions. See /LICENSE for license information.
 SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 -->
 
-This document provides high-density technical context for AI assistants (and
-humans!) contributing to the Carbon Language project. If you are an AI
-assistant, **read this first** to avoid common pitfalls.
-
-## Table of contents
-
--   [General instructions](#general-instructions)
--   [Project structure](#project-structure)
--   [Bazel usage](#bazel-usage)
--   [Toolchain development](#toolchain-development)
+This document provides high-density technical context for AI assistants
+contributing to the Carbon Language project.
 
 ## General instructions
 
 -   **Communication**: Be concise, professional, and technical. Use GitHub-style
     markdown.
 -   **Verification**: Always run relevant tests.
--   **Tool usage**: Use web search for any research outside the immediate
-    codebase or KIs.
-
-## Project structure
-
--   **[`common/`](common/)**: Common C++ utilities used across the project.
--   **[`core/`](core/)**: The Carbon standard library (Core).
--   **[`docs/`](docs/)**: Project documentation, design, and style guides.
--   **[`examples/`](examples/)**: Example Carbon programs and code snippets.
--   **[`proposals/`](proposals/)**: Evolution proposals.
--   **[`testing/`](testing/)**: Testing utilities and infrastructure.
--   **[`toolchain/`](toolchain/)**: The C++ implementation of the compiler
-    (Toolchain).
-
-## Tool usage
-
-See the "Tool usage" skill for instructions on what tools to use in the
-carbon-lang project.
 
 ## Bazel usage
 
@@ -47,12 +21,10 @@ carbon-lang project.
 > Carbon project. Refer to the
 > [Bazel usage skill](/.agents/skills/bazel/SKILL.md) for detailed instructions.
 
-## Code style
+## Version control
 
-See the "Code style" skill for instructions on formatting, style guides, and
-code conventions to follow.
-
-## Toolchain development
-
-See the "Toolchain Development" skill for instructions on architecture,
-building, testing, debugging, C++ patterns, and common pitfalls.
+> [!IMPORTANT] Never rewrite the history of a change that has been submitted as
+> a pull request. Reviewers track a PR by its commits, and rewriting them
+> discards their in-progress review. Ask before rewriting history in any case.
+> Refer to the [Jujutsu (jj) usage skill](/.agents/skills/jj/SKILL.md) for
+> details.

CONTRIBUTING.md

@@ -227,7 +227,7 @@ trying to write proposals, both types of contributor access will help.
 Please see our [contribution tool](/docs/project/contribution_tools.md)
 documentation for information on setting up a git client for Carbon development,
 as well as helpful tooling that will ease the contribution process. For example,
-[pre-commit](https://pre-commit.com) is used to simplify
+[prek](https://github.com/j178/prek) is used to simplify
 [code review](/docs/project/code_review.md).
 
 #### Using AI-based contribution tools
@@ -249,22 +249,22 @@ and our [guidelines and standards](#contribution-guidelines-and-standards)
 below. We also emphasize two additional requirements for contributors operating
 or using AI-based tools:
 
-1. **Contributions should not become extractive of the project and community**:
-   the value added should outweigh the overhead of landing the contribution. The
-   overhead of landing contributions ranges from code review, to discussions,
-   distractions from the current project priorities, or growing maintenance
-   burden without growing maintainers.
+1.  **Contributions should not become extractive of the project and community**:
+    the value added should outweigh the overhead of landing the contribution.
+    The overhead of landing contributions ranges from code review, to
+    discussions, distractions from the current project priorities, or growing
+    maintenance burden without growing maintainers.
 
-2. **Each PR should be transparent about the tooling used** in proportion to how
-   much of the PR was produced by the tool and whether the tool is a standard
-   one for the project. For example, formatting with the standard tools is
-   reasonable to assume without further comment. But if a PR is largely derived
-   from running a specific Python script, regular expression, or AI-based tool
-   over the codebase, we ask that its commit message is transparent about this
-   and include a description of how the tool was used to formulate the change.
-   For PRs largely derived from AI-based tooling, we suggest following the
-   pattern established by the Fedora Project to mark commits with
-   `Assisted-by: ...`.
+2.  **Each PR should be transparent about the tooling used** in proportion to
+    how much of the PR was produced by the tool and whether the tool is a
+    standard one for the project. For example, formatting with the standard
+    tools is reasonable to assume without further comment. But if a PR is
+    largely derived from running a specific Python script, regular expression,
+    or AI-based tool over the codebase, we ask that its commit message is
+    transparent about this and include a description of how the tool was used to
+    formulate the change. For PRs largely derived from AI-based tooling, we
+    suggest following the pattern established by the Fedora Project to mark
+    commits with `Assisted-by: ...`.
 
 Our policies and practices here are inspired by and aim to be roughly compatible
 with several other open source projects:
@@ -405,9 +405,9 @@ respectful, and don't drown out other discussion.
 Changes to Carbon documentation follow the
 [Google developer documentation style guide](https://developers.google.com/style).
 
-Markdown files should additionally use [Prettier](https://prettier.io) for
-formatting, which we automate with
-[pre-commit](/docs/project/contribution_tools.md#main-tools).
+Markdown files should additionally use [rumdl](https://github.com/rvben/rumdl)
+for formatting, which we automate with
+[prek](/docs/project/contribution_tools.md#running-prek).
 
 Other style points to be aware of are:
 

MODULE.bazel

@@ -83,8 +83,8 @@ git_override(
     build_file_content = "# empty",
     # We pin to specific upstream commits and try to track top-of-tree
     # reasonably closely rather than pinning to a specific release.
-    # HEAD as of 2026-04-01.
-    commit = "b71eacea7687f68c11299e3bda5654fbbaa1e20e",
+    # HEAD as of 2026-09-08.
+    commit = "7024b9e1b423b3c3c6ac76ab6a73cb2c9e4ef842",
     patch_cmds = ["echo \"module(name='llvm-raw')\" > MODULE.bazel"],
     patch_strip = 1,
     patches = [
@@ -92,7 +92,8 @@ git_override(
         "//bazel/llvm_project:0002_Added_Bazel_build_for_compiler_rt_fuzzer.patch",
         "//bazel/llvm_project:0004_Introduce_basic_sources_exporting_for_libunwind.patch",
         "//bazel/llvm_project:0005_Introduce_basic_sources_exporting_for_libcxx_and_libcxxabi.patch",
-        "//bazel/llvm_project:0009_Introduce_starlark_exporting_compiler-rt_build_information.patch",
+        "//bazel/llvm_project:0006_Add_more_libc_math_excludes.patch",
+        "//bazel/llvm_project:0011_Temporarily_remove_reference_to_hermetic_toolchain.patch",
     ],
     remote = "https://github.com/llvm/llvm-project.git",
 )
@@ -112,7 +113,7 @@ bazel_dep(name = "rules_python", version = "1.9.0")
 
 python = use_extension("@rules_python//python/extensions:python.bzl", "python")
 python.toolchain(
-    python_version = "3.11",
+    python_version = "3.12",
 )
 use_repo(python, "python_versions")
 

MODULE.bazel.lock

@@ -307,7 +307,7 @@
   "moduleExtensions": {
     "//bazel/cc_toolchains:clang_configuration.bzl%clang_toolchain_extension": {
       "general": {
-        "bzlTransitiveDigest": "H3RsK0MbgutDMSlPWTwZq4Vk1U5sjDtgJ5MXQxg7GLU=",
+        "bzlTransitiveDigest": "IGxGFknaFQQo7RudzfwIOuxREEsnEXwlM9KuPiNQYBM=",
         "usagesDigest": "lTxkeAFhR0iBEa3dg5hWvtd2HFCr5zCJx/fl27A+IKA=",
         "recordedFileInputs": {},
         "recordedDirentsInputs": {},
@@ -323,7 +323,7 @@
     },
     "//bazel/llvm_project:llvm_project.bzl%llvm_project": {
       "general": {
-        "bzlTransitiveDigest": "xDeO6VeJOhQ/KmsAtVhSrY+/XoomqAxdF2SG/XbIkX4=",
+        "bzlTransitiveDigest": "4DgU62e9O5rmV6Yzqa1tjFyBSVED44nXdYn84nxuJC8=",
         "usagesDigest": "uwwVdRj/NhFVoOIaadPP393kgC/Uu3/nTX9ln69oWp4=",
         "recordedFileInputs": {},
         "recordedDirentsInputs": {},

README.md

@@ -251,10 +251,10 @@ challenge for C++ and something a successor language needs to address.
 
 We plan to support a two step migration process:
 
-1. Highly automated, minimal supervision migration from C++ to a dialect of
-   Carbon designed for C++ interop and migration.
-2. Incremental refactoring of the Carbon code to adopt memory-safe designs,
-   patterns, and APIs.
+1.  Highly automated, minimal supervision migration from C++ to a dialect of
+    Carbon designed for C++ interop and migration.
+2.  Incremental refactoring of the Carbon code to adopt memory-safe designs,
+    patterns, and APIs.
 
 We also want to address important, low-hanging fruit in the safety space
 immediately when migrating into Carbon:
@@ -367,8 +367,16 @@ Carbon focused talks from the community:
 
 ### 2026
 
+-   Carbon memory safety: a first deep dive (July 10,
+    [video](https://drive.google.com/file/d/1tQlzpnbWZfn2WtTFMoJgF93QteByBBwm/view?usp=sharing),
+    [transcript](https://docs.google.com/document/d/1JB9H3KzVixAPC5WIytS4AMyrvjwzC7TXqp596veLT34/edit?usp=sharing),
+    [slides](https://chandlerc.blog/slides/2026-memory-safety-deep-3/))
 -   Benchmarking and optimizing the Carbon compiler, NDC {Toronto} (May 5-8)
+    ([video](https://www.youtube.com/watch?v=hN6KcAKfTN0),
+    [slides](https://chandlerc.blog/slides/2026-ndc-toronto-carbon-benchmarking))
 -   Carbon: graduating from the experiment, NDC {Toronto} (May 5-8)
+    ([video](https://www.youtube.com/watch?v=WJl4ftb5Fxg),
+    [slides](https://chandlerc.blog/slides/2026-ndc-toronto-carbon-update/))
 
 ### 2025
 

bazel-execroot

@@ -1 +0,0 @@
-bazel-out/../../_main

bazel/carbon_rules/defs.bzl

@@ -26,8 +26,10 @@ def _carbon_binary_impl(ctx):
     # Pass any C++ flags from our dependencies onto Carbon.
     dep_flags = []
     dep_hdrs = []
+    dep_api_files = []
     dep_link_inputs = []
-    for dep in ctx.attr.deps:
+    deps = ctx.attr.deps + ctx.attr._default_deps
+    for dep in deps:
         if CcInfo in dep:
             cc_info = dep[CcInfo]
 
@@ -46,16 +48,16 @@ def _carbon_binary_impl(ctx):
                     dep_link_inputs += lib.objects
         if DefaultInfo in dep:
             dep_link_inputs += dep[DefaultInfo].files.to_list()
+        if CarbonLibraryInfo in dep:
+            carbon_info = dep[CarbonLibraryInfo]
+            dep_link_inputs += carbon_info.objs.to_list()
+            dep_api_files += carbon_info.apis
 
     # Add the dependencies' link flags and inputs to the link flags.
     link_flags += [dep.path for dep in dep_link_inputs]
 
     # Build object files for the prelude and for the binary itself.
-    # TODO: Eventually the prelude should be build as a separate `carbon_library`.
-    srcs_and_flags = [
-        (ctx.files.prelude_srcs, ["--no-prelude-import"]),
-        (ctx.files.srcs, dep_flags),
-    ]
+    srcs_and_flags = [(ctx.files.srcs, dep_flags)]
 
     objs = []
     for (srcs, extra_flags) in srcs_and_flags:
@@ -76,13 +78,14 @@ def _carbon_binary_impl(ctx):
                 src.short_path.removeprefix(ctx.label.package).removesuffix(src.extension),
             ))
             objs.append(out)
-            srcs_reordered = [s for s in srcs if s != src] + [src]
+            srcs_reordered = dep_api_files + [s for s in srcs if s != src] + [src]
             ctx.actions.run(
                 outputs = [out],
                 inputs = depset(direct = srcs_reordered, transitive = dep_hdrs),
                 executable = toolchain_driver,
                 tools = depset(toolchain_data),
                 arguments = ["compile", "--output=" + out.path, "--output-last-input-only"] +
+                            ["--no-include-carbon-core"] +
                             [s.path for s in srcs_reordered] + extra_flags + ctx.attr.flags,
                 mnemonic = "CarbonCompile",
                 progress_message = "Compiling " + src.short_path,
@@ -119,19 +122,128 @@ def _carbon_binary_impl(ctx):
 
     ctx.actions.run(
         outputs = [bin],
-        inputs = objs + dep_link_inputs,
+        inputs = depset(direct = objs + dep_link_inputs),
         executable = toolchain_driver,
-        tools = depset(toolchain_data + prebuilt_runtimes),
+        tools = depset(direct = toolchain_data + prebuilt_runtimes),
         arguments = full_link_flags,
         mnemonic = "CarbonLink",
         progress_message = "Linking " + bin.short_path,
     )
     return [DefaultInfo(files = depset([bin]), executable = bin)]
 
+CarbonLibraryInfo = provider(
+    doc = "Contains information about a linkage unit of one or more compiled Carbon libraries.",
+    fields = {
+        "apis": "The api source files to provide to library consumers.",
+        "objs": "A depset of one or more compiled library files, including impl and api.",
+    },
+)
+
+def _carbon_library_impl(ctx):
+    toolchain_driver = ctx.executable.internal_exec_toolchain_driver
+    toolchain_data = ctx.files.internal_exec_toolchain_data
+
+    # If the exec driver isn't provided, that means we're trying to use a target
+    # config toolchain, likely to avoid build overhead of two configs.
+    if toolchain_driver == None:
+        toolchain_driver = ctx.executable.internal_target_toolchain_driver
+        toolchain_data = ctx.files.internal_target_toolchain_data
+
+    # Pass any C++ flags from our dependencies onto Carbon.
+    dep_flags = []
+    dep_hdrs = []
+    dep_api_srcs = []
+    for dep in ctx.attr.deps:
+        if CcInfo in dep:
+            cc_info = dep[CcInfo]
+
+            # TODO: We should reuse the feature-based flag generation in
+            # bazel/cc_toolchains here.
+            dep_flags += ["--clang-arg=-D{0}".format(define) for define in cc_info.compilation_context.defines.to_list()]
+            dep_flags += ["--clang-arg=-I{0}".format(path) for path in cc_info.compilation_context.includes.to_list()]
+            dep_flags += ["--clang-arg=-iquote{0}".format(path) for path in cc_info.compilation_context.quote_includes.to_list()]
+            dep_flags += ["--clang-arg=-isystem{0}".format(path) for path in cc_info.compilation_context.system_includes.to_list()]
+            dep_hdrs.append(cc_info.compilation_context.headers)
+        if CarbonLibraryInfo in dep:
+            carbon_info = dep[CarbonLibraryInfo]
+            dep_api_srcs += carbon_info.apis.to_list()
+
+    # Build object files for the library impls and api file
+    srcs_and_flags = [(ctx.files.srcs + ctx.files.hdrs, dep_flags)]
+
+    objs = []
+    for (srcs, extra_flags) in srcs_and_flags:
+        for src in srcs:
+            # Build each source file. For now, we pass all sources to each compile
+            # because we don't have visibility into dependencies and have no way to
+            # specify multiple output files. Object code for each input is written
+            # into the output file in turn, so the final carbon source file
+            # specified ends up determining the contents of the object file.
+            #
+            # TODO: This is a hack; replace with something better once the toolchain
+            # supports doing so.
+            #
+            # TODO: Switch to the `prefix` based rule similar to linking when
+            # the prelude moves there.
+            out = ctx.actions.declare_file("_objs/{0}/{1}o".format(
+                ctx.label.name,
+                src.short_path.removeprefix(ctx.label.package).removesuffix(src.extension),
+            ))
+            objs.append(out)
+            srcs_reordered = dep_api_srcs + [s for s in srcs if s != src] + [src]
+            ctx.actions.run(
+                outputs = [out],
+                inputs = depset(direct = srcs_reordered, transitive = dep_hdrs),
+                executable = toolchain_driver,
+                tools = depset(toolchain_data),
+                arguments = ["compile", "--output=" + out.path, "--output-last-input-only"] +
+                            ["--no-include-carbon-core"] +
+                            extra_flags + ctx.attr.flags + [s.path for s in srcs_reordered],
+                mnemonic = "CarbonCompile",
+                progress_message = "Compiling " + src.short_path,
+            )
+
+    return [CarbonLibraryInfo(apis = ctx.files.hdrs, objs = depset(objs))]
+
+# We synthesize two sets of attributes from mirrored `select`s here
+# because we want to select on an internal property of these attributes
+# but that isn't `select`-able. Instead, we have both attributes and
+# `select` which one we use.
+_select_internal_exec_toolchain_driver = select({
+    Label("//bazel/carbon_rules:use_target_config_carbon_rules_config"): None,
+    "//conditions:default": Label("//toolchain/install:carbon-busybox"),
+})
+_select_internal_exec_toolchain_data = select({
+    Label("//bazel/carbon_rules:use_target_config_carbon_rules_config"): None,
+    "//conditions:default": Label("//toolchain/install:install_data"),
+})
+_select_internal_exec_prebuilt_runtimes = select({
+    Label("//bazel/carbon_rules:use_target_config_carbon_rules_config"): None,
+    "//conditions:default": Label("//toolchain/install:built_runtimes"),
+})
+_select_internal_target_toolchain_driver = select({
+    Label(
+        "//bazel/carbon_rules:use_target_config_carbon_rules_config",
+    ): Label("//toolchain/install:carbon-busybox"),
+    "//conditions:default": None,
+})
+_select_internal_target_toolchain_data = select({
+    Label(
+        "//bazel/carbon_rules:use_target_config_carbon_rules_config",
+    ): Label("//toolchain/install:install_data"),
+    "//conditions:default": None,
+})
+_select_internal_target_prebuilt_runtimes = select({
+    Label(
+        "//bazel/carbon_rules:use_target_config_carbon_rules_config",
+    ): Label("//toolchain/install:built_runtimes"),
+    "//conditions:default": None,
+})
+
 _carbon_binary_internal = rule(
     implementation = _carbon_binary_impl,
     attrs = {
-        "deps": attr.label_list(allow_files = True, providers = [[CcInfo]]),
+        "deps": attr.label_list(allow_files = True, providers = [[CcInfo], [CarbonLibraryInfo]]),
         "flags": attr.string_list(),
 
         # The exec config toolchain attributes. These will be `None` when using
@@ -167,11 +279,58 @@ _carbon_binary_internal = rule(
             executable = True,
             cfg = "target",
         ),
-        "prelude_srcs": attr.label_list(allow_files = [".carbon"]),
+        "srcs": attr.label_list(allow_files = [".carbon"]),
+        "_cc_toolchain": attr.label(default = "//toolchain/install:carbon_stage1_cc_toolchain"),
+        "_default_deps": attr.label_list(default = [Label("//core:io")]),
+    },
+    executable = True,
+    fragments = ["cpp"],
+)
+
+_carbon_library_internal = rule(
+    implementation = _carbon_library_impl,
+    attrs = {
+        "deps": attr.label_list(allow_files = True),
+        "flags": attr.string_list(),
+        "hdrs": attr.label_list(allow_files = [".carbon"]),
+
+        # The exec config toolchain attributes. These will be `None` when using
+        # the target config and populated when using the exec config. We have to
+        # use duplicate attributes here and below to have different `cfg`
+        # settings, as that isn't `select`-able, and we'll use `select`s when
+        # populating these.
+        "internal_exec_prebuilt_runtimes": attr.label(
+            cfg = "exec",
+        ),
+        "internal_exec_toolchain_data": attr.label(
+            cfg = "exec",
+        ),
+        "internal_exec_toolchain_driver": attr.label(
+            allow_single_file = True,
+            executable = True,
+            cfg = "exec",
+        ),
+
+        # The target config toolchain attributes. These will be 'None' when
+        # using the exec config and populated when using the target config. We
+        # have to use duplicate attributes here and below to have different
+        # `cfg` settings, as that isn't `select`-able, and we'll use `select`s
+        # when populating these.
+        "internal_target_prebuilt_runtimes": attr.label(
+            cfg = "target",
+        ),
+        "internal_target_toolchain_data": attr.label(
+            cfg = "target",
+        ),
+        "internal_target_toolchain_driver": attr.label(
+            allow_single_file = True,
+            executable = True,
+            cfg = "target",
+        ),
         "srcs": attr.label_list(allow_files = [".carbon"]),
         "_cc_toolchain": attr.label(default = "//toolchain/install:carbon_stage1_cc_toolchain"),
     },
-    executable = True,
+    executable = False,
     fragments = ["cpp"],
 )
 
@@ -188,37 +347,49 @@ def carbon_binary(name, srcs, deps = [], flags = [], tags = []):
     _carbon_binary_internal(
         name = name,
         srcs = srcs,
-        prelude_srcs = ["//core:prelude_files"],
         deps = deps,
         flags = flags,
         tags = tags,
+        internal_exec_toolchain_driver = _select_internal_exec_toolchain_driver,
+        internal_exec_toolchain_data = _select_internal_exec_toolchain_data,
+        internal_exec_prebuilt_runtimes = _select_internal_exec_prebuilt_runtimes,
+        internal_target_toolchain_driver = _select_internal_target_toolchain_driver,
+        internal_target_toolchain_data = _select_internal_target_toolchain_data,
+        internal_target_prebuilt_runtimes = _select_internal_target_prebuilt_runtimes,
+    )
 
-        # We synthesize two sets of attributes from mirrored `select`s here
-        # because we want to select on an internal property of these attributes
-        # but that isn't `select`-able. Instead, we have both attributes and
-        # `select` which one we use.
-        internal_exec_toolchain_driver = select({
-            "//bazel/carbon_rules:use_target_config_carbon_rules_config": None,
-            "//conditions:default": "//toolchain/install:carbon-busybox",
-        }),
-        internal_exec_toolchain_data = select({
-            "//bazel/carbon_rules:use_target_config_carbon_rules_config": None,
-            "//conditions:default": "//toolchain/install:install_data",
-        }),
-        internal_exec_prebuilt_runtimes = select({
-            "//bazel/carbon_rules:use_target_config_carbon_rules_config": None,
-            "//conditions:default": "//toolchain/install:built_runtimes",
-        }),
-        internal_target_toolchain_driver = select({
-            "//bazel/carbon_rules:use_target_config_carbon_rules_config": "//toolchain/install:carbon-busybox",
-            "//conditions:default": None,
-        }),
-        internal_target_toolchain_data = select({
-            "//bazel/carbon_rules:use_target_config_carbon_rules_config": "//toolchain/install:install_data",
-            "//conditions:default": None,
-        }),
-        internal_target_prebuilt_runtimes = select({
-            "//bazel/carbon_rules:use_target_config_carbon_rules_config": "//toolchain/install:built_runtimes",
-            "//conditions:default": None,
-        }),
+def carbon_library(name, hdrs = [], srcs = [], deps = [], flags = [], tags = [], visibility = []):
+    """Compiles a Carbon library.
+
+    Note: This carbon_library is designed as a _linkage_unit_, and does not necessarily
+    have to correlate the Carbon language library concept. As such it is designed to
+    accommodate more than one api file.
+
+    The arguments `hdrs` and `srcs` are kept for reasons of convention and compatibility
+    with C++ toolchains, particularly build aspects that folks might want to reuse on
+    mixed projects.
+
+    Args:
+      name: The name of the build target.
+      hdrs: List of one or more api files.
+      srcs: List of zero or more implementation files.
+      deps: List of dependencies.
+      flags: Extra flags to pass to the Carbon compile command.
+      tags: Tags to apply to the rule.
+      visibility: Visibility rules for the library.
+    """
+    _carbon_library_internal(
+        name = name,
+        hdrs = hdrs,
+        srcs = srcs,
+        deps = deps,
+        flags = flags,
+        tags = tags,
+        visibility = visibility,
+        internal_exec_toolchain_driver = _select_internal_exec_toolchain_driver,
+        internal_exec_toolchain_data = _select_internal_exec_toolchain_data,
+        internal_exec_prebuilt_runtimes = _select_internal_exec_prebuilt_runtimes,
+        internal_target_toolchain_driver = _select_internal_target_toolchain_driver,
+        internal_target_toolchain_data = _select_internal_target_toolchain_data,
+        internal_target_prebuilt_runtimes = _select_internal_target_prebuilt_runtimes,
     )

bazel/cc_toolchains/carbon_cc_toolchain_config.bzl

@@ -11,6 +11,7 @@ load(
     "flag_group",
     "flag_set",
     "tool",
+    "tool_path",
 )
 load(
     "@rules_cc//cc:defs.bzl",
@@ -142,10 +143,15 @@ def _carbon_cc_toolchain_config_impl(ctx):
     # Only use a sysroot if a non-trivial one is set in Carbon's config.
     builtin_sysroot = None
     sysroot_include_search = []
+    sdk_settings = []
     if clang_sysroot != "None" and clang_sysroot != "/":
         builtin_sysroot = clang_sysroot
         sysroot_include_search = ["%sysroot%/usr/include"]
 
+        # On MacOS, the compiler depends on this file at the root of the SDK,
+        # and it ends up in the `.d` files.
+        sdk_settings = ["%sysroot%/SDKSettings.json"]
+
     runtimes_path = None
     if ctx.attr.runtimes:
         for f in ctx.files.runtimes:
@@ -162,6 +168,7 @@ def _carbon_cc_toolchain_config_impl(ctx):
         ctx.attr.target_cpu,
         ctx.attr.target_os,
     )
+
     return cc_common.create_cc_toolchain_config_info(
         ctx = ctx,
         features = clang_cc_toolchain_features(
@@ -184,7 +191,7 @@ def _carbon_cc_toolchain_config_impl(ctx):
             "runtimes/libcxxabi/include",
             "{}/include".format(clang_resource_dir),
             "runtimes/clang_resource_dir/include",
-        ] + _compute_clang_system_include_dirs() + sysroot_include_search,
+        ] + _compute_clang_system_include_dirs() + sysroot_include_search + sdk_settings,
         builtin_sysroot = builtin_sysroot,
 
         # This configuration only supports local non-cross builds so derive
@@ -197,7 +204,7 @@ def _carbon_cc_toolchain_config_impl(ctx):
 
         # Pass in our tool paths to expose Make variables like $(NM) and
         # $(OBJCOPY).
-        tool_paths = llvm_tool_paths(llvm_bindir, clang_bindir),
+        tool_paths = llvm_tool_paths(llvm_bindir, clang_bindir) + [tool_path(name = "carbon-busybox", path = "carbon-busybox")],
     )
 
 carbon_cc_toolchain_config = rule(

bazel/cc_toolchains/cc_toolchain_cpp_features.bzl

@@ -17,10 +17,49 @@ load(
     "preprocessor_compile_actions",
 )
 
+# Sysroots and MacOS are complicated:
+#
+# On Darwin/MacOS, the `-isysroot` flag is used for includes *and* libraries,
+# and if specified it wins over `--sysroot` which would be used for libraries
+# on other platforms.
+# https://discourse.llvm.org/t/silly-what-is-the-difference-between-sysroot-and-isysroot/55788/2
+#
+# Additionally, on a MacOS build of clang, the sysroot defaults to `/`, which
+# is incorrect and it needs to be pointed to the SDK root. However, as a
+# convenience, homebrew builds of clang automatically add `-isysroot` to the
+# command line, so that the user doesn't have to. But the SDK it chooses does
+# not always match the one returned from `xcrun --show-sdk-path`, which is the
+# SDK that we want to use. So we need to override homebrew's choice and specify
+# `-isysroot`. This will also supersede anything given to `--sysroot` (on
+# Darwin) so we don't need to specify both. For non-homebrew clang builds on
+# MacOS, specifying `-isysroot` will also work to point the compiler to the
+# correct SDK instead of `--sysroot`.
+_sysroot_flag_sets = [
+    flag_set(
+        actions = ACTION_NAME_GROUPS.all_cc_compile_actions + ACTION_NAME_GROUPS.all_cc_link_actions,
+        flag_groups = [
+            flag_group(
+                expand_if_available = "sysroot",
+                flags = ["--sysroot=%{sysroot}"],
+            ),
+        ],
+        with_features = [with_feature_set(not_features = ["macos_target"])],
+    ),
+    flag_set(
+        actions = ACTION_NAME_GROUPS.all_cc_compile_actions + ACTION_NAME_GROUPS.all_cc_link_actions,
+        flag_groups = [
+            flag_group(
+                flags = ["-isysroot", "%{sysroot}"],
+            ),
+        ],
+        with_features = [with_feature_set(["macos_target"])],
+    ),
+]
+
 clang_feature = feature(
     name = "clang",
     enabled = True,
-    flag_sets = [
+    flag_sets = _sysroot_flag_sets + [
         flag_set(
             actions = ACTION_NAME_GROUPS.all_cc_compile_actions + ACTION_NAME_GROUPS.all_cc_link_actions,
             flag_groups = [
@@ -28,10 +67,6 @@ clang_feature = feature(
                     "-no-canonical-prefixes",
                     "-fcolor-diagnostics",
                 ]),
-                flag_group(
-                    expand_if_available = "sysroot",
-                    flags = ["--sysroot=%{sysroot}"],
-                ),
             ],
         ),
         flag_set(
@@ -66,6 +101,8 @@ clang_feature = feature(
             # Flags specific to compiling C++ sources.
             actions = ACTION_NAME_GROUPS.all_cpp_compile_actions,
             flag_groups = [flag_group(flags = [
+                "-fno-exceptions",
+                "-fno-rtti",
                 "-std=c++20",
             ])],
         ),
@@ -284,8 +321,13 @@ def libcxx_feature(llvm_bindir = None, clang_bindir = None):
                     "-unwindlib=libunwind",
                 ])],
                 with_features = [
-                    # libc++ is only used on non-Windows platforms.
-                    with_feature_set(not_features = ["windows_target"]),
+                    # libc++ is only used on non-Windows platforms, and macOS
+                    # doesn't support a custom unwinding library (or need one)
+                    # even when using libc++.
+                    with_feature_set(not_features = [
+                        "macos_target",
+                        "windows_target",
+                    ]),
                 ],
             ),
             flag_set(

bazel/cc_toolchains/cc_tools_test.py

@@ -13,6 +13,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 import os
 import subprocess
 import sys
+
 from bazel_tools.tools.python.runfiles import runfiles
 
 

bazel/cc_toolchains/clang_cc_toolchain_config.bzl

@@ -6,6 +6,7 @@
 
 load("@rules_cc//cc:defs.bzl", "cc_toolchain")
 load("@rules_cc//cc/common:cc_common.bzl", "cc_common")
+load("@rules_cc//cc/toolchains:cc_toolchain_config_info.bzl", "CcToolchainConfigInfo")
 load(":cc_toolchain_carbon_project_features.bzl", "carbon_project_features")
 load(":cc_toolchain_cpp_features.bzl", "libcxx_feature")
 load(":cc_toolchain_features.bzl", "clang_cc_toolchain_features")
@@ -27,9 +28,14 @@ load(
 def _impl(ctx):
     # Only use a sysroot if one was found when detecting Clang.
     sysroot = None
+    sdk_settings = []
     if sysroot_dir != "None":
         sysroot = sysroot_dir
 
+        # On MacOS, the compiler depends on this file at the root of the SDK,
+        # and it ends up in the `.d` files.
+        sdk_settings = [sysroot_dir + "/SDKSettings.json"]
+
     identifier = "local-{0}-{1}".format(ctx.attr.target_cpu, ctx.attr.target_os)
     return cc_common.create_cc_toolchain_config_info(
         ctx = ctx,
@@ -40,7 +46,7 @@ def _impl(ctx):
             extra_cpp_features = [libcxx_feature(llvm_bindir, clang_bindir)],
         ),
         action_configs = llvm_action_configs(llvm_bindir, clang_bindir),
-        cxx_builtin_include_directories = clang_include_dirs + [
+        cxx_builtin_include_directories = clang_include_dirs + sdk_settings + [
             # Add Clang's resource directory to the end of the builtin include
             # directories to cover the use of sanitizer resource files by the
             # driver.

bazel/cc_toolchains/clang_configuration.bzl

@@ -83,7 +83,11 @@ def _compute_clang_resource_dir(repository_ctx, clang):
     ).stdout
 
     # The only line printed is this path.
-    return output.splitlines()[0]
+    dir_path = repository_ctx.path(output.splitlines()[0])
+
+    # Canonicalize the path to help ensure string matching succeeds
+    # even with clang installs returning a non-canonical path.
+    return str(dir_path.realpath)
 
 def _compute_mac_os_sysroot(repository_ctx):
     """Runs `xcrun` to extract the correct sysroot."""
@@ -148,7 +152,7 @@ def _compute_clang_cpp_include_search_paths(repository_ctx, clang, sysroot):
     if repository_ctx.os.name.lower().startswith("mac os"):
         if not sysroot:
             fail("Must provide a sysroot on macOS!")
-        cmd.append("--sysroot=" + sysroot)
+        cmd += ["-isysroot", sysroot]
 
     # Note that verbose output is on stderr, not stdout!
     output = _run(repository_ctx, cmd).stderr.splitlines()
@@ -184,9 +188,9 @@ def _configure_clang_toolchain_impl(repository_ctx):
     (clang, clang_version, clang_version_for_cache) = _detect_system_clang(
         repository_ctx,
     )
-    if clang_version and clang_version < 19:
+    if clang_version and clang_version < 21:
         fail("Found clang {0}. ".format(clang_version) +
-             "Carbon requires clang >=19. See " +
+             "Carbon requires clang >=21. See " +
              "https://github.com/carbon-language/carbon-lang/blob/trunk/docs/project/contribution_tools.md#old-llvm-versions")
 
     clang_cpp = clang.dirname.get_child("clang++")

bazel/cc_toolchains/defs.bzl

@@ -41,6 +41,6 @@ def cc_env():
     macos_env = {"MallocNanoZone": "0"}
 
     return common_env | select({
-        "//bazel/cc_toolchains:macos_asan": macos_env,
+        Label("//bazel/cc_toolchains:macos_asan"): macos_env,
         "//conditions:default": {},
     })

bazel/check_deps/update_roots.py

@@ -1,4 +1,9 @@
-#!/usr/bin/env python3
+#!/usr/bin/env -S uv run --script
+
+# /// script
+# requires-python = ">=3.12"
+# ///
+
 
 """Update the roots of the Carbon build used for dependency checking.
 

bazel/llvm_project/0006_Add_more_libc_math_excludes.patch

@@ -0,0 +1,37 @@
+Removes additional libc-backed arithmetic builtins added
+by https://github.com/llvm/llvm-project/pull/207092
+and https://github.com/llvm/llvm-project/pull/209984
+---
+--- a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
++++ b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
+@@ -303,6 +303,30 @@
+     "lib/builtins/extendsfdf2.cpp",
+     "lib/builtins/extendsftf2.cpp",
+     "lib/builtins/extendxftf2.cpp",
++    "lib/builtins/fixdfdi.cpp",
++    "lib/builtins/fixdfsi.cpp",
++    "lib/builtins/fixdfti.cpp",
++    "lib/builtins/fixsfdi.cpp",
++    "lib/builtins/fixsfsi.cpp",
++    "lib/builtins/fixsfti.cpp",
++    "lib/builtins/fixunsdfdi.cpp",
++    "lib/builtins/fixunsdfsi.cpp",
++    "lib/builtins/fixunsdfti.cpp",
++    "lib/builtins/fixunssfdi.cpp",
++    "lib/builtins/fixunssfsi.cpp",
++    "lib/builtins/fixunssfti.cpp",
++    "lib/builtins/floatdidf.cpp",
++    "lib/builtins/floatdisf.cpp",
++    "lib/builtins/floatsidf.cpp",
++    "lib/builtins/floatsisf.cpp",
++    "lib/builtins/floattidf.cpp",
++    "lib/builtins/floattisf.cpp",
++    "lib/builtins/floatundidf.cpp",
++    "lib/builtins/floatundisf.cpp",
++    "lib/builtins/floatunsidf.cpp",
++    "lib/builtins/floatunsisf.cpp",
++    "lib/builtins/floatuntidf.cpp",
++    "lib/builtins/floatuntisf.cpp",
+     "lib/builtins/muldf3.cpp",
+     "lib/builtins/mulsf3.cpp",
+     "lib/builtins/multf3.cpp",

bazel/llvm_project/0009_Introduce_starlark_exporting_compiler-rt_build_information.patch

@@ -1,521 +0,0 @@
-Commit ID: d3b82534c2546a892a27856672ed95a7db97dba3
-Change ID: zyxuvzwmzsnorloyuupuurxkppkoplnw
-Author   : Chandler Carruth <chandlerc@gmail.com> (2026-02-16 23:17:06)
-Committer: Chandler Carruth <chandlerc@gmail.com> (2026-03-11 07:54:02)
-
-    Improve compiler-rt build structure and export compilation info
-
-    This first improves the structure of the compiler-rt BUILD.bazel, fixing
-    bugs and exposing more carefully arranged source files.
-
-    It also exposes compilation info for builtins and CRT files for use in
-    compiling these source files.
-
-diff --git a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-index 4ded226174..3b5b8fc787 100644
---- a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-+++ b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-@@ -3,6 +3,7 @@
- # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
- 
- load("@rules_cc//cc:defs.bzl", "cc_library")
-+load("compiler-rt.bzl", "make_filtered_builtins_srcs_groups")
- 
- package(
-     default_visibility = ["//visibility:public"],
-@@ -160,9 +161,15 @@
-     srcs = BUILTINS_CRTEND_SRCS,
- )
- 
-+BUILTINS_EMUTLS_SRCS = ["lib/builtins/emutls.c"]
-+
-+filegroup(
-+    name = "builtins_emutls_srcs",
-+    srcs = BUILTINS_EMUTLS_SRCS,
-+)
-+
- BUILTINS_HOSTED_SRCS = [
-     "lib/builtins/clear_cache.c",
--    "lib/builtins/emutls.c",
-     "lib/builtins/enable_execute_stack.c",
-     "lib/builtins/eprintf.c",
- ]
-@@ -224,11 +231,11 @@
-     ),
- )
- 
--BUILTNS_ATOMICS_SRCS = ["lib/builtins/atomic.c"]
-+BUILTINS_ATOMICS_SRCS = ["lib/builtins/atomic.c"]
- 
- filegroup(
-     name = "builtins_atomics_srcs",
--    srcs = BUILTNS_ATOMICS_SRCS + ["lib/builtins/assembly.h"],
-+    srcs = BUILTINS_ATOMICS_SRCS + ["lib/builtins/assembly.h"],
- )
- 
- BUILTINS_MACOS_ATOMIC_SRCS_PATTERNS = [
-@@ -241,6 +248,28 @@
-     srcs = glob(BUILTINS_MACOS_ATOMIC_SRCS_PATTERNS),
- )
- 
-+# Source files for portable components of the compiler builtins library.
-+filegroup(
-+    name = "builtins_generic_srcs",
-+    srcs = ["lib/builtins/cpu_model/cpu_model.h"] + glob(
-+        [
-+            "lib/builtins/*.c",
-+            "lib/builtins/*.cpp",
-+            "lib/builtins/*.h",
-+            "lib/builtins/*.inc",
-+        ],
-+        allow_empty = True,
-+        exclude = (
-+            BUILTINS_CRTBEGIN_SRCS +
-+            BUILTINS_CRTEND_SRCS +
-+            BUILTINS_TF_EXCLUDES +
-+            BUILTINS_TF_SRCS_PATTERNS +
-+            BUILTINS_ATOMICS_SRCS +
-+            BUILTINS_MACOS_ATOMIC_SRCS_PATTERNS
-+        ),
-+    ),
-+)
-+
- # Apple-platform specific SME source file.
- filegroup(
-     name = "builtins_aarch64_apple_sme_srcs",
-@@ -305,10 +334,13 @@
- 
- # Source files for the AArch64 architecture-specific builtins.
- filegroup(
--    name = "builtins_aarch64_srcs",
-+    name = "builtins_unfiltered_aarch64_srcs",
-     srcs = [
-         "lib/builtins/cpu_model/aarch64.c",
-         "lib/builtins/cpu_model/aarch64.h",
-+        ":builtins_bf16_srcs",
-+        ":builtins_generic_srcs",
-+        ":builtins_tf_srcs",
-     ] + [
-         AARCH64_OUTLINE_ATOMICS_FMT.format(pat, size, model)
-         for (pat, size, model) in AARCH64_OUTLINE_ATOMICS
-@@ -328,10 +360,20 @@
-             "lib/builtins/aarch64/lse.S",
-             # These files are provided by SME-specific file groups above.
-             "lib/builtins/aarch64/*sme*",
-+            # This is only used with MinGW.
-+            "lib/builtins/aarch64/chkstk.S",
-+            # TODO: Remove this once we have a way of accessing `SipHash.h`.
-+            "lib/builtins/aarch64/emupac.cpp",
-         ],
-     ),
- )
- 
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_aarch64_srcs",
-+    srcs = [":builtins_unfiltered_aarch64_srcs"],
-+    textual_name = "builtins_aarch64_textual_srcs",
-+)
-+
- BUILTINS_ARM_VFP_SRCS_PATTERNS = [
-     "lib/builtins/arm/*vfp*.S",
-     "lib/builtins/arm/*vfp*.c",
-@@ -348,9 +390,19 @@
-     ),
- )
- 
-+BUILTINS_ARM_IMPLICIT_IT_SRCS = [
-+    "lib/builtins/arm/mulsf3.S",
-+    "lib/builtins/arm/divsf3.S",
-+]
-+
-+filegroup(
-+    name = "builtins_arm_implicit_it_srcs",
-+    srcs = BUILTINS_ARM_IMPLICIT_IT_SRCS,
-+)
-+
- # Source files for the ARM architecture-specific builtins.
- filegroup(
--    name = "builtins_arm_srcs",
-+    name = "builtins_arm_arch_srcs",
-     srcs = glob(
-         [
-             "lib/builtins/arm/*.S",
-@@ -359,14 +411,52 @@
-             "lib/builtins/arm/*.h",
-         ],
-         allow_empty = True,
--        exclude = BUILTINS_ARM_VFP_SRCS_PATTERNS,
-+        exclude = (BUILTINS_ARM_VFP_SRCS_PATTERNS +
-+                   BUILTINS_ARM_IMPLICIT_IT_SRCS) + [
-+            # This is only used with MinGW.
-+            "lib/builtins/arm/chkstk.S",
-+        ],
-     ),
- )
- 
--# Source files for the PPC architecture-specific builtins.
--filegroup(
--    name = "builtins_ppc_srcs",
--    srcs = glob(
-+filegroup(
-+    name = "builtins_unfiltered_armv7_srcs",
-+    srcs = [
-+        ":builtins_arm_arch_srcs",
-+        ":builtins_arm_vfp_srcs",
-+        ":builtins_bf16_srcs",
-+        ":builtins_generic_srcs",
-+    ],
-+)
-+
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_armv7_srcs",
-+    srcs = [":builtins_unfiltered_armv7_srcs"],
-+    textual_name = "builtins_armv7_textual_srcs",
-+)
-+
-+filegroup(
-+    name = "builtins_unfiltered_aarch32_srcs",
-+    srcs = [
-+        ":builtins_arm_arch_srcs",
-+        ":builtins_arm_vfp_srcs",
-+        ":builtins_bf16_srcs",
-+        ":builtins_generic_srcs",
-+    ],
-+)
-+
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_aarch32_srcs",
-+    srcs = [":builtins_unfiltered_aarch32_srcs"],
-+    textual_name = "builtins_aarch32_textual_srcs",
-+)
-+
-+filegroup(
-+    name = "builtins_unfiltered_ppc64_srcs",
-+    srcs = [
-+        ":builtins_generic_srcs",
-+        ":builtins_tf_srcs",
-+    ] + glob(
-         [
-             "lib/builtins/ppc/*.S",
-             "lib/builtins/ppc/*.c",
-@@ -377,17 +467,64 @@
-     ),
- )
- 
--# Source files for the RISC-V architecture-specific builtins.
--filegroup(
--    name = "builtins_riscv_srcs",
--    srcs = glob(
--        [
--            "lib/builtins/riscv/*.S",
--            "lib/builtins/riscv/*.c",
--            "lib/builtins/riscv/*.cpp",
--        ],
--        allow_empty = True,
--    ),
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_ppc64_srcs",
-+    srcs = [":builtins_unfiltered_ppc64_srcs"],
-+    textual_name = "builtins_ppc64_textual_srcs",
-+)
-+
-+filegroup(
-+    name = "builtins_unfiltered_ppc32_srcs",
-+    srcs = [":builtins_generic_srcs"],
-+)
-+
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_ppc32_srcs",
-+    srcs = [":builtins_unfiltered_ppc32_srcs"],
-+    textual_name = "builtins_ppc32_textual_srcs",
-+)
-+
-+filegroup(
-+    name = "builtins_unfiltered_riscv64_srcs",
-+    srcs = [
-+        ":builtins_generic_srcs",
-+        ":builtins_tf_srcs",
-+    ] + glob(
-+        [
-+            "lib/builtins/riscv/*.S",
-+            "lib/builtins/riscv/*.c",
-+            "lib/builtins/riscv/*.cpp",
-+            "lib/builtins/riscv/*.h",
-+        ],
-+        allow_empty = True,
-+    ),
-+)
-+
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_riscv64_srcs",
-+    srcs = [":builtins_unfiltered_riscv64_srcs"],
-+    textual_name = "builtins_riscv64_textual_srcs",
-+)
-+
-+filegroup(
-+    name = "builtins_unfiltered_riscv32_srcs",
-+    srcs = [
-+        ":builtins_generic_srcs",
-+    ] + glob(
-+        [
-+            "lib/builtins/riscv/*.S",
-+            "lib/builtins/riscv/*.c",
-+            "lib/builtins/riscv/*.cpp",
-+            "lib/builtins/riscv/*.h",
-+        ],
-+        allow_empty = True,
-+    ),
-+)
-+
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_riscv32_srcs",
-+    srcs = [":builtins_unfiltered_riscv32_srcs"],
-+    textual_name = "builtins_riscv32_textual_srcs",
- )
- 
- # Source files for the x86 architecture specific builtins (both 32-bit and
-@@ -402,8 +539,14 @@
- 
- # Source files for the x86-64 architecture specific builtins.
- filegroup(
--    name = "builtins_x86_64_srcs",
--    srcs = glob(
-+    name = "builtins_unfiltered_x86_64_srcs",
-+    srcs = [
-+        ":builtins_bf16_srcs",
-+        ":builtins_generic_srcs",
-+        ":builtins_tf_srcs",
-+        ":builtins_x86_arch_srcs",
-+        ":builtins_x86_fp80_srcs",
-+    ] + glob(
-         [
-             "lib/builtins/x86_64/*.S",
-             "lib/builtins/x86_64/*.c",
-@@ -411,13 +554,29 @@
-             "lib/builtins/x86_64/*.h",
-         ],
-         allow_empty = True,
-+        exclude = [
-+            # This is a Windows-specific routine.
-+            # TODO: We should expose this as a Windows source at some point.
-+            "lib/builtins/x86_64/chkstk.S",
-+        ],
-     ),
- )
- 
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_x86_64_srcs",
-+    srcs = [":builtins_unfiltered_x86_64_srcs"],
-+    textual_name = "builtins_x86_64_textual_srcs",
-+)
-+
- # Source files for the 32-bit-specific x86 architecture specific builtins.
- filegroup(
--    name = "builtins_i386_srcs",
--    srcs = glob(
-+    name = "builtins_unfiltered_i386_srcs",
-+    srcs = [
-+        ":builtins_bf16_srcs",
-+        ":builtins_generic_srcs",
-+        ":builtins_x86_arch_srcs",
-+        ":builtins_x86_fp80_srcs",
-+    ] + glob(
-         [
-             "lib/builtins/i386/*.S",
-             "lib/builtins/i386/*.c",
-@@ -429,28 +588,16 @@
-             # This file is used for both i386 and x86_64 and so included in the
-             # broader x86 sources.
-             "lib/builtins/i386/fp_mode.c",
-+            # These are Windows-specific routines.
-+            # TODO: We should expose these as Windows source at some point.
-+            "lib/builtins/i386/chkstk.S",
-+            "lib/builtins/i386/chkstk2.S",
-         ],
-     ),
- )
- 
--# Source files for portable components of the compiler builtins library.
--filegroup(
--    name = "builtins_generic_srcs",
--    srcs = ["lib/builtins/cpu_model/cpu_model.h"] + glob(
--        [
--            "lib/builtins/*.c",
--            "lib/builtins/*.cpp",
--            "lib/builtins/*.h",
--            "lib/builtins/*.inc",
--        ],
--        allow_empty = True,
--        exclude = (
--            BUILTINS_CRTBEGIN_SRCS +
--            BUILTINS_CRTEND_SRCS +
--            BUILTINS_TF_EXCLUDES +
--            BUILTINS_TF_SRCS_PATTERNS +
--            BUILTNS_ATOMICS_SRCS +
--            BUILTINS_MACOS_ATOMIC_SRCS_PATTERNS
--        ),
--    ),
-+make_filtered_builtins_srcs_groups(
-+    name = "builtins_i386_srcs",
-+    srcs = [":builtins_unfiltered_i386_srcs"],
-+    textual_name = "builtins_i386_textual_srcs",
- )
-diff --git a/utils/bazel/llvm-project-overlay/compiler-rt/compiler-rt.bzl b/utils/bazel/llvm-project-overlay/compiler-rt/compiler-rt.bzl
-new file mode 100644
-index 0000000000..e33ceb6a89
---- /dev/null
-+++ b/utils/bazel/llvm-project-overlay/compiler-rt/compiler-rt.bzl
-@@ -0,0 +1,153 @@
-+# This file is licensed under the Apache License v2.0 with LLVM Exceptions.
-+# See https://llvm.org/LICENSE.txt for license information.
-+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-+
-+"""Starlark for building parts of compiler-rt.
-+
-+Variables provide baseline information for how to build various parts of
-+compiler-rt. These can be used to generate non-Bazel builds of the library.
-+
-+Rules and macros support building the relevant filegroups of source files.
-+
-+TODO: Add macros that provide a convenient way to construct a Bazel target for
-+the Clang resource directory with builtins and crt files.
-+"""
-+
-+_common_copts = [
-+    "-O3",
-+    "-fPIC",
-+    "-ffreestanding",
-+    "-std=c11",
-+]
-+
-+crt_copts = _common_copts + [
-+    "-DCRT_HAS_INITFINI_ARRAY",
-+    "-DEH_USE_FRAME_REGISTRY",
-+    "-fno-lto",
-+]
-+
-+builtins_copts = _common_copts + [
-+    "-fno-builtin",
-+    "-fomit-frame-pointer",
-+    "-fvisibility=hidden",
-+    "-Wno-missing-prototypes",
-+    "-Wno-unused-parameter",
-+]
-+
-+def _get_rel_path(path_str):
-+    rel_path = path_str.rpartition("/lib/builtins/")[2]
-+    if rel_path == path_str:
-+        fail("Expected '/lib/builtins/' in path " + path_str)
-+    return rel_path
-+
-+def _filtered_builtins_srcs_impl(ctx):
-+    """Implementation of filter_builtins_srcs rule."""
-+
-+    # Build a map from generic file basename to list of overriding files.
-+    overrides = {}
-+    for f in ctx.files.srcs:
-+        rel_path = _get_rel_path(f.short_path)
-+        if "/" in rel_path:
-+            base_file = rel_path.rpartition("/")[2]
-+            if base_file.endswith(".S"):
-+                base_file = base_file.removesuffix(".S") + ".c"
-+            overrides[base_file] = True
-+
-+    filtered_files = []
-+    for f in ctx.files.srcs:
-+        rel_path = _get_rel_path(f.short_path)
-+        if "/" not in rel_path:
-+            # This is a generic file. Check if it's overridden.
-+            if rel_path not in overrides:
-+                filtered_files.append(f)
-+        else:
-+            # This is an arch-specific file, include it.
-+            filtered_files.append(f)
-+
-+    # Remove any textual sources from this list.
-+    filtered_files = [
-+        f
-+        for f in filtered_files
-+        if f.extension not in ["inc", "def"]
-+    ]
-+
-+    return [DefaultInfo(files = depset(filtered_files))]
-+
-+filtered_builtins_srcs = rule(
-+    implementation = _filtered_builtins_srcs_impl,
-+    attrs = {
-+        "srcs": attr.label_list(
-+            mandatory = True,
-+            allow_files = True,
-+            doc = "Input files.",
-+        ),
-+    },
-+    doc = """Build a filtered filegroup of non-textual srcs for builtins.
-+
-+    Accepts a filegroup whose files are in lib/builtins/, and produces a target
-+    behaving like a filegroup containing filtered files.
-+
-+    This removes any textual source files (`.inc` or `.def`) from the input.
-+
-+    It also replaces generic srcs that are overridden by architecture-specific
-+    sources. For example, given a list of sources from filegroup of the form:
-+
-+    - `.../lib/builtins/file_0.c`
-+    - `.../lib/builtins/file_1.c`
-+    - `.../lib/builtins/file_2.c`
-+    - `.../lib/builtins/arch/file_0.c`
-+    - `.../lib/builtins/arch/file_1.S`
-+
-+    It removes any source-file at the top level of lib/builtins/ (e.g.
-+    lib/builtins/file_0.c) that has a corresponding source-file in an arch
-+    directory (e.g. lib/builtins/arch/file_0.c or lib/builtins/arch/file_1.S),
-+    producing a list like:
-+
-+    - `.../lib/builtins/file_2.c`
-+    - `.../lib/builtins/arch/file_0.c`
-+    - `.../lib/builtins/arch/file_1.S`
-+
-+    This allows a target architecture to simply add a specialized file to the
-+    list of sources with the architecture prefix and have the specialized
-+    version override the generic version.
-+    """,
-+)
-+
-+def _filtered_builtins_textual_srcs_impl(ctx):
-+    """Implementation of filter_builtins_textual_srcs rule."""
-+
-+    filtered_files = [
-+        f
-+        for f in ctx.files.srcs
-+        if f.extension in ["inc", "def"]
-+    ]
-+
-+    return [DefaultInfo(files = depset(filtered_files))]
-+
-+filtered_builtins_textual_srcs = rule(
-+    implementation = _filtered_builtins_textual_srcs_impl,
-+    attrs = {
-+        "srcs": attr.label_list(
-+            mandatory = True,
-+            allow_files = True,
-+            doc = "Input files.",
-+        ),
-+    },
-+    doc = """Build a filegroup of the textual srcs for builtins.
-+
-+    Textual sources are those that can't be compiled directly and aren't
-+    recognized as header files by Bazel. The extensions recognized here are
-+    `.inc` and `.def`.
-+    """,
-+)
-+
-+def make_filtered_builtins_srcs_groups(name, textual_name, srcs):
-+    """Macro to expand both the non-textual and textual filtered srcs groups."""
-+    filtered_builtins_srcs(
-+        name = name,
-+        srcs = srcs,
-+    )
-+    filtered_builtins_textual_srcs(
-+        name = textual_name,
-+        srcs = srcs,
-+    )

bazel/llvm_project/0011_Temporarily_remove_reference_to_hermetic_toolchain.patch

@@ -0,0 +1,24 @@
+Temporarily undo
+https://github.com/llvm/llvm-project/pull/207295
+Which introduces a dependency on the hermetic llvm
+toolchain. A fix-forward is in progress, at which
+point we can remove this patch.
+---
+--- a/utils/bazel/llvm-project-overlay/llvm/config.bzl
++++ b/utils/bazel/llvm-project-overlay/llvm/config.bzl
+@@ -72,7 +72,6 @@
+ backtrace_defines = select({
+     "@platforms//os:emscripten": [],
+     "@platforms//os:windows": [],
+-    "@llvm//platforms/config:musl": [],
+     "//conditions:default": [
+         "HAVE_BACKTRACE=1",
+         "BACKTRACE_HEADER=<execinfo.h>",
+@@ -80,7 +79,6 @@
+ })
+ 
+ mallinfo_defines = select({
+-    "@llvm//platforms/config:gnu": ["HAVE_MALLINFO=1"],
+     "//conditions:default": [],
+ })
+ 

bazel/manifest/defs.bzl

@@ -13,8 +13,16 @@ def _get_files(ctx):
         # Files may or may not be prefixed with the bin directory, and then
         # may or may not be prefixed with the package directory. Strip both.
         bin_dir = ctx.bin_dir.path + "/"
+        workspace_root = (
+            ctx.label.workspace_root + "/" if ctx.label.workspace_root else ""
+        )
         package_dir = ctx.label.package + "/"
-        files_stripped = [f.removeprefix(bin_dir).removeprefix(package_dir) for f in files]
+        files_stripped = [
+            f.removeprefix(bin_dir)
+                .removeprefix(workspace_root)
+                .removeprefix(package_dir)
+            for f in files
+        ]
     else:
         files_stripped = files
 

bazel/version/gen_tmpl.py

@@ -1,4 +1,13 @@
-#!/usr/bin/env python3
+#!/usr/bin/env -S uv run --script
+
+# /// script
+# requires-python = ">=3.12"
+# ///
+
+# NOTE: The `uv` shebang and inline metadata above are only used for direct
+# execution of this script outside of Bazel. When executed by Bazel (e.g., as a
+# tool in a rule or as a test), Bazel uses its own hermetic Python toolchain
+# and ignores this metadata.
 
 """Generate a file from a template, substituting the provided key/value pairs.
 

bazel/version/rules.bzl

@@ -135,7 +135,7 @@ def expand_version_build_info(name, **kwargs):
     expand_version_build_info_internal(
         name = name,
         internal_stamp_flag_detect = False if kwargs.get("stamp") == 0 else select({
-            "//bazel/version:internal_stamp_flag_detect": True,
+            Label("//bazel/version:internal_stamp_flag_detect"): True,
             "//conditions:default": False,
         }),
         **kwargs

common/BUILD

@@ -287,9 +287,9 @@ sh_test(
     srcs = [":filesystem_benchmark"],
     args = [
         "--benchmark_dry_run",
-        # Restrict the sizes to 4-digit ones or smaller to keep test times low.
+        # Restrict the sizes to 2-digit ones or smaller to keep test times low.
         # The `$$` is repeated for Bazel escaping of `$`.
-        "--benchmark_filter=^[^/]+(/[0-9]{1,4}(/[0-9]+)?)?/real_time$$",
+        "--benchmark_filter=^[^/]+(/[0-9]{1,2}(/[0-9]+)?)?/real_time$$",
     ],
 )
 
@@ -342,12 +342,22 @@ cc_library(
     ],
 )
 
+cc_library(
+    name = "hashing_llvm",
+    hdrs = ["hashing_llvm.h"],
+    deps = [
+        ":hashing",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
 cc_test(
     name = "hashing_test",
     size = "small",
     srcs = ["hashing_test.cpp"],
     deps = [
         ":hashing",
+        ":hashing_llvm",
         ":raw_string_ostream",
         "//testing/base:gtest_main",
         "@googletest//:gtest",
@@ -384,6 +394,7 @@ cc_test(
     size = "small",
     srcs = ["hashtable_key_context_test.cpp"],
     deps = [
+        ":hashing_llvm",
         ":hashtable_key_context",
         "//testing/base:gtest_main",
         "@googletest//:gtest",
@@ -460,6 +471,7 @@ cc_test(
         ":raw_hashtable_test_helpers",
         "//testing/base:gtest_main",
         "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
     ],
 )
 
@@ -489,7 +501,7 @@ sh_test(
     args = [
         "--benchmark_dry_run",
         # The `$$` is repeated for Bazel escaping of `$`.
-        "--benchmark_filter=^[^/]*/[1-9][0-9]{0,3}(/[0-9]+)?$$",
+        "--benchmark_filter=^[^/]*/[1-9][0-9]{0,2}(/[0-9]+)?$$",
     ],
 )
 
@@ -506,6 +518,19 @@ cc_library(
     ],
 )
 
+cc_test(
+    name = "ostream_test",
+    size = "small",
+    srcs = ["ostream_test.cpp"],
+    deps = [
+        ":ostream",
+        ":raw_string_ostream",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
 cc_library(
     name = "pretty_stack_trace_function",
     hdrs = ["pretty_stack_trace_function.h"],
@@ -635,6 +660,7 @@ cc_test(
         ":set",
         "//testing/base:gtest_main",
         "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
     ],
 )
 
@@ -662,7 +688,7 @@ sh_test(
     args = [
         "--benchmark_dry_run",
         # The `$$` is repeated for Bazel escaping of `$`.
-        "--benchmark_filter=^[^/]*/[1-9][0-9]{0,3}(/[0-9]+)?$$",
+        "--benchmark_filter=^[^/]*/[1-9][0-9]{0,2}(/[0-9]+)?$$",
     ],
 )
 

common/check_internal.cpp

@@ -8,18 +8,61 @@
 #include <string>
 
 #include "common/ostream.h"
+#include "llvm/Support/FormatCommon.h"
+#include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/Signals.h"
+#include "llvm/Support/raw_ostream.h"
 
 namespace Carbon::Internal {
 
-auto CheckFailImpl(const char* kind, const char* file, int line,
-                   const char* condition_str, llvm::StringRef extra_message)
+namespace {
+// Renders `fmt` over the externally-built, type-erased `adapters` into `out`,
+// with the same semantics as `llvm::formatv` (including runtime format-string
+// validation).
+//
+// TODO: We should add a type-erased helper to upstream LLVM instead of rolling
+// our own type-erased version of `format` here.
+auto FormatvInto(
+    llvm::raw_ostream& out, llvm::StringRef format_str,
+    llvm::ArrayRef<llvm::support::detail::FormatFunctorRef> adapters) -> void {
+  for (const llvm::ReplacementItem& replacement :
+       llvm::formatv_object_base::parseFormatString(format_str, adapters.size(),
+                                                    /*Validate=*/true)) {
+    if (replacement.Type == llvm::ReplacementType::Literal ||
+        replacement.Index >= adapters.size()) {
+      out << replacement.Spec;
+      continue;
+    }
+    llvm::FmtAlign(adapters[replacement.Index], replacement.Where,
+                   replacement.Width, replacement.Pad)
+        .format(out, replacement.Options);
+  }
+}
+}  // namespace
+
+auto CheckFailImpl(
+    const char* kind, const char* file, int line, const char* condition_str,
+    const char* extra_format,
+    llvm::ArrayRef<llvm::support::detail::FormatFunctorRef> extra_adapters)
     -> void {
-  // Render the final check string here.
-  std::string message = llvm::formatv(
-      "{0} failure at {1}:{2}{3}{4}{5}{6}\n", kind, file, line,
-      llvm::StringRef(condition_str).empty() ? "" : ": ", condition_str,
-      extra_message.empty() ? "" : ": ", extra_message);
+  // Render the final check string directly into one stream. The extra message
+  // is rendered in place from its format string and type-erased adapters, so
+  // we never materialize a separate string just for it.
+  //
+  // `llvm::raw_string_ostream` (rather than `common/raw_string_ostream.h`) is
+  // used to avoid a dependency cycle: `RawStringOstream` itself uses
+  // `CARBON_CHECK`. It is unbuffered, so `message` is populated directly.
+  std::string message;
+  llvm::raw_string_ostream message_stream(message);
+  message_stream << kind << " failure at " << file << ":" << line;
+  if (*condition_str != '\0') {
+    message_stream << ": " << condition_str;
+  }
+  if (*extra_format != '\0') {
+    message_stream << ": ";
+    FormatvInto(message_stream, extra_format, extra_adapters);
+  }
+  message_stream << "\n";
 
   // This macro is defined by `--config=non-fatal-checks`.
 #ifdef CARBON_NON_FATAL_CHECKS

common/check_internal.h

@@ -31,25 +31,29 @@ CheckCondition(bool condition)
 // Implements the check failure message printing.
 //
 // This is out-of-line and will arrange to stop the program, print any debugging
-// information and this string. In `!NDEBUG` mode (`dbg` and `fastbuild`), check
-// failures can be made non-fatal by a build flag, so this is not `[[noreturn]]`
-// in that case.
+// information and the failure message. In `!NDEBUG` mode (`dbg` and
+// `fastbuild`), check failures can be made non-fatal by a build flag, so this
+// is not `[[noreturn]]` in that case.
 //
 // This API uses `const char*` C string arguments rather than `llvm::StringRef`
 // because we know that these are available as C strings and passing them that
 // way lets the code size of calling it be smaller: it only needs to materialize
 // a single pointer argument for each. The runtime cost of re-computing the size
-// should be minimal. The extra message however might not be compile-time
-// guaranteed to be a C string so we use a normal `StringRef` there.
+// should be minimal.
+//
+// The user can provide an extra format string along with an array of
+// type-erased format adapters. This will be rendered into the final message.
 #ifdef NDEBUG
 [[noreturn]]
 #endif
-auto CheckFailImpl(const char* kind, const char* file, int line,
-                   const char* condition_str, llvm::StringRef extra_message)
+auto CheckFailImpl(
+    const char* kind, const char* file, int line, const char* condition_str,
+    const char* extra_format,
+    llvm::ArrayRef<llvm::support::detail::FormatFunctorRef> extra_adapters)
     -> void;
 
-// Allow converting format values; the default behaviour is to just pass them
-// through.
+// Allow custom conversion of format values; the default behaviour is to just
+// pass them through.
 template <typename T>
 auto ConvertFormatValue(T&& t) -> T&& {
   return std::forward<T>(t);
@@ -70,36 +74,53 @@ auto ConvertFormatValue(T&& t) -> auto {
   }
 }
 
+// Builds one type-erased format functor per value -- forwarding each value
+// through the conversion machinery. References to each of these functors are
+// then collected into an init list that can be accessed with an `ArrayRef`. All
+// of this is then passed to the out-of-line rendering function `CheckFailImpl`.
+//
+// This is templated only on the value types, not on the per-check-site
+// metadata (file, line, etc., which are passed as ordinary arguments), so the
+// adapter-building is instantiated once per distinct sequence of value types in
+// the TU.
+template <typename... Ts>
+#ifdef NDEBUG
+[[noreturn]]
+#endif
+auto CheckFailFormat(const char* kind, const char* file, int line,
+                     const char* condition_str, const char* extra_format,
+                     Ts&&... values) -> void {
+  CheckFailImpl(kind, file, line, condition_str, extra_format,
+                {llvm::support::detail::FormatFunctor(
+                    ConvertFormatValue(std::forward<Ts>(values)))...});
+}
+
 // Prints a check failure, including rendering any user-provided message using
 // a format string.
 //
-// Most of the parameters are passed as compile-time template strings to avoid
-// runtime cost of parameter setup in optimized builds. Each of these are passed
-// along to the underlying implementation to include in the final printed
-// message.
-//
-// Any user-provided format string and values are directly passed to
-// `llvm::formatv` which handles all of the formatting of output.
+// The check-site metadata is passed as compile-time template strings to avoid
+// runtime cost of parameter setup in optimized builds. This function is
+// instantiated once per check site (its template arguments are unique to the
+// site), so it is kept trivial: it just lowers those template strings to
+// ordinary arguments and forwards everything to `CheckFailFormat`, where the
+// adapter-building is shared across sites with the same value types.
 template <TemplateString Kind, TemplateString File, int Line,
           TemplateString ConditionStr, TemplateString FormatStr, typename... Ts>
 #ifdef NDEBUG
 [[noreturn]]
 #endif
 [[gnu::cold, clang::noinline]] auto CheckFail(Ts&&... values) -> void {
-  if constexpr (llvm::StringRef(FormatStr).empty()) {
-    // Skip the format string rendering if empty. Note that we don't skip it
-    // even if there are no values as we want to have consistent handling of
-    // `{}`s in the format string. This case is about when there is no message
-    // at all, just the condition.
-    CheckFailImpl(Kind.c_str(), File.c_str(), Line, ConditionStr.c_str(), "");
-  } else {
-    CheckFailImpl(Kind.c_str(), File.c_str(), Line, ConditionStr.c_str(),
-                  llvm::formatv(FormatStr.c_str(),
-                                ConvertFormatValue(std::forward<Ts>(values))...)
-                      .str());
-  }
+  CheckFailFormat(Kind.c_str(), File.c_str(), Line, ConditionStr.c_str(),
+                  FormatStr.c_str(), std::forward<Ts>(values)...);
 }
 
+// Type-checks the arguments of a `DCHECK` in optimized builds, where the check
+// itself is dead code, without instantiating any formatting machinery for them
+// and without provoking unused-variable warnings. It is only ever named from
+// dead code, so it is never actually called.
+template <typename... Ts>
+auto IgnoreDeadCheckArgs(Ts&&... /*values*/) -> void {}
+
 }  // namespace Carbon::Internal
 
 // Evaluates the condition of a CHECK as a boolean value.
@@ -148,21 +169,23 @@ template <TemplateString Kind, TemplateString File, int Line,
    CARBON_INTERNAL_FATAL_NORETURN_SUFFIX())
 
 #ifdef NDEBUG
-// For `DCHECK` in optimized builds we have a dead check that we want to
-// potentially "use" arguments, but otherwise have the minimal overhead. We
-// avoid forming interesting format strings here so that we don't have to
-// repeatedly instantiate the `Check` function above. This format string would
-// be an error if actually used.
+// For `DCHECK` in optimized builds the check is dead code, but we still want to
+// type-check its arguments so they can't bitrot. We route them through
+// `IgnoreDeadCheckArgs`, which uses the arguments (avoiding unused-variable
+// warnings) but builds no format adapters, so the dead check doesn't pull in
+// the formatting machinery -- in particular not the per-value-type adapters
+// that the live `CheckFail` path would. The format string is a literal, so it
+// needs no type-checking and is dropped.
 #define CARBON_INTERNAL_DEAD_DCHECK(condition, ...) \
   CARBON_INTERNAL_DEAD_DCHECK_IMPL##__VA_OPT__(_FORMAT)(__VA_ARGS__)
 
 #define CARBON_INTERNAL_DEAD_DCHECK_IMPL() \
-  Carbon::Internal::CheckFail<"", "", 0, "", "">()
+  Carbon::Internal::IgnoreDeadCheckArgs()
 
 #define CARBON_INTERNAL_DEAD_DCHECK_IMPL_FORMAT(format_str, ...) \
-  Carbon::Internal::CheckFail<"", "", 0, "", "">(__VA_ARGS__)
+  Carbon::Internal::IgnoreDeadCheckArgs(__VA_ARGS__)
 
-// The CheckFail function itself is noreturn in NDEBUG.
+// The `CheckFail` function itself is noreturn in NDEBUG.
 #define CARBON_INTERNAL_FATAL_NORETURN_SUFFIX() void()
 #else
 #define CARBON_INTERNAL_FATAL_NORETURN_SUFFIX() std::abort()

common/filesystem.cpp

@@ -103,15 +103,8 @@ auto Internal::FileRefBase::ReadFileToString()
 auto Internal::FileRefBase::WriteFileFromString(llvm::StringRef str)
     -> ErrorOr<Success, FdError> {
   CARBON_RETURN_IF_ERROR(SeekFromBeginning(0));
-  auto bytes = llvm::ArrayRef<std::byte>(
-      reinterpret_cast<const std::byte*>(str.data()), str.size());
-  while (!bytes.empty()) {
-    auto write_result = WriteFromBuffer(bytes);
-    if (!write_result.ok()) {
-      return std::move(write_result).error();
-    }
-    bytes = *write_result;
-  }
+  CARBON_RETURN_IF_ERROR(WriteCompleteBuffer(llvm::ArrayRef<std::byte>(
+      reinterpret_cast<const std::byte*>(str.data()), str.size())));
   CARBON_RETURN_IF_ERROR(Truncate(str.size()));
   return Success();
 }

common/filesystem.h

@@ -219,6 +219,24 @@ namespace Internal {
 class FileRefBase;
 }  // namespace Internal
 
+// Convenience type defs for the three access combinations.
+using ReadFileRef = FileRef<OpenAccess::ReadOnly>;
+using WriteFileRef = FileRef<OpenAccess::WriteOnly>;
+using ReadWriteFileRef = FileRef<OpenAccess::ReadWrite>;
+
+// Returns constant references to the standard streams the process is started
+// with.
+//
+// The returned references are non-owning: the process shares these descriptors
+// with whatever started it, closing them is never correct, and unrelated code
+// throughout the process may be reading or writing the same descriptor.
+//
+// Their descriptor numbers are fixed by the platform rather than discovered at
+// runtime, so these are constant expressions.
+consteval auto Stdin() -> ReadFileRef;
+consteval auto Stdout() -> WriteFileRef;
+consteval auto Stderr() -> WriteFileRef;
+
 // Returns a constant `Dir` object that models the open current working
 // directory.
 //
@@ -348,7 +366,13 @@ class Internal::FileRefBase {
   FileRefBase() = default;
 
   // Returns true if this refers to a valid open file, and false otherwise.
-  auto is_valid() const -> bool { return fd_ != -1; }
+  constexpr auto is_valid() const -> bool { return fd_ != -1; }
+
+  // Non-portable API only available on Unix-like systems. Returns the
+  // underlying file descriptor, for the platform calls this type doesn't wrap,
+  // such as `isatty` and `ioctl`. The descriptor remains owned by whatever owns
+  // this file.
+  constexpr auto unix_fd() const -> int { return fd_; }
 
   // Reads the file status.
   //
@@ -405,6 +429,24 @@ class Internal::FileRefBase {
   auto WriteFromBuffer(llvm::ArrayRef<std::byte> buffer)
       -> ErrorOr<llvm::ArrayRef<std::byte>, FdError>;
 
+  // Writes the complete contents of the provided buffer.
+  //
+  // Unlike `WriteFromBuffer`, this doesn't return until every byte has been
+  // written or an error occurs. It repeats `WriteFromBuffer` over whatever is
+  // left, so each write is issued for as much of the buffer as remains and the
+  // whole is written in as few writes as the file allows. Anything else writing
+  // to the same file can only interleave between those writes, which leaves no
+  // room to interleave at all when the file accepts the buffer in one write.
+  //
+  // On an error, an unspecified prefix of the buffer has already been written
+  // and can't be un-written. How much isn't reported; a caller that needs to
+  // know should drive `WriteFromBuffer` itself.
+  //
+  // This method retries `EINTR` on Unix-like systems and returns other errors
+  // to the caller.
+  auto WriteCompleteBuffer(llvm::ArrayRef<std::byte> buffer)
+      -> ErrorOr<Success, FdError>;
+
   // Returns an LLVM `raw_fd_ostream` that writes to this file.
   //
   // Note that this doesn't expose any write errors here, those will surface
@@ -458,7 +500,7 @@ class Internal::FileRefBase {
                Duration poll_interval = {}) -> ErrorOr<FileLock, FdError>;
 
  protected:
-  explicit FileRefBase(int fd) : fd_(fd) {}
+  explicit constexpr FileRefBase(int fd) : fd_(fd) {}
 
   // Note: this should only be used or made part of the public API by subclasses
   // that provide *ownership* of the open file. It is implemented here to
@@ -536,6 +578,9 @@ class FileRef : public Internal::FileRefBase {
   auto WriteFromBuffer(llvm::ArrayRef<std::byte> buffer)
       -> ErrorOr<llvm::ArrayRef<std::byte>, FdError>
     requires Writeable;
+  auto WriteCompleteBuffer(llvm::ArrayRef<std::byte> buffer)
+      -> ErrorOr<Success, FdError>
+    requires Writeable;
   auto WriteStream() -> llvm::raw_fd_ostream
     requires Writeable;
   auto ReadFileToString() -> ErrorOr<std::string, FdError>
@@ -546,16 +591,14 @@ class FileRef : public Internal::FileRefBase {
  protected:
   friend File<A>;
   friend DirRef;
+  friend consteval auto Stdin() -> ReadFileRef;
+  friend consteval auto Stdout() -> WriteFileRef;
+  friend consteval auto Stderr() -> WriteFileRef;
 
   // Other constructors from the base are also available, but remain protected.
   using FileRefBase::FileRefBase;
 };
 
-// Convenience type defs for the three access combinations.
-using ReadFileRef = FileRef<OpenAccess::ReadOnly>;
-using WriteFileRef = FileRef<OpenAccess::WriteOnly>;
-using ReadWriteFileRef = FileRef<OpenAccess::ReadWrite>;
-
 // An owning handle to an open file.
 //
 // This extends the `FileRef` API to provide ownership of the file handle. Most
@@ -1320,6 +1363,10 @@ inline auto DurationToTimespec(Duration d) -> timespec {
 
 }  // namespace Internal
 
+consteval auto Stdin() -> ReadFileRef { return ReadFileRef(STDIN_FILENO); }
+consteval auto Stdout() -> WriteFileRef { return WriteFileRef(STDOUT_FILENO); }
+consteval auto Stderr() -> WriteFileRef { return WriteFileRef(STDERR_FILENO); }
+
 consteval auto Cwd() -> Dir { return Dir(AT_FDCWD); }
 
 inline auto FileLock::Destroy() -> void {
@@ -1431,6 +1478,14 @@ inline auto Internal::FileRefBase::WriteFromBuffer(
   }
 }
 
+inline auto Internal::FileRefBase::WriteCompleteBuffer(
+    llvm::ArrayRef<std::byte> buffer) -> ErrorOr<Success, FdError> {
+  while (!buffer.empty()) {
+    CARBON_ASSIGN_OR_RETURN(buffer, WriteFromBuffer(buffer));
+  }
+  return Success();
+}
+
 inline auto Internal::FileRefBase::WriteStream() -> llvm::raw_fd_ostream {
   return llvm::raw_fd_ostream(fd_, /*shouldClose=*/false);
 }
@@ -1495,6 +1550,14 @@ auto FileRef<A>::WriteFromBuffer(llvm::ArrayRef<std::byte> buffer)
   return FileRefBase::WriteFromBuffer(buffer);
 }
 
+template <OpenAccess A>
+auto FileRef<A>::WriteCompleteBuffer(llvm::ArrayRef<std::byte> buffer)
+    -> ErrorOr<Success, FdError>
+  requires Writeable
+{
+  return FileRefBase::WriteCompleteBuffer(buffer);
+}
+
 template <OpenAccess A>
 auto FileRef<A>::WriteStream() -> llvm::raw_fd_ostream
   requires Writeable

common/filesystem_benchmark.cpp

@@ -444,9 +444,9 @@ auto BM_CreateDirectories(benchmark::State& state) -> void {
   CARBON_CHECK(existing_depth <= depth);
   CARBON_CHECK(depth > 0);
 
-  // Use a batch size of 10 to get avoid completely swamping the measurements
+  // Use a batch size of 5 to get avoid completely swamping the measurements
   // with overhead from creating existing directories and cleaning up.
-  constexpr int BatchSize = 10;
+  constexpr int BatchSize = 5;
 
   // Pre-build both the paths and the existing paths. Note that we use
   // relatively short paths here, which if anything makes the benefits of the

common/filesystem_test.cpp

@@ -411,6 +411,58 @@ TEST_F(FilesystemTest, WriteStream) {
   EXPECT_THAT(dir_.ReadFileToString("test"), IsSuccess(Eq(content_str)));
 }
 
+TEST_F(FilesystemTest, WriteCompleteBuffer) {
+  std::string content_str = "0123456789";
+  auto bytes = llvm::ArrayRef<std::byte>(
+      reinterpret_cast<const std::byte*>(content_str.data()),
+      content_str.size());
+
+  auto write = dir_.OpenWriteOnly("test", CreationOptions::CreateNew);
+  ASSERT_THAT(write, IsSuccess(_));
+  EXPECT_THAT(write->WriteCompleteBuffer(bytes), IsSuccess(_));
+  // Writing appends rather than replacing, unlike `WriteFileFromString`.
+  EXPECT_THAT(write->WriteCompleteBuffer(bytes), IsSuccess(_));
+  // An empty buffer is a no-op rather than an error.
+  EXPECT_THAT(write->WriteCompleteBuffer(llvm::ArrayRef<std::byte>()),
+              IsSuccess(_));
+  (*std::move(write)).Close().Check();
+
+  EXPECT_THAT(dir_.ReadFileToString("test"),
+              IsSuccess(Eq(content_str + content_str)));
+}
+
+TEST_F(FilesystemTest, StandardStreams) {
+  // The standard streams name descriptors the process already has, so these
+  // are constants and never open or close anything.
+  static_assert(Stdin().unix_fd() == STDIN_FILENO);
+  static_assert(Stdout().unix_fd() == STDOUT_FILENO);
+  static_assert(Stderr().unix_fd() == STDERR_FILENO);
+  EXPECT_TRUE(Stderr().is_valid());
+
+  // Writing through one reaches the descriptor. Tests run with stdout captured,
+  // so this uses a pipe put in its place for the duration.
+  int fds[2];
+  ASSERT_EQ(pipe(fds), 0);
+  int saved = dup(STDOUT_FILENO);
+  ASSERT_GE(saved, 0);
+  ASSERT_GE(dup2(fds[1], STDOUT_FILENO), 0);
+
+  llvm::StringRef message = "through stdout";
+  auto result = Stdout().WriteCompleteBuffer(llvm::ArrayRef<std::byte>(
+      reinterpret_cast<const std::byte*>(message.data()), message.size()));
+
+  ASSERT_GE(dup2(saved, STDOUT_FILENO), 0);
+  ASSERT_EQ(close(saved), 0);
+  ASSERT_EQ(close(fds[1]), 0);
+  EXPECT_THAT(result, IsSuccess(_));
+
+  char buffer[64];
+  ssize_t n = read(fds[0], buffer, sizeof(buffer));
+  ASSERT_EQ(close(fds[0]), 0);
+  ASSERT_GT(n, 0);
+  EXPECT_EQ(llvm::StringRef(buffer, n), message);
+}
+
 TEST_F(FilesystemTest, Rename) {
   // Rename a file within a directory.
   ASSERT_THAT(dir_.WriteFileFromString("file1", "content1"), IsSuccess(_));

common/find.h

@@ -15,11 +15,11 @@ namespace Carbon {
 namespace Internal {
 
 template <typename Range>
-using RangePointerType = typename std::iterator_traits<decltype(std::begin(
-    std::declval<Range>()))>::pointer;
+using RangePointerType =
+    std::iterator_traits<decltype(std::begin(std::declval<Range>()))>::pointer;
 
 template <typename Range>
-using RangeValueType = typename std::iterator_traits<decltype(std::begin(
+using RangeValueType = std::iterator_traits<decltype(std::begin(
     std::declval<Range>()))>::value_type;
 
 template <typename Range, typename Pred>

common/hashing.cpp

@@ -6,8 +6,14 @@
 
 #include <cstddef>
 
+#include "llvm/Support/FormatVariadic.h"
+
 namespace Carbon {
 
+auto HashCode::Print(llvm::raw_ostream& out) const -> void {
+  out << llvm::formatv("{0:x16}", value_);
+}
+
 auto Hasher::HashSizedBytesLarge(llvm::ArrayRef<std::byte> bytes) -> void {
   const std::byte* data_ptr = bytes.data();
   const ssize_t size = bytes.size();

common/hashing.h

@@ -13,12 +13,9 @@
 
 #include "common/check.h"
 #include "common/ostream.h"
-#include "llvm/ADT/APFloat.h"
-#include "llvm/ADT/APInt.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
-#include "llvm/Support/FormatVariadic.h"
 
 #ifdef __ARM_ACLE
 #include <arm_acle.h>
@@ -72,9 +69,7 @@ class HashCode : public Printable<HashCode> {
   // other recursive hashing where that is needed or more efficient.
   explicit operator uint64_t() const { return value_; }
 
-  auto Print(llvm::raw_ostream& out) const -> void {
-    out << llvm::formatv("{0:x16}", value_);
-  }
+  auto Print(llvm::raw_ostream& out) const -> void;
 
  private:
   uint64_t value_ = 0;
@@ -527,30 +522,6 @@ inline auto CarbonHashValue(const T (&arg)[N], uint64_t seed) -> HashCode {
   return CarbonHashValue(llvm::ArrayRef(arg), seed);
 }
 
-inline auto CarbonHashValue(llvm::APInt value, uint64_t seed) -> HashCode {
-  Hasher hasher(seed);
-  if (LLVM_LIKELY(value.isSingleWord())) {
-    hasher.Hash(value.getBitWidth(), value.getZExtValue());
-  } else {
-    hasher.HashRaw(value.getBitWidth());
-    hasher.HashSizedBytes(
-        llvm::ArrayRef(value.getRawData(), value.getNumWords()));
-  }
-  return static_cast<HashCode>(hasher);
-}
-
-inline auto CarbonHashValue(llvm::APFloat value, uint64_t seed) -> HashCode {
-  Hasher hasher(seed);
-  // Hashing floating point numbers is complex and depends on the specific
-  // internal semantics of `APFloat`, so delegate to the LLVM hashing framework
-  // here. We re-hash the result to mix in our seed. All of this is a bit
-  // inefficient, and we can revisit this to provide a dedicated implementation
-  // if it becomes a bottleneck.
-  using llvm::hash_value;
-  hasher.HashRaw(hash_value(value));
-  return static_cast<HashCode>(hasher);
-}
-
 template <typename... Ts>
 inline auto CarbonHashValue(const std::tuple<Ts...>& value, uint64_t seed)
     -> HashCode {
@@ -567,6 +538,14 @@ inline auto CarbonHashValue(const std::pair<T, U>& value, uint64_t seed)
   return static_cast<HashCode>(hasher);
 }
 
+// Extension point for types defined outside of Carbon that cannot be found by
+// ADL in their own namespace and cannot be declared before this point.
+template <typename T>
+struct CustomHashValue;
+
+template <typename T>
+concept HasCustomHashValue = requires { CustomHashValue<T>::Hash; };
+
 // Implementation detail predicate to detect if there is a `CarbonHashValue`
 // overload available for a particular type, either in this namespace or found
 // via ADL. Note that this should not be moved above any overloads.
@@ -618,14 +597,19 @@ concept CanHashAsRawDataType = std::same_as<T, std::nullptr_t> ||
 // `HasCarbonHashValue`, this must not be moved above any of those overloads.
 template <typename T>
 inline auto DispatchImpl(const T& value, uint64_t seed) -> HashCode {
-  // If we have an explicit overload for `CarbonHashValue`, call it. This may be
-  // provided above or via ADL, and is preferred as it represents an explicit
-  // request for how the type is hashed.
   if constexpr (HasCarbonHashValue<T>) {
+    // If we have an explicit overload for `CarbonHashValue`, call it. This may
+    // be provided above or via ADL, and is preferred as it represents an
+    // explicit request for how the type is hashed.
     return CarbonHashValue(value, seed);
+  } else if constexpr (HasCustomHashValue<T>) {
+    // If we have an explicit specialization for `CustomHashValue`, call it.
+    // This is a fallback explicit hashing path that doesn't require ADL or
+    // being in this header.
+    return CustomHashValue<T>::Hash(value, seed);
   } else if constexpr (CanHashAsRawDataType<T>) {
-    // There was no explicit overload to call, but the type allows us to hash it
-    // as raw data, do so.
+    // There was no explicit overload or specialization to call, but the type
+    // allows us to hash it as raw data, do so.
     Hasher hasher(seed);
     hasher.HashRaw(MapToRawDataType(value));
     return static_cast<HashCode>(hasher);
@@ -813,11 +797,13 @@ inline auto Hasher::Hash(const Ts&... values) -> void {
 
   using InternalHashDispatch::CanHashAsRawDataType;
   using InternalHashDispatch::HasCarbonHashValue;
+  using InternalHashDispatch::HasCustomHashValue;
   using InternalHashDispatch::MapToRawDataType;
 
   // Special-case a single element tuple that we will hash as raw data.
-  if constexpr (sizeof...(Ts) == 1 && (... && (!HasCarbonHashValue<Ts> &&
-                                               CanHashAsRawDataType<Ts>))) {
+  if constexpr (sizeof...(Ts) == 1 &&
+                (... && (!HasCarbonHashValue<Ts> && !HasCustomHashValue<Ts> &&
+                         CanHashAsRawDataType<Ts>))) {
     HashRaw(MapToRawDataType(values)...);
     return;
   }
@@ -830,12 +816,15 @@ inline auto Hasher::Hash(const Ts&... values) -> void {
   // a little bit wasteful in some cases, collapsing down to a flat array of
   // 64-bit integers is more efficient to hash.
   auto map_value = []<typename T>(const T& value) -> uint64_t {
-    if constexpr (HasCarbonHashValue<T>) {
+    if constexpr (HasCarbonHashValue<T> || HasCustomHashValue<T>) {
       // Use the top-level `HashValue` to re-dispatch to the custom
       // implementation with a fixed seed.
       return static_cast<uint64_t>(HashValue(value));
     } else if constexpr (CanHashAsRawDataType<T>) {
       auto raw_value = MapToRawDataType(value);
+      // If we are hashing a pointer, then `raw_value` is a pointer, but that
+      // is what we want the size of.
+      // NOLINTNEXTLINE(bugprone-sizeof-expression)
       if constexpr (sizeof(raw_value) <= 8) {
         return ReadSmall(raw_value);
       } else {
@@ -866,11 +855,12 @@ template <typename T>
 inline auto Hasher::HashArray(llvm::ArrayRef<T> values) -> void {
   using InternalHashDispatch::CanHashAsRawDataType;
   using InternalHashDispatch::HasCarbonHashValue;
+  using InternalHashDispatch::HasCustomHashValue;
 
   // This logic similarly mirrors `InternalHashDispatch::DispatchImpl`, but is
   // specialized here to allow us to efficiently process the array when it
   // *doesn't* require recursive hashing.
-  if constexpr (HasCarbonHashValue<T>) {
+  if constexpr (HasCarbonHashValue<T> || HasCustomHashValue<T>) {
     // Use a trivial loop to give consistent behavior for arrays requiring
     // recursive hashing. This isn't terribly efficient, but if clients care
     // they should specialize the entire hashing operation. For simple, tiny

common/hashing_llvm.h

@@ -0,0 +1,47 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_HASHING_LLVM_H_
+#define CARBON_COMMON_HASHING_LLVM_H_
+
+#include "common/hashing.h"
+#include "llvm/ADT/APFloat.h"
+#include "llvm/ADT/APInt.h"
+#include "llvm/ADT/Hashing.h"
+
+namespace Carbon::InternalHashDispatch {
+
+template <>
+struct CustomHashValue<llvm::APInt> {
+  static auto Hash(llvm::APInt value, uint64_t seed) -> HashCode {
+    Hasher hasher(seed);
+    if (LLVM_LIKELY(value.isSingleWord())) {
+      hasher.Hash(value.getBitWidth(), value.getZExtValue());
+    } else {
+      hasher.HashRaw(value.getBitWidth());
+      hasher.HashSizedBytes(
+          llvm::ArrayRef(value.getRawData(), value.getNumWords()));
+    }
+    return static_cast<HashCode>(hasher);
+  }
+};
+
+template <>
+struct CustomHashValue<llvm::APFloat> {
+  static auto Hash(llvm::APFloat value, uint64_t seed) -> HashCode {
+    Hasher hasher(seed);
+    // Hashing floating point numbers is complex and depends on the specific
+    // internal semantics of `APFloat`, so delegate to the LLVM hashing
+    // framework here. We re-hash the result to mix in our seed. All of this is
+    // a bit inefficient, and we can revisit this to provide a dedicated
+    // implementation if it becomes a bottleneck.
+    using llvm::hash_value;
+    hasher.HashRaw(hash_value(value));
+    return static_cast<HashCode>(hasher);
+  }
+};
+
+}  // namespace Carbon::InternalHashDispatch
+
+#endif  // CARBON_COMMON_HASHING_LLVM_H_

common/hashing_test.cpp

@@ -13,6 +13,7 @@
 #include <type_traits>
 #include <utility>
 
+#include "common/hashing_llvm.h"
 #include "common/raw_string_ostream.h"
 #include "llvm/ADT/Sequence.h"
 #include "llvm/ADT/StringExtras.h"
@@ -491,41 +492,6 @@ struct HashedValue {
 
 using HashedString = HashedValue<std::string>;
 
-template <typename T>
-auto PrintFullWidthHex(llvm::raw_ostream& os, T value) {
-  static_assert(sizeof(T) == 1 || sizeof(T) == 2 || sizeof(T) == 4 ||
-                sizeof(T) == 8);
-  // Given the nature of a format string and the good formatting, a nested
-  // conditional seems like the most readable structure.
-  // NOLINTBEGIN(readability-avoid-nested-conditional-operator)
-  os << llvm::formatv(sizeof(T) == 1   ? "{0:x2}"
-                      : sizeof(T) == 2 ? "{0:x4}"
-                      : sizeof(T) == 4 ? "{0:x8}"
-                                       : "{0:x16}",
-                      static_cast<uint64_t>(value));
-  // NOLINTEND(readability-avoid-nested-conditional-operator)
-}
-
-template <typename T>
-  requires std::integral<T>
-auto operator<<(llvm::raw_ostream& os, HashedValue<T> hv)
-    -> llvm::raw_ostream& {
-  os << "hash " << hv.hash << " for value ";
-  PrintFullWidthHex(os, hv.v);
-  return os;
-}
-
-template <typename T, typename U>
-  requires std::integral<T> && std::integral<U>
-auto operator<<(llvm::raw_ostream& os, HashedValue<std::pair<T, U>> hv)
-    -> llvm::raw_ostream& {
-  os << "hash " << hv.hash << " for pair of ";
-  PrintFullWidthHex(os, hv.v.first);
-  os << " and ";
-  PrintFullWidthHex(os, hv.v.second);
-  return os;
-}
-
 struct Collisions {
   int total;
   int median;
@@ -770,8 +736,8 @@ struct SparseHashTestParamRanges {
 
 template <typename ParamRanges>
 struct SparseHashTest : ::testing::Test {
-  using ByteCount = typename ParamRanges::ByteCount;
-  using SetBitCount = typename ParamRanges::SetBitCount;
+  using ByteCount = ParamRanges::ByteCount;
+  using SetBitCount = ParamRanges::SetBitCount;
 
   static auto GetHashedByteStrings() {
     llvm::SmallVector<HashedString> hashes;

common/hashtable_key_context_test.cpp

@@ -7,6 +7,8 @@
 #include <gmock/gmock.h>
 #include <gtest/gtest.h>
 
+#include "common/hashing_llvm.h"
+
 namespace Carbon {
 namespace {
 

common/map.h

@@ -61,13 +61,21 @@ class MapView
     : RawHashtable::ViewImpl<InputKeyT, InputValueT, InputKeyContextT> {
   using ImplT =
       RawHashtable::ViewImpl<InputKeyT, InputValueT, InputKeyContextT>;
-  using EntryT = typename ImplT::EntryT;
+  using EntryT = ImplT::EntryT;
 
  public:
-  using KeyT = typename ImplT::KeyT;
-  using ValueT = typename ImplT::ValueT;
-  using KeyContextT = typename ImplT::KeyContextT;
-  using MetricsT = typename ImplT::MetricsT;
+  using KeyT = ImplT::KeyT;
+  using ValueT = ImplT::ValueT;
+  using KeyContextT = ImplT::KeyContextT;
+  using MetricsT = ImplT::MetricsT;
+
+  // A key and its value, as a pair of references. This is what iterating the
+  // map produces; there is no object in the table combining the two.
+  using Entry = ImplT::EntryRefT;
+
+  // A range over the key-value entries of the map. Bound to the lifetime of
+  // the viewed map, and invalidated by mutating it.
+  using Range = ImplT::EntryRange;
 
   // This type represents the result of lookup operations. It encodes whether
   // the lookup was a success as well as accessors for the key and value.
@@ -111,10 +119,8 @@ class MapView
   auto operator[](LookupKeyT lookup_key) const -> ValueT*
     requires(std::default_initializable<KeyContextT>);
 
-  // Run the provided callback for every key and value in the map.
-  template <typename CallbackT>
-  auto ForEach(CallbackT callback) -> void
-    requires(std::invocable<CallbackT, KeyT&, ValueT&>);
+  // Returns a range for iterating over all key-value entries in the map.
+  auto entries() const -> Range;
 
   // This routine is relatively inefficient and only intended for use in
   // benchmarking or logging of performance anomalies. The specific metrics
@@ -160,15 +166,17 @@ class MapBase : protected RawHashtable::BaseImpl<InputKeyT, InputValueT,
  protected:
   using ImplT =
       RawHashtable::BaseImpl<InputKeyT, InputValueT, InputKeyContextT>;
-  using EntryT = typename ImplT::EntryT;
+  using EntryT = ImplT::EntryT;
 
  public:
-  using KeyT = typename ImplT::KeyT;
-  using ValueT = typename ImplT::ValueT;
-  using KeyContextT = typename ImplT::KeyContextT;
+  using KeyT = ImplT::KeyT;
+  using ValueT = ImplT::ValueT;
+  using KeyContextT = ImplT::KeyContextT;
   using ViewT = MapView<KeyT, ValueT, KeyContextT>;
-  using LookupKVResult = typename ViewT::LookupKVResult;
-  using MetricsT = typename ImplT::MetricsT;
+  using LookupKVResult = ViewT::LookupKVResult;
+  using MetricsT = ImplT::MetricsT;
+  using Entry = ViewT::Entry;
+  using Range = ViewT::Range;
 
   // The result type for insertion operations both indicates whether an insert
   // was needed (as opposed to finding an existing element), and provides access
@@ -228,12 +236,12 @@ class MapBase : protected RawHashtable::BaseImpl<InputKeyT, InputValueT,
   }
 
   // Convenience forwarder to the view type.
-  template <typename CallbackT>
-  auto ForEach(CallbackT callback) const -> void
-    requires(std::invocable<CallbackT, KeyT&, ValueT&>)
-  {
-    return ViewT(*this).ForEach(callback);
-  }
+  auto entries() const& -> Range { return ViewT(*this).entries(); }
+  // Deleted on rvalues: the range refers to storage owned by this table, so a
+  // range built from a temporary map would dangle. Both qualifiers are needed
+  // as `&&` alone would leave a const rvalue binding to the `const&` overload.
+  auto entries() && = delete;
+  auto entries() const&& = delete;
 
   // Convenience forwarder to the view type.
   auto ComputeMetrics(KeyContextT key_context = KeyContextT()) const
@@ -385,8 +393,8 @@ class Map : public RawHashtable::TableImpl<
   using ImplT = RawHashtable::TableImpl<BaseT, SmallSize>;
 
  public:
-  using KeyT = typename BaseT::KeyT;
-  using ValueT = typename BaseT::ValueT;
+  using KeyT = BaseT::KeyT;
+  using ValueT = BaseT::ValueT;
 
   Map() = default;
   Map(const Map& arg) = default;
@@ -424,14 +432,9 @@ auto MapView<InputKeyT, InputValueT, InputKeyContextT>::operator[](
 }
 
 template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
-template <typename CallbackT>
-auto MapView<InputKeyT, InputValueT, InputKeyContextT>::ForEach(
-    CallbackT callback) -> void
-  requires(std::invocable<CallbackT, KeyT&, ValueT&>)
-{
-  this->ForEachEntry(
-      [callback](EntryT& entry) { callback(entry.key(), entry.value()); },
-      [](auto...) {});
+auto MapView<InputKeyT, InputValueT, InputKeyContextT>::entries() const
+    -> Range {
+  return this->ImplT::EntriesImpl();
 }
 
 template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>

common/map_benchmark.cpp

@@ -66,8 +66,8 @@ static constexpr bool IsCarbonMap =
 template <typename InMapT>
 struct MapWrapperImpl {
   using MapT = InMapT;
-  using KeyT = typename MapT::key_type;
-  using ValueT = typename MapT::mapped_type;
+  using KeyT = MapT::key_type;
+  using ValueT = MapT::mapped_type;
 
   MapT m;
 
@@ -93,6 +93,17 @@ struct MapWrapperImpl {
   }
 
   auto BenchErase(KeyT k) -> bool { return m.erase(k) != 0; }
+
+  // Visits every entry in the map, calling `cb` with the key and value of each
+  // one. Each map type is expected to traverse using whatever API it provides
+  // for this, so that the benchmark measures iterating the map rather than any
+  // specific iteration API.
+  template <typename CallbackT>
+  auto BenchIterate(CallbackT cb) -> void {
+    for (const auto& entry : m) {
+      cb(entry.first, entry.second);
+    }
+  }
 };
 
 // Explicit (partial) specialization for the Carbon map type that uses its
@@ -126,6 +137,13 @@ struct MapWrapperImpl<Map<KT, VT, MinSmallSize>> {
   }
 
   auto BenchErase(KeyT k) -> bool { return m.Erase(k); }
+
+  template <typename CallbackT>
+  auto BenchIterate(CallbackT cb) -> void {
+    for (auto [k, v] : m.entries()) {
+      cb(k, v);
+    }
+  }
 };
 
 // Provide a way to override the Carbon Map specific benchmark runs with another
@@ -218,8 +236,8 @@ auto ReportMetrics(const MapWrapper<MapT>& m_wrapper, benchmark::State& state)
 template <typename MapT>
 static void BM_MapContainsHit(benchmark::State& state) {
   using MapWrapperT = MapWrapper<MapT>;
-  using KT = typename MapWrapperT::KeyT;
-  using VT = typename MapWrapperT::ValueT;
+  using KT = MapWrapperT::KeyT;
+  using VT = MapWrapperT::ValueT;
   MapWrapperT m;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -254,8 +272,8 @@ MAP_BENCHMARK_ONE_OP(BM_MapContainsHit, HitArgs);
 template <typename MapT>
 static void BM_MapContainsMiss(benchmark::State& state) {
   using MapWrapperT = MapWrapper<MapT>;
-  using KT = typename MapWrapperT::KeyT;
-  using VT = typename MapWrapperT::ValueT;
+  using KT = MapWrapperT::KeyT;
+  using VT = MapWrapperT::ValueT;
   MapWrapperT m;
   auto [keys, lookup_keys] = GetKeysAndMissKeys<KT>(state.range(0));
   for (auto k : keys) {
@@ -307,8 +325,8 @@ MAP_BENCHMARK_ONE_OP(BM_MapContainsMiss, SizeArgs);
 template <typename MapT>
 static void BM_MapLookupHit(benchmark::State& state) {
   using MapWrapperT = MapWrapper<MapT>;
-  using KT = typename MapWrapperT::KeyT;
-  using VT = typename MapWrapperT::ValueT;
+  using KT = MapWrapperT::KeyT;
+  using VT = MapWrapperT::ValueT;
   MapWrapperT m;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -363,8 +381,8 @@ MAP_BENCHMARK_ONE_OP_SIZE(BM_MapLookupHit, HitArgs, LowZeroBitInt<32>, int);
 template <typename MapT>
 static void BM_MapUpdateHit(benchmark::State& state) {
   using MapWrapperT = MapWrapper<MapT>;
-  using KT = typename MapWrapperT::KeyT;
-  using VT = typename MapWrapperT::ValueT;
+  using KT = MapWrapperT::KeyT;
+  using VT = MapWrapperT::ValueT;
   MapWrapperT m;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -405,8 +423,8 @@ MAP_BENCHMARK_ONE_OP(BM_MapUpdateHit, HitArgs);
 template <typename MapT>
 static void BM_MapEraseUpdateHit(benchmark::State& state) {
   using MapWrapperT = MapWrapper<MapT>;
-  using KT = typename MapWrapperT::KeyT;
-  using VT = typename MapWrapperT::ValueT;
+  using KT = MapWrapperT::KeyT;
+  using VT = MapWrapperT::ValueT;
   MapWrapperT m;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -463,8 +481,8 @@ MAP_BENCHMARK_ONE_OP(BM_MapEraseUpdateHit, HitArgs);
 template <typename MapT>
 static void BM_MapInsertSeq(benchmark::State& state) {
   using MapWrapperT = MapWrapper<MapT>;
-  using KT = typename MapWrapperT::KeyT;
-  using VT = typename MapWrapperT::ValueT;
+  using KT = MapWrapperT::KeyT;
+  using VT = MapWrapperT::ValueT;
   constexpr ssize_t LookupKeysSize = 1 << 8;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), LookupKeysSize);
@@ -516,5 +534,45 @@ static void BM_MapInsertSeq(benchmark::State& state) {
 }
 MAP_BENCHMARK_ONE_OP(BM_MapInsertSeq, SizeArgs);
 
+// Benchmark visiting every entry in a map.
+//
+// Unlike the lookup benchmarks, this walks the table's storage from end to end
+// rather than probing it, so it is largely a measure of how densely entries are
+// packed and how cheaply empty slots can be skipped. There is no dependency
+// between the entries visited, and so this is a throughput measurement.
+//
+// Each batch is a single complete traversal of the map, with the batch size set
+// to the number of entries so that the reported time is the per-entry cost.
+template <typename MapT>
+static void BM_MapIterate(benchmark::State& state) {
+  using MapWrapperT = MapWrapper<MapT>;
+  using KT = typename MapWrapperT::KeyT;
+  using VT = typename MapWrapperT::ValueT;
+  MapWrapperT m;
+  auto [keys, _] = GetKeysAndMissKeys<KT>(state.range(0));
+  for (auto k : keys) {
+    bool inserted = m.BenchInsert(k, MakeValue<VT>());
+    CARBON_DCHECK(inserted, "Must be a successful insert!");
+  }
+
+  while (state.KeepRunningBatch(keys.size())) {
+    ssize_t sum = 0;
+    m.BenchIterate([&sum](const KT& k, const VT& v) {
+      // Consume both the key and the value so that neither the traversal nor
+      // the loads out of the entries can be optimized away.
+      sum += ValueToBool(k) + ValueToBool(v);
+    });
+    benchmark::DoNotOptimize(sum);
+  }
+
+  // The time is already per-entry, so an iteration-invariant rate of one gives
+  // the throughput of entries visited.
+  state.counters["KeyRate"] =
+      benchmark::Counter(1, benchmark::Counter::kIsIterationInvariantRate);
+
+  ReportMetrics(m, state);
+}
+MAP_BENCHMARK_ONE_OP(BM_MapIterate, SizeArgs);
+
 }  // namespace
 }  // namespace Carbon

common/map_test.cpp

@@ -7,7 +7,10 @@
 #include <gmock/gmock.h>
 #include <gtest/gtest.h>
 
+#include <concepts>
 #include <initializer_list>
+#include <iterator>
+#include <ranges>
 #include <type_traits>
 #include <utility>
 #include <vector>
@@ -37,19 +40,20 @@ using RawHashtable::MoveOnlyTestData;
 using RawHashtable::TestData;
 using RawHashtable::TestKeyContext;
 using ::testing::Pair;
+using ::testing::UnorderedElementsAre;
 using ::testing::UnorderedElementsAreArray;
 
 template <typename MapT, typename MatcherRangeT>
 auto ExpectMapElementsAre(MapT&& m, MatcherRangeT element_matchers) -> void {
   // Now collect the elements into a container.
-  using KeyT = typename std::remove_reference<MapT>::type::KeyT;
-  using ValueT = typename std::remove_reference<MapT>::type::ValueT;
+  using KeyT = std::remove_reference<MapT>::type::KeyT;
+  using ValueT = std::remove_reference<MapT>::type::ValueT;
   std::vector<
       std::pair<std::reference_wrapper<KeyT>, std::reference_wrapper<ValueT>>>
       map_entries;
-  m.ForEach([&map_entries](KeyT& k, ValueT& v) {
+  for (auto [k, v] : m.entries()) {
     map_entries.push_back({std::ref(k), std::ref(v)});
-  });
+  }
 
   // Use the GoogleMock unordered container matcher to validate and show errors
   // on wrong elements.
@@ -68,7 +72,7 @@ auto ExpectMapElementsAre(MapT&& m,
 template <typename ValueCB, typename RangeT, typename... RangeTs>
 auto MakeKeyValues(ValueCB value_cb, RangeT&& range, RangeTs&&... ranges)
     -> auto {
-  using KeyT = typename RangeT::value_type;
+  using KeyT = RangeT::value_type;
   using ValueT = decltype(value_cb(std::declval<KeyT>()));
   std::vector<std::pair<KeyT, ValueT>> elements;
   auto add_range = [&](RangeT&& r) {
@@ -865,5 +869,112 @@ TEST(MapContextTest, Basic) {
       m, MakeKeyValues([](int k) { return k * 100 + 1; }, llvm::seq(1, 512)));
 }
 
+TYPED_TEST(MapTest, Range) {
+  using MapT = TypeParam;
+  using Range = decltype(std::declval<const MapT&>().entries());
+  using Iter = typename Range::Iterator;
+
+  static_assert(std::forward_iterator<Iter>);
+  static_assert(std::same_as<decltype(std::declval<Range>().begin()), Iter>);
+  static_assert(std::same_as<decltype(std::declval<Range>().end()), Iter>);
+  static_assert(std::ranges::forward_range<Range>);
+  static_assert(std::ranges::common_range<Range>);
+
+  MapT m;
+  EXPECT_EQ(m.entries().begin(), m.entries().end());
+  for (auto [k, v] : m.entries()) {
+    static_cast<void>(k);
+    static_cast<void>(v);
+    FAIL() << "Empty map range should have no elements";
+  }
+
+  for (int i = 1; i <= 5; ++i) {
+    m.Insert(i, i * 10);
+  }
+
+  int count = 0;
+  for (const auto& [k, v] : m.entries()) {
+    EXPECT_EQ(v, m.Lookup(k).value());
+    ++count;
+  }
+  EXPECT_EQ(count, 5);
+
+  EXPECT_THAT(m.entries(),
+              UnorderedElementsAre(Pair(1, 10), Pair(2, 20), Pair(3, 30),
+                                   Pair(4, 40), Pair(5, 50)));
+
+  using KeyT = typename MapT::KeyT;
+  using ValueT = typename MapT::ValueT;
+  using KeyContextT = typename MapT::KeyContextT;
+  MapView<const KeyT, const ValueT, KeyContextT> cv = m;
+  int cv_count = 0;
+  for (auto [k, v] : cv.entries()) {
+    static_assert(std::is_const_v<std::remove_reference_t<decltype(k)>>);
+    static_assert(std::is_const_v<std::remove_reference_t<decltype(v)>>);
+    EXPECT_EQ(v, m.Lookup(k).value());
+    ++cv_count;
+  }
+  EXPECT_EQ(cv_count, 5);
+  EXPECT_THAT(cv.entries(),
+              UnorderedElementsAre(Pair(1, 10), Pair(2, 20), Pair(3, 30),
+                                   Pair(4, 40), Pair(5, 50)));
+
+  for (auto [k, v] : m.entries()) {
+    if constexpr (requires { v.value; }) {
+      v.value = 99;
+    } else {
+      v = 99;
+    }
+  }
+  for (const auto& [k, v] : m.entries()) {
+    if constexpr (requires { v.value; }) {
+      EXPECT_EQ(v.value, 99);
+    } else {
+      EXPECT_EQ(v, 99);
+    }
+  }
+  EXPECT_THAT(m.entries(),
+              UnorderedElementsAre(Pair(1, 99), Pair(2, 99), Pair(3, 99),
+                                   Pair(4, 99), Pair(5, 99)));
+
+  auto r = m.entries();
+  int iter_count = 0;
+  for (auto it = r.begin(); it != r.end(); ++it) {
+    EXPECT_EQ(it->second, m.Lookup(it->first).value());
+    EXPECT_EQ((*it).second, m.Lookup((*it).first).value());
+    ++iter_count;
+  }
+  EXPECT_EQ(iter_count, 5);
+
+  auto it = r.begin();
+  auto prev = it++;
+  EXPECT_NE(it, prev);
+}
+
+TYPED_TEST(MoveOnlyMapTest, Range) {
+  TypeParam m;
+  m.Insert(1, 10);
+  m.Insert(2, 20);
+
+  int count = 0;
+  for (auto [k, v] : m.entries()) {
+    EXPECT_EQ(v.value, k.value * 10);
+    ++count;
+  }
+  EXPECT_EQ(count, 2);
+}
+
+#ifndef NDEBUG
+TEST(MapDeathTest, MutateDuringIterationFails) {
+  EXPECT_DEATH(([] {
+                 Map<int, int> m;
+                 m.Insert(1, 10);
+                 auto range = m.entries();
+                 m.Insert(2, 20);
+               }()),
+               "Hashtable mutated during iteration");
+}
+#endif
+
 }  // namespace
 }  // namespace Carbon::Testing

common/ostream.h

@@ -7,6 +7,7 @@
 
 // Libraries should include this header instead of raw_ostream.
 
+#include <compare>
 #include <concepts>
 #include <ostream>
 #include <type_traits>
@@ -17,11 +18,31 @@
 
 namespace Carbon {
 
-// CRTP base class for printable types. Children (DerivedT) must implement:
+// CRTP base class for printable types. Derived classes (DerivedT) must
+// implement:
 // - auto Print(llvm::raw_ostream& out) const -> void
 template <typename DerivedT>
 // NOLINTNEXTLINE(bugprone-crtp-constructor-accessibility)
 class Printable {
+  // Comparisons of the base class itself, which is empty and so always compares
+  // equal, allowing derived classes to default their own comparison operators.
+  //
+  // These are templated so that they are only used when the types of the
+  // arguments are exactly `Printable`, rather than a derived class, and are
+  // hidden friends so that they aren't candidates for unrelated comparisons.
+  template <typename T>
+    requires std::same_as<T, Printable>
+  friend constexpr auto operator==(const T& /*lhs*/, const T& /*rhs*/) noexcept
+      -> bool {
+    return true;
+  }
+  template <typename T>
+    requires std::same_as<T, Printable>
+  friend constexpr auto operator<=>(const T& /*lhs*/, const T& /*rhs*/) noexcept
+      -> std::strong_ordering {
+    return std::strong_ordering::equal;
+  }
+
   // Supports printing to llvm::raw_ostream.
   friend auto operator<<(llvm::raw_ostream& out, const DerivedT& obj)
       -> llvm::raw_ostream& {

common/ostream_test.cpp

@@ -0,0 +1,253 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/ostream.h"
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+
+#include <compare>
+#include <concepts>
+#include <limits>
+#include <sstream>
+#include <string>
+#include <type_traits>
+#include <utility>
+
+#include "common/raw_string_ostream.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallVector.h"
+
+namespace Carbon::Testing {
+namespace {
+
+using ::testing::ElementsAre;
+
+// Whether two types can be compared with both `==` and `<=>`.
+template <typename LhsT, typename RhsT>
+concept Comparable = requires(const LhsT& lhs, const RhsT& rhs) {
+  lhs == rhs;
+  lhs <=> rhs;
+};
+
+// A child that defaults its comparisons with member declarations.
+struct Point : Printable<Point> {
+  int x;
+  int y;
+
+  constexpr Point(int x, int y) : x(x), y(y) {}
+
+  auto Print(llvm::raw_ostream& out) const -> void {
+    out << "(" << x << ", " << y << ")";
+  }
+
+  auto operator<=>(const Point& rhs) const = default;
+};
+
+// A child that defaults its comparisons with friend declarations, and whose
+// comparisons are neither trivial nor `noexcept`.
+struct Label : Printable<Label> {
+  std::string text;
+
+  explicit Label(std::string text) : text(std::move(text)) {}
+
+  auto Print(llvm::raw_ostream& out) const -> void { out << text; }
+
+  friend auto operator<=>(const Label& lhs, const Label& rhs) = default;
+};
+
+// A child that defaults equality without providing any ordering.
+struct Id : Printable<Id> {
+  int value;
+
+  constexpr explicit Id(int value) : value(value) {}
+
+  auto Print(llvm::raw_ostream& out) const -> void { out << "#" << value; }
+
+  auto operator==(const Id& rhs) const -> bool = default;
+};
+
+// A child whose defaulted comparison is only a partial ordering.
+struct Measure : Printable<Measure> {
+  double value;
+
+  constexpr explicit Measure(double value) : value(value) {}
+
+  auto Print(llvm::raw_ostream& out) const -> void { out << value; }
+
+  auto operator<=>(const Measure& rhs) const = default;
+};
+
+// A child that requests a weaker ordering than its members provide.
+struct Version : Printable<Version> {
+  int major;
+  int minor;
+
+  constexpr Version(int major, int minor) : major(major), minor(minor) {}
+
+  auto Print(llvm::raw_ostream& out) const -> void {
+    out << major << "." << minor;
+  }
+
+  auto operator<=>(const Version& rhs) const -> std::weak_ordering = default;
+};
+
+// A child that doesn't want to be compared at all.
+struct Opaque : Printable<Opaque> {
+  int value;
+
+  constexpr explicit Opaque(int value) : value(value) {}
+
+  auto Print(llvm::raw_ostream& out) const -> void { out << value; }
+};
+
+// A child that compares through an implicit conversion rather than through
+// operators of its own, the way `EnumBase` children do.
+class Level : public Printable<Level> {
+ public:
+  enum RawLevel { Low, High };
+
+  constexpr explicit Level(RawLevel value) : value_(value) {}
+
+  // NOLINTNEXTLINE(google-explicit-constructor)
+  explicit(false) constexpr operator RawLevel() const { return value_; }
+
+  auto Print(llvm::raw_ostream& out) const -> void {
+    out << (value_ == Low ? "low" : "high");
+  }
+
+ private:
+  RawLevel value_;
+};
+
+TEST(PrintableTest, Printing) {
+  RawStringOstream raw_out;
+  raw_out << Point(1, 2) << " " << Label("label");
+  EXPECT_EQ(raw_out.TakeStr(), "(1, 2) label");
+
+  std::ostringstream standard_out;
+  standard_out << Point(1, 2) << " " << Label("label");
+  EXPECT_EQ(standard_out.str(), "(1, 2) label");
+
+  EXPECT_EQ(PrintToString(Point(1, 2)), "(1, 2)");
+}
+
+TEST(PrintableTest, DefaultedEquality) {
+  EXPECT_EQ(Point(1, 2), Point(1, 2));
+  EXPECT_NE(Point(1, 2), Point(1, 3));
+  EXPECT_NE(Point(1, 2), Point(2, 2));
+
+  static_assert(Point(1, 2) == Point(1, 2));
+  static_assert(Point(1, 2) != Point(1, 3));
+
+  // The base class comparisons don't make defaulted comparisons throwing.
+  Point point(1, 2);
+  static_assert(noexcept(point == point));
+}
+
+TEST(PrintableTest, DefaultedOrdering) {
+  // Ordering is lexicographic in declaration order, with the empty base class
+  // contributing nothing.
+  EXPECT_LT(Point(1, 2), Point(1, 3));
+  EXPECT_LT(Point(1, 9), Point(2, 0));
+  EXPECT_LE(Point(1, 2), Point(1, 2));
+  EXPECT_GT(Point(2, 0), Point(1, 9));
+  EXPECT_GE(Point(1, 2), Point(1, 2));
+
+  EXPECT_EQ(Point(1, 2) <=> Point(1, 3), std::strong_ordering::less);
+  EXPECT_EQ(Point(1, 2) <=> Point(1, 2), std::strong_ordering::equal);
+  EXPECT_EQ(Point(1, 3) <=> Point(1, 2), std::strong_ordering::greater);
+
+  static_assert(std::totally_ordered<Point>);
+  static_assert(std::same_as<std::compare_three_way_result_t<Point>,
+                             std::strong_ordering>);
+  static_assert(Point(1, 2) < Point(1, 3));
+
+  Point point(1, 2);
+  static_assert(noexcept(point <=> point));
+}
+
+TEST(PrintableTest, DefaultedFriendComparison) {
+  EXPECT_EQ(Label("a"), Label("a"));
+  EXPECT_NE(Label("a"), Label("b"));
+  EXPECT_LT(Label("a"), Label("b"));
+  EXPECT_EQ(Label("a") <=> Label("b"), std::strong_ordering::less);
+
+  static_assert(std::totally_ordered<Label>);
+}
+
+TEST(PrintableTest, DefaultedEqualityWithoutOrdering) {
+  EXPECT_EQ(Id(1), Id(1));
+  EXPECT_NE(Id(1), Id(2));
+
+  static_assert(std::equality_comparable<Id>);
+  static_assert(!std::totally_ordered<Id>);
+  static_assert(!std::three_way_comparable<Id>);
+}
+
+TEST(PrintableTest, DefaultedComparisonCategories) {
+  static_assert(std::same_as<std::compare_three_way_result_t<Measure>,
+                             std::partial_ordering>);
+  static_assert(std::same_as<std::compare_three_way_result_t<Version>,
+                             std::weak_ordering>);
+
+  EXPECT_LT(Measure(1.0), Measure(2.0));
+  EXPECT_EQ(Measure(1.0) <=> Measure(2.0), std::partial_ordering::less);
+
+  // The base class comparing equal must not make unordered values ordered.
+  Measure nan(std::numeric_limits<double>::quiet_NaN());
+  EXPECT_EQ(nan <=> Measure(1.0), std::partial_ordering::unordered);
+  EXPECT_NE(nan, nan);
+
+  EXPECT_LT(Version(1, 0), Version(1, 1));
+  EXPECT_EQ(Version(1, 0) <=> Version(1, 1), std::weak_ordering::less);
+}
+
+TEST(PrintableTest, NoComparisonWithoutDefaulting) {
+  // Inheriting from `Printable` must not by itself make a type comparable, and
+  // in particular must not make distinct values compare equal.
+  static_assert(!std::equality_comparable<Opaque>);
+  static_assert(!std::totally_ordered<Opaque>);
+  static_assert(!std::three_way_comparable<Opaque>);
+
+  EXPECT_EQ(PrintToString(Opaque(1)), "1");
+}
+
+TEST(PrintableTest, NoComparisonBetweenBaseAndChild) {
+  // The base class comparisons are viable only between two base class
+  // subobjects, and so don't apply to comparing a child with one, whether or
+  // not the child provides comparisons of its own.
+  static_assert(Comparable<Printable<Opaque>, Printable<Opaque>>);
+  static_assert(!Comparable<Printable<Opaque>, Opaque>);
+  static_assert(!Comparable<Opaque, Printable<Opaque>>);
+  static_assert(!Comparable<Printable<Label>, Label>);
+  static_assert(!Comparable<Label, Printable<Label>>);
+}
+
+TEST(PrintableTest, ComparisonThroughConversion) {
+  // The base class comparisons must not displace comparisons that a child
+  // provides through a conversion.
+  EXPECT_TRUE(Level(Level::Low) == Level(Level::Low));
+  EXPECT_FALSE(Level(Level::Low) == Level(Level::High));
+  EXPECT_TRUE(Level(Level::Low) < Level(Level::High));
+  EXPECT_TRUE(Level(Level::High) == Level::High);
+}
+
+TEST(PrintableTest, ComparisonsUsableGenerically) {
+  llvm::SmallVector<Point> points = {Point(2, 1), Point(1, 2), Point(1, 1)};
+  llvm::sort(points);
+  EXPECT_THAT(points, ElementsAre(Point(1, 1), Point(1, 2), Point(2, 1)));
+
+  EXPECT_EQ(llvm::find(points, Point(1, 2)), points.begin() + 1);
+}
+
+TEST(PrintableTest, EmptyBaseClass) {
+  // The comparison support must not add any state to children.
+  static_assert(sizeof(Point) == 2 * sizeof(int));
+  static_assert(sizeof(Id) == sizeof(int));
+  static_assert(std::is_empty_v<Printable<Point>>);
+}
+
+}  // namespace
+}  // namespace Carbon::Testing

common/raw_hashtable.cpp

@@ -10,4 +10,9 @@ namespace Carbon::RawHashtable {
 
 volatile std::byte global_addr_seed{1};
 
+#ifndef NDEBUG
+std::atomic<HashCode> entropy_hash =
+    Carbon::HashValue(reinterpret_cast<uint64_t>(&global_addr_seed));
+#endif
+
 }  // namespace Carbon::RawHashtable

common/raw_hashtable.h

@@ -6,6 +6,7 @@
 #define CARBON_COMMON_RAW_HASHTABLE_H_
 
 #include <algorithm>
+#include <atomic>
 #include <concepts>
 #include <cstddef>
 #include <cstring>
@@ -18,6 +19,7 @@
 #include "common/concepts.h"
 #include "common/hashing.h"
 #include "common/raw_hashtable_metadata_group.h"
+#include "llvm/ADT/iterator.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/MathExtras.h"
 
@@ -122,10 +124,15 @@
 //   null. Since it doesn't track the exact number of filled entries in a table,
 //   it doesn't support a container-style `size` API.
 //
-// - There is no direct iterator support because of the complexity of embedding
-//   the group-based metadata scanning into an iterator model. Instead, there is
-//   just a for-each method that is passed a lambda to observe all entries. The
-//   order of this observation is also not guaranteed.
+// - Iteration is provided by a range object rather than by iterators hanging
+//   directly off the table, because the debug-only checks for mutation during
+//   iteration need state that outlives a single iterator: see `EntryRange`
+//   below. Obtaining one is an explicit call (`entries()`), as scanning an
+//   entire table is a costly operation that shouldn't be hidden behind a bare
+//   `begin()`/`end()` pair.
+//
+//   The order of iteration is not guaranteed, and debug builds actively vary it
+//   between ranges to keep callers from depending on it.
 namespace Carbon::RawHashtable {
 
 // Which prefetch strategies to enable can be controlled via macros to enable
@@ -152,7 +159,7 @@ inline constexpr ssize_t MinAllocatedSize = std::max<ssize_t>(64, MaxGroupSize);
 // An entry in the hashtable storage of a `KeyT` and `ValueT` object.
 //
 // Allows manual construction, destruction, and access to these values so we can
-// create arrays af the entries prior to populating them with actual keys and
+// create arrays of the entries prior to populating them with actual keys and
 // values.
 template <typename KeyT, typename ValueT>
 struct StorageEntry {
@@ -168,6 +175,20 @@ struct StorageEntry {
       IsTriviallyRelocatable || (std::is_copy_constructible_v<KeyT> &&
                                  std::is_copy_constructible_v<ValueT>);
 
+  // How iteration refers to an entry, and the iterator traits that follow.
+  //
+  // The key and value are stored side by side with nothing combining them, so
+  // a reference to an entry is a pair of references built on demand. That pair
+  // is a *proxy* reference: C++20 forward iterators permit one, but C++17
+  // algorithms may assume a forward iterator's reference is a real lvalue, so
+  // the C++17 category is `input`.
+  using RefT = std::pair<KeyT&, ValueT&>;
+  using IterValueT = RefT;
+  using IterPointerT = const RefT*;
+  using IterCategoryT = std::input_iterator_tag;
+
+  auto ref() -> RefT { return RefT(key(), value()); }
+
   auto key() const -> const KeyT& {
     // Ensure we don't need more alignment than available. Inside a method body
     // to apply to the complete type.
@@ -194,11 +215,21 @@ struct StorageEntry {
   // construction. As a consequence, this struct only provides the storage and
   // we have to manually manage the construction, move, and destruction of the
   // objects.
+  //
+  // Destroys the key and value behind an entry reference. Iteration hands back
+  // `RefT` rather than the entry, so this is how a walked entry is destroyed.
+  static auto DestroyRef(RefT ref) -> void {
+    ref.first.~KeyT();
+    ref.second.~ValueT();
+  }
+
+  // Destroys the key and value of this entry. The common case is destroying an
+  // entry found in the table's storage, where there is no reference to hand to
+  // `DestroyRef`.
   auto Destroy() -> void {
     static_assert(!IsTriviallyDestructible,
                   "Should never instantiate when trivial!");
-    key().~KeyT();
-    value().~ValueT();
+    DestroyRef(ref());
   }
 
   auto CopyFrom(const StorageEntry& entry) -> void {
@@ -241,6 +272,15 @@ struct StorageEntry<KeyT, void> {
   static constexpr bool IsCopyable =
       IsTriviallyRelocatable || std::is_copy_constructible_v<KeyT>;
 
+  // As above, but a set's entry is nothing but its key, so a reference to an
+  // entry is a true lvalue reference and the iterator is a plain forward one.
+  using RefT = KeyT&;
+  using IterValueT = std::remove_cv_t<KeyT>;
+  using IterPointerT = KeyT*;
+  using IterCategoryT = std::forward_iterator_tag;
+
+  auto ref() -> RefT { return key(); }
+
   auto key() const -> const KeyT& {
     // Ensure we don't need more alignment than available.
     static_assert(
@@ -254,10 +294,12 @@ struct StorageEntry<KeyT, void> {
     return const_cast<KeyT&>(const_cast<const StorageEntry*>(this)->key());
   }
 
+  static auto DestroyRef(RefT ref) -> void { ref.~KeyT(); }
+
   auto Destroy() -> void {
     static_assert(!IsTriviallyDestructible,
                   "Should never instantiate when trivial!");
-    key().~KeyT();
+    DestroyRef(ref());
   }
 
   auto CopyFrom(const StorageEntry& entry) -> void
@@ -360,6 +402,13 @@ class ViewImpl {
   using EntryT = StorageEntry<KeyT, ValueT>;
   using MetricsT = Metrics;
 
+  // What iterating over the table's entries produces: a `KeyT&` for a set, and
+  // a `std::pair<KeyT&, ValueT&>` for a map. See `StorageEntry`.
+  using EntryRefT = EntryT::RefT;
+
+  // The range type produced by `EntriesImpl`.
+  class EntryRange;
+
   friend class BaseImpl<KeyT, ValueT, KeyContextT>;
   template <typename InputBaseT, ssize_t SmallSize>
   friend class TableImpl;
@@ -385,13 +434,11 @@ class ViewImpl {
   auto LookupEntry(LookupKeyT lookup_key, KeyContextT key_context) const
       -> EntryT*;
 
-  // Calls `entry_callback` for each entry in the hashtable. All the entries
-  // within a specific group are visited first, and then `group_callback` is
-  // called on the group itself. The `group_callback` is typically only used by
-  // the internals of the hashtable.
-  template <typename EntryCallbackT, typename GroupCallbackT>
-  auto ForEachEntry(EntryCallbackT entry_callback,
-                    GroupCallbackT group_callback) const -> void;
+  // Returns a range for iterating over all entries in the hashtable.
+  //
+  // The returned range copies this view, so it remains valid for as long as the
+  // underlying table does, independent of this view's lifetime.
+  auto EntriesImpl() const -> EntryRange;
 
   // Returns a collection of informative metrics on the the current state of the
   // table, useful for performance analysis. These include relatively slow to
@@ -425,7 +472,7 @@ class ViewImpl {
   auto metadata() const -> uint8_t* {
     return reinterpret_cast<uint8_t*>(storage_);
   }
-  auto entries() const -> EntryT* {
+  auto entries_data() const -> EntryT* {
     return reinterpret_cast<EntryT*>(reinterpret_cast<std::byte*>(storage_) +
                                      EntriesOffset(alloc_size_));
   }
@@ -457,6 +504,172 @@ class ViewImpl {
   Storage* storage_;
 };
 
+// A range over the entries of a hashtable.
+//
+// A dedicated range object is used rather than a plain pair of iterators (such
+// as `llvm::iterator_range`) because the range scopes two debug-only behaviors
+// that a bare iterator pair has nowhere to store:
+//
+// - Mutation checking: the range snapshots a hash of the table's metadata on
+//   construction and re-checks it on destruction, catching tables that were
+//   mutated while iteration was active.
+// - Traversal order: the group at which iteration starts, and the stride it
+//   walks the groups with, are drawn from an entropy pool once when the range
+//   is constructed.
+//   Deriving them here rather than in `begin()` keeps `begin()` a pure function
+//   of the range so that it can be called repeatedly, as forward ranges
+//   require, while still varying the order between separately created ranges.
+//
+// The range holds the view *by value*; views are two words and designed to be
+// cheap to copy. It deliberately does not point back at the view it was created
+// from, as views are routinely temporaries or by-value parameters whose
+// lifetime is shorter than the table they refer to.
+//
+// This type provides only the minimal `begin()` and `end()` interface needed by
+// range-based for loops and the range concepts, which also avoids any
+// compile-time cost from including `<ranges>`.
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+class ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntryRange {
+ public:
+  class Iterator;
+
+  using value_type = typename EntryT::IterValueT;
+  using reference = EntryRefT;
+  using difference_type = ssize_t;
+
+  explicit EntryRange(ViewImpl view);
+
+  // Copyable: every member is a scalar snapshot of the table. Copying a range
+  // in a debug build simply validates the same table state more than once.
+  EntryRange(const EntryRange&) = default;
+  auto operator=(const EntryRange&) -> EntryRange& = default;
+
+#ifndef NDEBUG
+  // Only debug builds declare a destructor, and so only they re-check the
+  // table on the way out. Release builds leave the range trivially
+  // destructible, and so trivial for the purposes of calls, letting it be
+  // passed and returned in registers.
+  ~EntryRange() { CheckInvariants(); }
+#endif
+
+  auto begin() const -> Iterator;
+  auto end() const -> Iterator;
+
+ private:
+  // The facade `Iterator` derives from. A class can't name one of its own
+  // aliases in its base-specifier, so naming it here lets `Iterator` spell it
+  // once instead of repeating it to get at the members it inherits.
+  using IteratorBase =
+      llvm::iterator_facade_base<Iterator, typename EntryT::IterCategoryT,
+                                 value_type, difference_type,
+                                 typename EntryT::IterPointerT, reference>;
+
+#ifndef NDEBUG
+  // Checks that the table's metadata has not changed since construction.
+  auto CheckInvariants() const -> void;
+#endif
+
+  ViewImpl view_;
+#ifndef NDEBUG
+  HashCode initial_metadata_hash_ = {};
+  ssize_t start_group_ = 0;
+  ssize_t step_ = GroupSize;
+#endif
+};
+
+// Two-level forward iterator through present hashtable entries.
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+class ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntryRange::Iterator
+    : public EntryRange::IteratorBase {
+ public:
+  // Both the set and map forms satisfy C++20's `std::forward_iterator`. A
+  // map's `reference` is a proxy, which pins its C++17 `iterator_category` to
+  // `input`, but the C++20 concept is unaffected. See `EntryRefT`.
+  using iterator_concept = std::forward_iterator_tag;
+
+  Iterator() = default;
+
+  using EntryRange::IteratorBase::operator++;
+
+  [[clang::always_inline]] auto operator*() const -> EntryRefT {
+    CARBON_DCHECK(present_bits_ != 0, "Dereferencing end iterator!");
+    __builtin_assume(present_bits_ != 0);
+    // `index_ptr` folds scaling the match index by the entry size together
+    // with decoding the index itself, which saves a shift on the portable
+    // byte-encoded code path.
+    return MatchIndex(present_bits_).index_ptr(group_entries())->ref();
+  }
+
+  [[clang::always_inline]] auto operator++() -> Iterator& {
+    CARBON_DCHECK(present_bits_ != 0, "Incrementing end iterator!");
+    __builtin_assume(present_bits_ != 0);
+    present_bits_ &= (present_bits_ - 1);
+    if (LLVM_LIKELY(present_bits_ != 0)) {
+      return *this;
+    }
+    AdvanceToNextPresentGroup();
+    return *this;
+  }
+
+  friend auto operator==(const Iterator& lhs, const Iterator& rhs) -> bool {
+    if (lhs.present_bits_ == 0 || rhs.present_bits_ == 0) {
+      return lhs.present_bits_ == rhs.present_bits_;
+    }
+    // The entry pointer already encodes the base and the group offset, so it
+    // uniquely identifies the group without a separate index.
+    return lhs.group_entries() == rhs.group_entries() &&
+           lhs.present_bits_ == rhs.present_bits_;
+  }
+
+ private:
+  friend class EntryRange;
+
+  using MatchBitsT = typename MetadataGroup::MatchPresentRange::BitsT;
+  using MatchIndex = typename MetadataGroup::MatchIndex;
+
+  // Builds an iterator to the first present entry of `range`, or an iterator
+  // equal to `end()` when the range has no entries to walk. The parameters of
+  // the walk differ between builds, so both are drawn from the range here
+  // rather than passed in.
+  [[clang::always_inline]] explicit Iterator(const EntryRange& range);
+
+  [[clang::always_inline]] auto AdvanceToNextPresentGroup() -> void;
+
+  // The entries of the group the iterator is currently within. Both builds
+  // track the current group, but they encode it differently, so the encoding
+  // is hidden behind this accessor.
+  auto group_entries() const -> EntryT* {
+#ifndef NDEBUG
+    return group_entries_;
+#else
+    return entries_end_ + group_offset_;
+#endif
+  }
+
+#ifndef NDEBUG
+  // Debug builds walk groups in a randomized order and so must retain the
+  // array bases along with the parameters of the walk. The randomized walk
+  // revisits no group but also never reaches the end of the array, so it does
+  // need an explicit count of the groups left to visit.
+  EntryT* group_entries_ = nullptr;
+  const uint8_t* metadata_ = nullptr;
+  EntryT* entries_ = nullptr;
+  ssize_t groups_remaining_ = 0;
+  ssize_t group_index_ = 0;
+  size_t probe_mask_ = 0;
+  ssize_t step_ = GroupSize;
+#else
+  // Release builds walk the groups in order, tracking the position as a
+  // *negative* byte offset from the end of each array that counts up to zero.
+  // Anchoring at the ends rather than the beginnings means the walk needs only
+  // this one induction variable, and reaching zero is the bound.
+  EntryT* entries_end_ = nullptr;
+  const uint8_t* metadata_end_ = nullptr;
+  ssize_t group_offset_ = 0;
+#endif
+  MatchBitsT present_bits_ = 0;
+};
+
 // Implementation helper for defining a read-write base type for a hashtable
 // that type-erases any SSO buffer.
 //
@@ -474,8 +687,8 @@ class BaseImpl {
   using ValueT = InputValueT;
   using KeyContextT = InputKeyContextT;
   using ViewImplT = ViewImpl<KeyT, ValueT, KeyContextT>;
-  using EntryT = typename ViewImplT::EntryT;
-  using MetricsT = typename ViewImplT::MetricsT;
+  using EntryT = ViewImplT::EntryT;
+  using MetricsT = ViewImplT::MetricsT;
 
   BaseImpl(int small_alloc_size, Storage* small_storage)
       : small_alloc_size_(small_alloc_size) {
@@ -495,7 +708,10 @@ class BaseImpl {
   // NOLINTNEXTLINE(google-explicit-constructor): Designed to implicitly decay.
   explicit(false) operator ViewImplT() const { return view_impl(); }
 
-  auto view_impl() const -> ViewImplT { return view_impl_; }
+  auto view_impl() const -> const ViewImplT& { return view_impl_; }
+
+  // Destroys all non-trivially destructible entries in the table.
+  auto DestroyEntries() -> void;
 
   // Looks up the provided key in the hashtable. If found, returns a pointer to
   // that entry and `false`.
@@ -510,7 +726,7 @@ class BaseImpl {
 
   // Grow the table to specific allocation size.
   //
-  // This will grow the the table if necessary for it to have an allocation size
+  // This will grow the table if necessary for it to have an allocation size
   // of `target_alloc_size` which must be a power of two. Note that this will
   // not allow that many keys to be inserted into the hashtable, but a smaller
   // number based on the load factor. If a specific number of insertions need to
@@ -561,7 +777,7 @@ class BaseImpl {
   auto storage() const -> Storage* { return view_impl_.storage_; }
   auto storage() -> Storage*& { return view_impl_.storage_; }
   auto metadata() const -> uint8_t* { return view_impl_.metadata(); }
-  auto entries() const -> EntryT* { return view_impl_.entries(); }
+  auto entries_data() const -> EntryT* { return view_impl_.entries_data(); }
   auto small_alloc_size() const -> ssize_t {
     return static_cast<unsigned>(small_alloc_size_);
   }
@@ -665,6 +881,25 @@ inline auto ComputeSeed() -> uint64_t {
   return reinterpret_cast<uint64_t>(&global_addr_seed);
 }
 
+#ifndef NDEBUG
+// A pool of entropy used to vary the iteration order of hashtables in debug
+// builds. It is seeded from ASLR where available.
+extern std::atomic<HashCode> entropy_hash;
+
+// Returns a pseudo-random value from the entropy pool, advancing the pool.
+//
+// The load and store are separate relaxed operations rather than one atomic
+// read-modify-write so that consuming entropy is just a load, and refreshing
+// the pool doesn't block the iteration that follows. Racing callers can lose an
+// update and draw the same value, which is fine for a debug aid.
+inline auto NextRangeEntropy() -> HashCode {
+  HashCode prev_entropy_hash = entropy_hash.load(std::memory_order_relaxed);
+  entropy_hash.store(Carbon::HashValue(prev_entropy_hash),
+                     std::memory_order_relaxed);
+  return prev_entropy_hash;
+}
+#endif
+
 inline auto ComputeProbeMaskFromSize(ssize_t size) -> size_t {
   CARBON_DCHECK(llvm::isPowerOf2_64(size),
                 "Size must be a power of two for a hashed buffer!");
@@ -748,7 +983,7 @@ auto ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::LookupEntry(
   HashCode hash = key_context.HashKey(lookup_key, ComputeSeed());
   auto [hash_index, tag] = hash.ExtractIndexAndTag<7>();
 
-  EntryT* local_entries = entries();
+  EntryT* local_entries = entries_data();
 
   // Walk through groups of entries using a quadratic probe starting from
   // `hash_index`.
@@ -799,41 +1034,11 @@ auto ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::LookupEntry(
   } while (LLVM_UNLIKELY(true));
 }
 
-// Note that we force inlining here because we expect to be called with lambdas
-// that will in turn be inlined to form the loop body. We don't want function
-// boundaries within the loop for performance, and recognizing the degree of
-// simplification from inlining these callbacks may be difficult to
-// automatically recognize.
-template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
-template <typename EntryCallbackT, typename GroupCallbackT>
-[[clang::always_inline]] auto
-ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::ForEachEntry(
-    EntryCallbackT entry_callback, GroupCallbackT group_callback) const
-    -> void {
-  uint8_t* local_metadata = metadata();
-  EntryT* local_entries = entries();
-
-  ssize_t local_size = alloc_size_;
-  for (ssize_t group_index = 0; group_index < local_size;
-       group_index += GroupSize) {
-    auto g = MetadataGroup::Load(local_metadata, group_index);
-    auto present_matched_range = g.MatchPresent();
-    if (!present_matched_range) {
-      continue;
-    }
-    for (ssize_t byte_index : present_matched_range) {
-      entry_callback(local_entries[group_index + byte_index]);
-    }
-
-    group_callback(&local_metadata[group_index]);
-  }
-}
-
 template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
 auto ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::ComputeMetricsImpl(
     KeyContextT key_context) const -> Metrics {
   uint8_t* local_metadata = metadata();
-  EntryT* local_entries = entries();
+  EntryT* local_entries = entries_data();
   ssize_t local_size = alloc_size_;
 
   Metrics metrics;
@@ -898,6 +1103,147 @@ auto ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::ComputeMetricsImpl(
   return metrics;
 }
 
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+[[clang::always_inline]] auto
+ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntriesImpl() const
+    -> EntryRange {
+  return EntryRange(*this);
+}
+
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+[[clang::always_inline]]
+ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntryRange::Iterator::
+    Iterator(const EntryRange& range) {
+  const ViewImpl& view = range.view_;
+  ssize_t alloc_size = view.alloc_size_;
+
+  // An empty or moved-from table has no groups to load from, and the
+  // default-initialized state left behind already compares equal to `end()`.
+  if (alloc_size == 0 || view.storage_ == nullptr) {
+    return;
+  }
+
+#ifndef NDEBUG
+  entries_ = view.entries_data();
+  metadata_ = view.metadata();
+  // The starting group and stride were drawn when the range was constructed,
+  // so every iterator built from it walks the same order.
+  group_index_ = range.start_group_;
+  group_entries_ = entries_ + group_index_;
+  groups_remaining_ = alloc_size / GroupSize - 1;
+  probe_mask_ = ComputeProbeMaskFromSize(alloc_size);
+  step_ = range.step_;
+
+  auto g = MetadataGroup::Load(metadata_, group_index_);
+#else
+  // The allocation size bounds the metadata array directly, so anchoring at
+  // the ends of the arrays lets the walk run off a single induction variable
+  // without ever dividing by the group size.
+  entries_end_ = view.entries_data() + alloc_size;
+  metadata_end_ = view.metadata() + alloc_size;
+  group_offset_ = -alloc_size;
+
+  auto g = MetadataGroup::Load(metadata_end_, group_offset_);
+#endif
+
+  auto present_range = g.MatchPresent();
+  if (present_range) {
+    present_bits_ = static_cast<MatchBitsT>(present_range);
+  } else {
+    AdvanceToNextPresentGroup();
+  }
+}
+
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+[[clang::always_inline]] auto
+ViewImpl<InputKeyT, InputValueT,
+         InputKeyContextT>::EntryRange::Iterator::AdvanceToNextPresentGroup()
+    -> void {
+#ifndef NDEBUG
+  while (--groups_remaining_ >= 0) {
+    group_index_ = static_cast<ssize_t>(
+        static_cast<size_t>(group_index_ + step_) & probe_mask_);
+    auto g = MetadataGroup::Load(metadata_, group_index_);
+    auto range = g.MatchPresent();
+    if (range) {
+      group_entries_ = entries_ + group_index_;
+      present_bits_ = static_cast<MatchBitsT>(range);
+      return;
+    }
+  }
+#else
+  for (group_offset_ += GroupSize; group_offset_ != 0;
+       group_offset_ += GroupSize) {
+    auto g = MetadataGroup::Load(metadata_end_, group_offset_);
+    auto range = g.MatchPresent();
+    if (range) {
+      present_bits_ = static_cast<MatchBitsT>(range);
+      return;
+    }
+  }
+#endif
+  present_bits_ = 0;
+}
+
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntryRange::EntryRange(
+    ViewImpl view)
+    : view_(view) {
+#ifndef NDEBUG
+  if (view_.alloc_size_ <= 0 || view_.storage_ == nullptr) {
+    return;
+  }
+  initial_metadata_hash_ = Carbon::HashValue(
+      llvm::ArrayRef<uint8_t>(view_.metadata(), view_.alloc_size_));
+
+  // Draw the traversal order once, here, so that `begin()` remains a pure
+  // function of the range and can be called repeatedly. Two separately
+  // constructed ranges still walk the table in different orders.
+  start_group_ = NextRangeEntropy().ExtractIndex() &
+                 ComputeProbeMaskFromSize(view_.alloc_size_);
+
+  // Walk the groups with a stride of an odd number of groups. The group count
+  // is always a power of two, so any odd stride is coprime with it and visits
+  // every group exactly once before repeating. That scrambles the group order
+  // far more thoroughly than a forward or reverse scan, and costs nothing in
+  // the loop itself as the increment already adds a stride and masks.
+  ssize_t num_groups = view_.alloc_size_ / GroupSize;
+  ssize_t stride_groups =
+      (NextRangeEntropy().ExtractIndex() & (num_groups - 1)) | 1;
+  step_ = stride_groups * GroupSize;
+#endif
+}
+
+#ifndef NDEBUG
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+auto ViewImpl<InputKeyT, InputValueT,
+              InputKeyContextT>::EntryRange::CheckInvariants() const -> void {
+  if (view_.alloc_size_ <= 0 || view_.storage_ == nullptr) {
+    return;
+  }
+  HashCode current_hash = Carbon::HashValue(
+      llvm::ArrayRef<uint8_t>(view_.metadata(), view_.alloc_size_));
+  CARBON_CHECK(current_hash == initial_metadata_hash_,
+               "Hashtable mutated during iteration: metadata changed!");
+}
+#endif
+
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+[[clang::always_inline]] auto
+ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntryRange::begin() const
+    -> Iterator {
+  // The traversal order is fixed when the range is constructed, so repeated
+  // calls yield equal iterators as forward ranges require.
+  return Iterator(*this);
+}
+
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+[[clang::always_inline]] auto
+ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::EntryRange::end() const
+    -> Iterator {
+  return Iterator();
+}
+
 // TODO: Evaluate whether it is worth forcing this out-of-line given the
 // reasonable ABI boundary it forms and large volume of code necessary to
 // implement it.
@@ -921,7 +1267,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::InsertImpl(
   ssize_t group_with_deleted_index;
   MetadataGroup::MatchIndex deleted_match = {};
 
-  EntryT* local_entries = entries();
+  EntryT* local_entries = entries_data();
 
   auto return_insert_at_index = [&](ssize_t index) -> std::pair<EntryT*, bool> {
     // We'll need to insert at this index so set the control group byte to the
@@ -1017,7 +1363,7 @@ BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::GrowToAllocSizeImpl(
   bool old_small = is_small();
   Storage* old_storage = storage();
   uint8_t* old_metadata = metadata();
-  EntryT* old_entries = entries();
+  EntryT* old_entries = entries_data();
 
   // Configure for the new size and allocate the new storage.
   alloc_size() = target_alloc_size;
@@ -1093,7 +1439,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::EraseImpl(
   // If we mark the slot as empty, we'll also need to increase the growth
   // budget.
   uint8_t* local_metadata = metadata();
-  EntryT* local_entries = entries();
+  EntryT* local_entries = entries_data();
   ssize_t index = entry - local_entries;
   ssize_t group_index = index & ~GroupMask;
   auto g = MetadataGroup::Load(local_metadata, group_index);
@@ -1114,16 +1460,10 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::EraseImpl(
 
 template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
 auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::ClearImpl() -> void {
-  view_impl_.ForEachEntry(
-      [](EntryT& entry) {
-        if constexpr (!EntryT::IsTriviallyDestructible) {
-          entry.Destroy();
-        }
-      },
-      [](uint8_t* metadata_group) {
-        // Clear the group.
-        std::memset(metadata_group, 0, GroupSize);
-      });
+  DestroyEntries();
+  if (storage() != nullptr) {
+    std::memset(metadata(), 0, alloc_size());
+  }
   growth_budget_ = GrowthThresholdForAllocSize(alloc_size());
 }
 
@@ -1186,10 +1526,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::Destroy() -> void {
   }
 
   // Destroy all the entries.
-  if constexpr (!EntryT::IsTriviallyDestructible) {
-    view_impl_.ForEachEntry([](EntryT& entry) { entry.Destroy(); },
-                            [](auto...) {});
-  }
+  DestroyEntries();
 
   // If small, nothing to deallocate.
   if (is_small()) {
@@ -1201,6 +1538,16 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::Destroy() -> void {
   Deallocate(storage(), alloc_size());
 }
 
+template <typename InputKeyT, typename InputValueT, typename InputKeyContextT>
+auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::DestroyEntries()
+    -> void {
+  if constexpr (!EntryT::IsTriviallyDestructible) {
+    for (typename EntryT::RefT entry : view_impl_.EntriesImpl()) {
+      EntryT::DestroyRef(entry);
+    }
+  }
+}
+
 // Copy all of the slots over from another table that is exactly the same
 // allocation size.
 //
@@ -1224,9 +1571,9 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::CopySlotsFrom(
   // all of the keys. This is especially important as we don't have an easy way
   // to access the key context needed for rehashing here.
   uint8_t* local_metadata = metadata();
-  EntryT* local_entries = entries();
+  EntryT* local_entries = entries_data();
   const uint8_t* local_arg_metadata = arg.metadata();
-  const EntryT* local_arg_entries = arg.entries();
+  const EntryT* local_arg_entries = arg.entries_data();
   memcpy(local_metadata, local_arg_metadata, local_size);
 
   for (ssize_t group_index = 0; group_index < local_size;
@@ -1269,9 +1616,9 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::MoveFrom(
     // themselves. We do this preserving their slots and even tombstones to
     // avoid rehashing.
     uint8_t* local_metadata = this->metadata();
-    EntryT* local_entries = this->entries();
+    EntryT* local_entries = this->entries_data();
     uint8_t* local_arg_metadata = arg.metadata();
-    EntryT* local_arg_entries = arg.entries();
+    EntryT* local_arg_entries = arg.entries_data();
     memcpy(local_metadata, local_arg_metadata, local_size);
     if (EntryT::IsTriviallyRelocatable) {
       memcpy(local_entries, local_arg_entries, local_size * sizeof(EntryT));
@@ -1306,7 +1653,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::InsertIntoEmpty(
     HashCode hash) -> EntryT* {
   auto [hash_index, tag] = hash.ExtractIndexAndTag<7>();
   uint8_t* local_metadata = metadata();
-  EntryT* local_entries = entries();
+  EntryT* local_entries = entries_data();
 
   for (ProbeSequence s(hash_index, alloc_size());; s.Next()) {
     ssize_t group_index = s.index();
@@ -1392,7 +1739,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::GrowToNextAllocSize(
   bool old_small = is_small();
   Storage* old_storage = storage();
   uint8_t* old_metadata = metadata();
-  EntryT* old_entries = entries();
+  EntryT* old_entries = entries_data();
 
 #ifndef NDEBUG
   // Count how many of the old table slots will end up being empty after we grow
@@ -1417,7 +1764,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::GrowToNextAllocSize(
 
   // Now extract the new components of the table.
   uint8_t* new_metadata = metadata();
-  EntryT* new_entries = entries();
+  EntryT* new_entries = entries_data();
 
   // Walk the metadata groups, clearing deleted to empty, duplicating the
   // metadata for the low and high halves, and updating it based on where each
@@ -1596,10 +1943,7 @@ auto TableImpl<InputBaseT, SmallSize>::operator=(const TableImpl& arg)
       return *this;
     }
     CARBON_DCHECK(arg.storage() != this->storage());
-    if constexpr (!EntryT::IsTriviallyDestructible) {
-      this->view_impl_.ForEachEntry([](EntryT& entry) { entry.Destroy(); },
-                                    [](auto...) {});
-    }
+    this->DestroyEntries();
   } else {
     // The sizes don't match so destroy everything and re-setup the table
     // storage.

common/raw_hashtable_test_helpers.h

@@ -70,15 +70,9 @@ struct MoveOnlyTestData : Printable<TestData> {
   }
   auto Print(llvm::raw_ostream& out) const -> void { out << value; }
 
-  friend auto operator==(const MoveOnlyTestData& lhs,
-                         const MoveOnlyTestData& rhs) -> bool {
-    return lhs.value == rhs.value;
-  }
-
   friend auto operator<=>(const MoveOnlyTestData& lhs,
-                          const MoveOnlyTestData& rhs) -> std::strong_ordering {
-    return lhs.value <=> rhs.value;
-  }
+                          const MoveOnlyTestData& rhs)
+      -> std::strong_ordering = default;
 
   friend auto CarbonHashValue(const MoveOnlyTestData& data, uint64_t seed)
       -> HashCode {

common/raw_string_ostream.h

@@ -39,6 +39,10 @@ class RawStringOstream : public llvm::raw_pwrite_stream {
   auto empty() -> bool { return str_.empty(); }
   auto size() -> size_t { return str_.size(); }
 
+  auto reserveExtraSpace(uint64_t extra_size) -> void override {
+    str_.reserve(str_.size() + extra_size);
+  }
+
  private:
   auto current_pos() const -> uint64_t override { return str_.size(); }
 
@@ -51,10 +55,6 @@ class RawStringOstream : public llvm::raw_pwrite_stream {
     str_.append(ptr, size);
   }
 
-  auto reserveExtraSpace(uint64_t extra_size) -> void override {
-    str_.reserve(str_.size() + extra_size);
-  }
-
   // The actual buffer.
   std::string str_;
 };

common/set.h

@@ -7,6 +7,7 @@
 
 #include <concepts>
 #include <type_traits>
+#include <utility>
 
 #include "common/check.h"
 #include "common/hashtable_key_context.h"
@@ -56,9 +57,13 @@ class SetView : RawHashtable::ViewImpl<InputKeyT, void, InputKeyContextT> {
   using ImplT = RawHashtable::ViewImpl<InputKeyT, void, InputKeyContextT>;
 
  public:
-  using KeyT = typename ImplT::KeyT;
-  using KeyContextT = typename ImplT::KeyContextT;
-  using MetricsT = typename ImplT::MetricsT;
+  using KeyT = ImplT::KeyT;
+  using KeyContextT = ImplT::KeyContextT;
+  using MetricsT = ImplT::MetricsT;
+
+  // A range over the keys of the set. Bound to the lifetime of the viewed set,
+  // and invalidated by mutating it.
+  using Range = ImplT::EntryRange;
 
   // This type represents the result of lookup operations. It encodes whether
   // the lookup was a success as well as accessors for the key.
@@ -91,10 +96,8 @@ class SetView : RawHashtable::ViewImpl<InputKeyT, void, InputKeyContextT> {
   auto Lookup(LookupKeyT lookup_key,
               KeyContextT key_context = KeyContextT()) const -> LookupResult;
 
-  // Run the provided callback for every key in the set.
-  template <typename CallbackT>
-  auto ForEach(CallbackT callback) const -> void
-    requires(std::invocable<CallbackT, KeyT&>);
+  // Returns a range for iterating over all keys in the set.
+  auto entries() const -> Range;
 
   // This routine is relatively inefficient and only intended for use in
   // benchmarking or logging of performance anomalies. The specific metrics
@@ -110,7 +113,7 @@ class SetView : RawHashtable::ViewImpl<InputKeyT, void, InputKeyContextT> {
   friend class SetBase<KeyT, KeyContextT>;
   friend class SetView<const KeyT, KeyContextT>;
 
-  using EntryT = typename ImplT::EntryT;
+  using EntryT = ImplT::EntryT;
 
   SetView() = default;
   explicit(false) SetView(ImplT base) : ImplT(base) {}
@@ -131,18 +134,19 @@ class SetView : RawHashtable::ViewImpl<InputKeyT, void, InputKeyContextT> {
 // A pointer or reference to this type is the preferred way to pass a mutable
 // handle to a `Set` type across API boundaries as it avoids encoding specific
 // SSO sizing information while providing a near-complete mutable API.
-template <typename InputKeyT, typename InputKeyContextT>
+template <typename InputKeyT, typename InputKeyContextT = DefaultKeyContext>
 class SetBase
     : protected RawHashtable::BaseImpl<InputKeyT, void, InputKeyContextT> {
  protected:
   using ImplT = RawHashtable::BaseImpl<InputKeyT, void, InputKeyContextT>;
 
  public:
-  using KeyT = typename ImplT::KeyT;
-  using KeyContextT = typename ImplT::KeyContextT;
+  using KeyT = ImplT::KeyT;
+  using KeyContextT = ImplT::KeyContextT;
   using ViewT = SetView<KeyT, KeyContextT>;
-  using LookupResult = typename ViewT::LookupResult;
-  using MetricsT = typename ImplT::MetricsT;
+  using LookupResult = ViewT::LookupResult;
+  using MetricsT = ImplT::MetricsT;
+  using Range = ViewT::Range;
 
   // The result type for insertion operations both indicates whether an insert
   // was needed (as opposed to the key already being in the set), and provides
@@ -190,12 +194,12 @@ class SetBase
   }
 
   // Convenience forwarder to the view type.
-  template <typename CallbackT>
-  auto ForEach(CallbackT callback) const -> void
-    requires(std::invocable<CallbackT, KeyT&>)
-  {
-    return ViewT(*this).ForEach(callback);
-  }
+  auto entries() const& -> Range { return ViewT(*this).entries(); }
+  // Deleted on rvalues: the range refers to storage owned by this table, so a
+  // range built from a temporary set would dangle. Both qualifiers are needed
+  // as `&&` alone would leave a const rvalue binding to the `const&` overload.
+  auto entries() && = delete;
+  auto entries() const&& = delete;
 
   // Convenience forwarder to the view type.
   auto ComputeMetrics(KeyContextT key_context = KeyContextT()) const
@@ -211,10 +215,10 @@ class SetBase
   auto Insert(LookupKeyT lookup_key, KeyContextT key_context = KeyContextT())
       -> InsertResult;
 
-  // Insert a key into the map and call the provided callback if necessary to
-  // produce a new key when no existing value is found.
+  // Insert a key into the set and call the provided callback if necessary to
+  // produce a new key when no existing key is found.
   //
-  // Example: `m.Insert(key_equivalent, [] { return real_key; });`
+  // Example: `s.Insert(key_equivalent, [] { return real_key; });`
   //
   // The point of this function is when the lookup key is _different_from the
   // stored key. However, we don't restrict it in case that blocks generic
@@ -299,7 +303,7 @@ class Set : public RawHashtable::TableImpl<SetBase<InputKeyT, InputKeyContextT>,
   using ImplT = RawHashtable::TableImpl<BaseT, SmallSize>;
 
  public:
-  using KeyT = typename BaseT::KeyT;
+  using KeyT = BaseT::KeyT;
 
   Set() = default;
   Set(const Set& arg) = default;
@@ -333,13 +337,8 @@ auto SetView<InputKeyT, InputKeyContextT>::Lookup(LookupKeyT lookup_key,
 }
 
 template <typename InputKeyT, typename InputKeyContextT>
-template <typename CallbackT>
-auto SetView<InputKeyT, InputKeyContextT>::ForEach(CallbackT callback) const
-    -> void
-  requires(std::invocable<CallbackT, KeyT&>)
-{
-  this->ForEachEntry([callback](EntryT& entry) { callback(entry.key()); },
-                     [](auto...) {});
+auto SetView<InputKeyT, InputKeyContextT>::entries() const -> Range {
+  return this->ImplT::EntriesImpl();
 }
 
 template <typename InputKeyT, typename InputKeyContextT>

common/set_benchmark.cpp

@@ -35,9 +35,10 @@ static constexpr bool IsCarbonSet = IsCarbonSetImpl<SetT>::value;
 // support different APIs. The primary template assumes a roughly
 // `std::unordered_set` API design, and types with a different API design are
 // supported through specializations.
-template <typename SetT>
+template <typename InSetT>
 struct SetWrapperImpl {
-  using KeyT = typename SetT::key_type;
+  using SetT = InSetT;
+  using KeyT = SetT::key_type;
 
   SetT s;
 
@@ -58,6 +59,17 @@ struct SetWrapperImpl {
   }
 
   auto BenchErase(KeyT k) -> bool { return s.erase(k) != 0; }
+
+  // Visits every key in the set, calling `cb` with each one. Each set type is
+  // expected to traverse using whatever API it provides for this, so that the
+  // benchmark measures iterating the set rather than any specific iteration
+  // API.
+  template <typename CallbackT>
+  auto BenchIterate(CallbackT cb) -> void {
+    for (const auto& k : s) {
+      cb(k);
+    }
+  }
 };
 
 // Explicit (partial) specialization for the Carbon map type that uses its
@@ -85,6 +97,13 @@ struct SetWrapperImpl<Set<KT, MinSmallSize>> {
   }
 
   auto BenchErase(KeyT k) -> bool { return s.Erase(k); }
+
+  template <typename CallbackT>
+  auto BenchIterate(CallbackT cb) -> void {
+    for (const auto& k : s.entries()) {
+      cb(k);
+    }
+  }
 };
 
 // Provide a way to override the Carbon Set specific benchmark runs with another
@@ -123,6 +142,17 @@ using SetWrapper =
     SetWrapperOverride<SetT, SetOverride::CARBON_SET_BENCH_OVERRIDE>;
 #endif
 
+// Reports extra statistics about the table, when it is in fact a Carbon table.
+// Note that this has to inspect the *wrapped* type in order to work correctly
+// when the Carbon benchmarks are overridden with another implementation.
+template <typename SetT>
+auto ReportMetrics(const SetWrapper<SetT>& s_wrapper, benchmark::State& state)
+    -> void {
+  if constexpr (IsCarbonSet<typename SetWrapper<SetT>::SetT>) {
+    ReportTableMetrics(s_wrapper.s, state);
+  }
+}
+
 // NOLINTBEGIN(bugprone-macro-parentheses): Parentheses are incorrect here.
 #define MAP_BENCHMARK_ONE_OP_SIZE(NAME, APPLY, KT)        \
   BENCHMARK(NAME<Set<KT>>)->Apply(APPLY);                 \
@@ -158,7 +188,7 @@ using SetWrapper =
 template <typename SetT>
 static void BM_SetContainsHitPtr(benchmark::State& state) {
   using SetWrapperT = SetWrapper<SetT>;
-  using KT = typename SetWrapperT::KeyT;
+  using KT = SetWrapperT::KeyT;
   SetWrapperT s;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -190,7 +220,7 @@ MAP_BENCHMARK_ONE_OP(BM_SetContainsHitPtr, HitArgs);
 template <typename SetT>
 static void BM_SetContainsMissPtr(benchmark::State& state) {
   using SetWrapperT = SetWrapper<SetT>;
-  using KT = typename SetWrapperT::KeyT;
+  using KT = SetWrapperT::KeyT;
   SetWrapperT s;
   auto [keys, lookup_keys] = GetKeysAndMissKeys<KT>(state.range(0));
   for (auto k : keys) {
@@ -225,7 +255,7 @@ MAP_BENCHMARK_ONE_OP(BM_SetContainsMissPtr, SizeArgs);
 template <typename SetT>
 static void BM_SetLookupHitPtr(benchmark::State& state) {
   using SetWrapperT = SetWrapper<SetT>;
-  using KT = typename SetWrapperT::KeyT;
+  using KT = SetWrapperT::KeyT;
   SetWrapperT s;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -265,7 +295,7 @@ MAP_BENCHMARK_ONE_OP(BM_SetLookupHitPtr, HitArgs);
 template <typename SetT>
 static void BM_SetEraseInsertHitPtr(benchmark::State& state) {
   using SetWrapperT = SetWrapper<SetT>;
-  using KT = typename SetWrapperT::KeyT;
+  using KT = SetWrapperT::KeyT;
   SetWrapperT s;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), state.range(1));
@@ -324,7 +354,7 @@ MAP_BENCHMARK_ONE_OP(BM_SetEraseInsertHitPtr, HitArgs);
 template <typename SetT>
 static void BM_SetInsertSeq(benchmark::State& state) {
   using SetWrapperT = SetWrapper<SetT>;
-  using KT = typename SetWrapperT::KeyT;
+  using KT = SetWrapperT::KeyT;
   constexpr ssize_t LookupKeysSize = 1 << 8;
   auto [keys, lookup_keys] =
       GetKeysAndHitKeys<KT>(state.range(0), LookupKeysSize);
@@ -375,5 +405,44 @@ static void BM_SetInsertSeq(benchmark::State& state) {
 }
 MAP_BENCHMARK_OP_SEQ(BM_SetInsertSeq);
 
+// Benchmark visiting every key in a set.
+//
+// Unlike the lookup benchmarks, this walks the table's storage from end to end
+// rather than probing it, so it is largely a measure of how densely keys are
+// packed and how cheaply empty slots can be skipped. There is no dependency
+// between the keys visited, and so this is a throughput measurement.
+//
+// Each batch is a single complete traversal of the set, with the batch size set
+// to the number of keys so that the reported time is the per-key cost.
+template <typename SetT>
+static void BM_SetIterate(benchmark::State& state) {
+  using SetWrapperT = SetWrapper<SetT>;
+  using KT = typename SetWrapperT::KeyT;
+  SetWrapperT s;
+  auto [keys, _] = GetKeysAndMissKeys<KT>(state.range(0));
+  for (auto k : keys) {
+    bool inserted = s.BenchInsert(k);
+    CARBON_DCHECK(inserted, "Must be a successful insert!");
+  }
+
+  while (state.KeepRunningBatch(keys.size())) {
+    ssize_t sum = 0;
+    s.BenchIterate([&sum](const KT& k) {
+      // Consume the key so that neither the traversal nor the loads out of the
+      // entries can be optimized away.
+      sum += ValueToBool(k);
+    });
+    benchmark::DoNotOptimize(sum);
+  }
+
+  // The time is already per-key, so an iteration-invariant rate of one gives
+  // the throughput of keys visited.
+  state.counters["KeyRate"] =
+      benchmark::Counter(1, benchmark::Counter::kIsIterationInvariantRate);
+
+  ReportMetrics(s, state);
+}
+MAP_BENCHMARK_ONE_OP(BM_SetIterate, SizeArgs);
+
 }  // namespace
 }  // namespace Carbon

common/set_test.cpp

@@ -7,7 +7,12 @@
 #include <gmock/gmock.h>
 #include <gtest/gtest.h>
 
+#include <concepts>
 #include <initializer_list>
+#include <iterator>
+#include <ranges>
+#include <set>
+#include <string>
 #include <type_traits>
 #include <vector>
 
@@ -19,14 +24,17 @@ namespace {
 using RawHashtable::IndexKeyContext;
 using RawHashtable::MoveOnlyTestData;
 using RawHashtable::TestData;
+using ::testing::UnorderedElementsAre;
 using ::testing::UnorderedElementsAreArray;
 
 template <typename SetT, typename MatcherRangeT>
 auto ExpectSetElementsAre(SetT&& s, MatcherRangeT element_matchers) -> void {
   // Collect the elements into a container.
-  using KeyT = typename std::remove_reference<SetT>::type::KeyT;
+  using KeyT = std::remove_reference<SetT>::type::KeyT;
   std::vector<std::reference_wrapper<KeyT>> entries;
-  s.ForEach([&entries](KeyT& k) { entries.push_back(std::ref(k)); });
+  for (auto& k : s.entries()) {
+    entries.push_back(std::ref(k));
+  }
 
   // Use the GoogleMock unordered container matcher to validate and show errors
   // on wrong elements.
@@ -176,6 +184,8 @@ TYPED_TEST(SetTest, Move) {
 
   SetT other_s1 = std::move(s);
   ExpectSetElementsAre(other_s1, MakeElements(llvm::seq(1, 24)));
+  // A moved-from set has a size but no storage, and must iterate as empty.
+  EXPECT_EQ(s.entries().begin(), s.entries().end());
 
   // Add some more elements.
   for (int i : llvm::seq(24, 32)) {
@@ -432,5 +442,175 @@ TEST(SetContextTest, Basic) {
   ExpectSetElementsAre(s, MakeElements(llvm::seq(1, 512)));
 }
 
+TYPED_TEST(SetTest, Range) {
+  using SetT = TypeParam;
+  using Range = decltype(std::declval<const SetT&>().entries());
+  using Iter = typename Range::Iterator;
+
+  static_assert(std::forward_iterator<Iter>);
+  static_assert(std::same_as<decltype(std::declval<Range>().begin()), Iter>);
+  static_assert(std::same_as<decltype(std::declval<Range>().end()), Iter>);
+  static_assert(std::ranges::forward_range<Range>);
+  static_assert(std::ranges::common_range<Range>);
+
+  SetT s;
+  EXPECT_EQ(s.entries().begin(), s.entries().end());
+  for (const auto& k : s.entries()) {
+    static_cast<void>(k);
+    FAIL() << "Empty set range should have no elements";
+  }
+
+  for (int i = 1; i <= 5; ++i) {
+    s.Insert(i);
+  }
+
+  // Range-for traversal by const ref.
+  int count = 0;
+  for (const auto& k : s.entries()) {
+    EXPECT_GE(k, 1);
+    EXPECT_LE(k, 5);
+    ++count;
+  }
+  EXPECT_EQ(count, 5);
+
+  // Direct GMock container matching.
+  EXPECT_THAT(s.entries(), UnorderedElementsAre(1, 2, 3, 4, 5));
+
+  // Const view range iteration.
+  using KeyT = typename SetT::KeyT;
+  using KeyContextT = typename SetT::KeyContextT;
+  SetView<const KeyT, KeyContextT> cv = s;
+  int cv_count = 0;
+  for (const auto& k : cv.entries()) {
+    static_assert(std::is_const_v<std::remove_reference_t<decltype(k)>>);
+    EXPECT_GE(k, 1);
+    EXPECT_LE(k, 5);
+    ++cv_count;
+  }
+  EXPECT_EQ(cv_count, 5);
+  EXPECT_THAT(cv.entries(), UnorderedElementsAre(1, 2, 3, 4, 5));
+
+  // Explicit iterator traversal, dereference, and post-increment.
+  auto r = s.entries();
+  int iter_count = 0;
+  for (auto it = r.begin(); it != r.end(); ++it) {
+    EXPECT_NE(*it, 0);
+    ++iter_count;
+  }
+  EXPECT_EQ(iter_count, 5);
+
+  auto it = r.begin();
+  auto prev = it++;
+  EXPECT_NE(it, prev);
+}
+
+TYPED_TEST(MoveOnlySetTest, Range) {
+  TypeParam s;
+  s.Insert(1);
+  s.Insert(2);
+
+  int count = 0;
+  for (const auto& k : s.entries()) {
+    EXPECT_GT(k.value, 0);
+    ++count;
+  }
+  EXPECT_EQ(count, 2);
+}
+
+#ifndef NDEBUG
+TEST(SetDeathTest, MutateDuringIterationFails) {
+  EXPECT_DEATH(([] {
+                 Set<int> s;
+                 s.Insert(1);
+                 auto range = s.entries();
+                 s.Insert(2);
+               }()),
+               "Hashtable mutated during iteration");
+}
+#endif
+
+// A range outlives the *view* it was built from: views don't own storage, and
+// the range copies the view rather than pointing at it.
+TEST(SetTest, RangeOutlivesTemporaryView) {
+  Set<int> s;
+  s.Insert(1);
+
+  auto make_view = [&s]() -> SetView<int> { return s; };
+  auto range = make_view().entries();
+  EXPECT_THAT(range, UnorderedElementsAre(1));
+}
+
+#ifdef NDEBUG
+// Release iteration state is two end pointers, a group offset, and the
+// present-bit mask; it needs to stay small enough to live in registers across
+// the loop. Debug builds add the randomized walk and mutation-check state.
+static_assert(sizeof(Set<int>::Range::Iterator) <= 4 * sizeof(void*));
+#endif
+
+// Forward ranges guarantee multi-pass: `begin()` must be a pure function of the
+// range. Debug builds draw their traversal entropy when the range is
+// constructed rather than in `begin()` precisely so that repeated calls start
+// from the same group.
+TEST(SetTest, RangeIsMultiPass) {
+  Set<int, 16> s;
+  for (int i = 1; i <= 64; ++i) {
+    s.Insert(i);
+  }
+
+  auto range = s.entries();
+  EXPECT_EQ(range.begin(), range.begin());
+
+  // Two passes over the same range must agree on both the keys visited and the
+  // order they're visited in.
+  std::vector<int> first;
+  for (int k : range) {
+    first.push_back(k);
+  }
+  std::vector<int> second;
+  for (int k : range) {
+    second.push_back(k);
+  }
+  EXPECT_EQ(first, second);
+  EXPECT_EQ(static_cast<ssize_t>(first.size()), 64);
+}
+
+// Whatever order a range picks, it has to be a genuine permutation of the
+// table. Debug builds additionally vary that order between ranges over the same
+// table so that callers can't come to depend on it.
+TEST(SetTest, TraversalOrderIsAVaryingPermutation) {
+  Set<int, 64> s;
+  std::vector<int> inserted;
+  // Enough keys to populate every group of the small storage.
+  for (int i = 0; i < 36; ++i) {
+    int key = i * 17 + 7;
+    EXPECT_TRUE(s.Insert(key).is_inserted());
+    inserted.push_back(key);
+  }
+
+  std::set<std::vector<int>> distinct_orders;
+  for (int i = 0; i < 64; ++i) {
+    std::vector<int> visited;
+    for (int k : s.entries()) {
+      visited.push_back(k);
+    }
+    // A walk that skipped a group would drop keys and one that revisited a
+    // group would duplicate them, so comparing as a multiset covers both. This
+    // is what makes an odd group stride a valid traversal.
+    EXPECT_THAT(visited, UnorderedElementsAreArray(inserted));
+    distinct_orders.insert(visited);
+  }
+
+#ifndef NDEBUG
+  // Debug builds randomize both the starting group and the stride, so across
+  // this many ranges we should see more than the two orders (pure forward and
+  // pure reverse) that a simple direction flip would produce.
+  EXPECT_GT(distinct_orders.size(), 2)
+      << "Debug traversal order does not appear to be randomized.";
+#else
+  // Release builds always scan the groups in order.
+  EXPECT_EQ(distinct_orders.size(), 1);
+#endif
+}
+
 }  // namespace
 }  // namespace Carbon

common/struct_reflection.h

@@ -55,13 +55,7 @@ struct AnyField {
 
 // Detector for whether we can list-initialize T from the given list of fields.
 template <typename T, typename... Fields>
-constexpr auto CanListInitialize(decltype(T{Fields()...})* /*unused*/) -> bool {
-  return true;
-}
-template <typename T, typename... Fields>
-constexpr auto CanListInitialize(...) -> bool {
-  return false;
-}
+concept CanListInitialize = requires { T{Fields()...}; };
 
 #pragma clang diagnostic pop
 
@@ -72,7 +66,7 @@ constexpr auto CanListInitialize(...) -> bool {
 // 2) Add more AnyField<T>s until we can't initialize any more.
 template <typename T, bool AnyWorkedSoFar = false, typename... Fields>
 constexpr auto CountFields() -> int {
-  if constexpr (CanListInitialize<T, Fields...>(nullptr)) {
+  if constexpr (CanListInitialize<T, Fields...>) {
     return CountFields<T, true, Fields..., AnyField<T>>();
   } else if constexpr (AnyWorkedSoFar) {
     constexpr int NumFields = sizeof...(Fields) - 1;

common/struct_reflection_test.cpp

@@ -53,17 +53,17 @@ TEST(StructReflectionTest, CanListInitialize) {
   {
     using Type = OneField;
     using Field = Internal::AnyField<Type>;
-    static_assert(Internal::CanListInitialize<Type>(nullptr));
-    static_assert(Internal::CanListInitialize<Type, Field>(nullptr));
-    static_assert(!Internal::CanListInitialize<Type, Field, Field>(0));
+    static_assert(Internal::CanListInitialize<Type>);
+    static_assert(Internal::CanListInitialize<Type, Field>);
+    static_assert(!Internal::CanListInitialize<Type, Field, Field>);
   }
 
   {
     using Type = OneFieldNoDefaultConstructor;
     using Field = Internal::AnyField<Type>;
-    static_assert(!Internal::CanListInitialize<Type>(0));
-    static_assert(Internal::CanListInitialize<Type, Field>(nullptr));
-    static_assert(!Internal::CanListInitialize<Type, Field, Field>(0));
+    static_assert(!Internal::CanListInitialize<Type>);
+    static_assert(Internal::CanListInitialize<Type, Field>);
+    static_assert(!Internal::CanListInitialize<Type, Field, Field>);
   }
 }
 

common/terminal/BUILD

@@ -0,0 +1,195 @@
+# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+# Exceptions. See /LICENSE for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+# Terminal rendering: what the attached terminal can do, and how to draw styled
+# text for it. Used for diagnostic rendering and for command line output.
+
+load("@rules_shell//shell:sh_test.bzl", "sh_test")
+load("//bazel/cc_rules:defs.bzl", "cc_binary", "cc_library", "cc_test")
+
+package(default_visibility = ["//visibility:public"])
+
+cc_library(
+    name = "output_buffer_ref",
+    hdrs = ["output_buffer_ref.h"],
+    deps = ["@llvm-project//llvm:Support"],
+)
+
+cc_test(
+    name = "output_buffer_ref_test",
+    size = "small",
+    srcs = ["output_buffer_ref_test.cpp"],
+    deps = [
+        ":output_buffer_ref",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_library(
+    name = "color",
+    srcs = ["color.cpp"],
+    hdrs = ["color.h"],
+    deps = [
+        ":output_buffer_ref",
+        "//common:check",
+        "//common:ostream",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "color_test",
+    size = "small",
+    srcs = ["color_test.cpp"],
+    deps = [
+        ":color",
+        "//common:ostream",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_library(
+    name = "style",
+    srcs = ["style.cpp"],
+    hdrs = ["style.h"],
+    deps = [
+        ":color",
+        ":output_buffer_ref",
+        "//common:check",
+        "//common:ostream",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "style_test",
+    size = "small",
+    srcs = ["style_test.cpp"],
+    deps = [
+        ":style",
+        "//common:ostream",
+        "//common:raw_string_ostream",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_library(
+    name = "capabilities",
+    srcs = ["capabilities.cpp"],
+    hdrs = ["capabilities.h"],
+    deps = [
+        ":color",
+        "//common:filesystem",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "capabilities_test",
+    size = "small",
+    srcs = ["capabilities_test.cpp"],
+    deps = [
+        ":capabilities",
+        "//common:filesystem",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+    ],
+)
+
+cc_library(
+    name = "metrics",
+    srcs = ["metrics.cpp"],
+    hdrs = ["metrics.h"],
+    deps = [
+        ":capabilities",
+        "//common:check",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "metrics_test",
+    size = "small",
+    srcs = ["metrics_test.cpp"],
+    deps = [
+        ":metrics",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_library(
+    name = "buffer",
+    srcs = ["buffer.cpp"],
+    hdrs = ["buffer.h"],
+    deps = [
+        ":capabilities",
+        ":color",
+        ":metrics",
+        ":output_buffer_ref",
+        ":style",
+        "//common:check",
+        "//common:filesystem",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "buffer_test",
+    size = "small",
+    srcs = ["buffer_test.cpp"],
+    deps = [
+        ":buffer",
+        ":metrics",
+        "//common:filesystem",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "pressure_test",
+    size = "small",
+    srcs = ["pressure_test.cpp"],
+    deps = [
+        ":buffer",
+        ":capabilities",
+        ":metrics",
+        ":style",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_binary(
+    name = "terminal_benchmark",
+    testonly = 1,
+    srcs = ["terminal_benchmark.cpp"],
+    deps = [
+        ":buffer",
+        ":capabilities",
+        ":color",
+        ":style",
+        "//testing/base:benchmark_main",
+        "@abseil-cpp//absl/random",
+        "@google_benchmark//:benchmark",
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+sh_test(
+    name = "terminal_benchmark_test",
+    size = "small",
+    srcs = [":terminal_benchmark"],
+    args = ["--benchmark_dry_run"],
+)

common/terminal/buffer.cpp

@@ -0,0 +1,580 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/buffer.h"
+
+#include <algorithm>
+#include <array>
+#include <cstdint>
+#include <utility>
+
+#include "common/check.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/Sequence.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/ConvertUTF.h"
+#include "llvm/Support/Unicode.h"
+
+namespace Carbon::Terminal {
+
+// The most bytes of combining marks kept on one cell. Text stacking more than
+// this is either adversarial or already illegible, and keeping all of it would
+// let a single column of output carry unbounded bytes.
+static constexpr size_t MaxCombiningBytes = 32;
+
+// Glyphs for every combination of line directions, indexed by the direction
+// bits.
+static constexpr std::array<char32_t, 16> Utf8LineGlyphs = {
+    U'·',  // (none): a line between one center and itself, which is a point
+    U'╴',  // left
+    U'╶',  // right
+    U'─',  // left, right
+    U'╵',  // up
+    U'╯',  // left, up
+    U'╰',  // right, up
+    U'┴',  // left, right, up
+    U'╷',  // down
+    U'╮',  // left, down
+    U'╭',  // right, down
+    U'┬',  // left, right, down
+    U'│',  // up, down
+    U'┤',  // left, up, down
+    U'├',  // right, up, down
+    U'┼',  // left, right, up, down
+};
+
+// The ASCII stand-ins. Each keeps the axis its line runs through, which leaves
+// `+` meaning a crossing and nothing else:
+//
+// - Running through horizontally is `-`, vertically `|`, and both ways `+`.
+// - A tee keeps its through-stroke and leaves the branch to what is drawn
+//   beside it: the dashes either side of a `|` are what `├` and `┤` reach, and
+//   the line under a `-` is what a `┬` reaches. Drawing a tee as `+` reads as
+//   the crossing it is not.
+// - A corner is `.` where its line leaves downward and `'` where it arrives
+//   from above, which is where those characters sit in their cells.
+// - A point, a line between one center and itself, is `.`.
+//
+// What a diagnostic draws is then still told apart: the rule closing a snippet
+// from the one separating two, and the anchor opening a diagnostic from the one
+// carrying it on.
+static constexpr std::array<char32_t, 16> AsciiLineGlyphs = {
+    U'.',   // (none): a point
+    U'-',   // left
+    U'-',   // right
+    U'-',   // left, right
+    U'|',   // up
+    U'\'',  // left, up
+    U'\'',  // right, up
+    U'-',   // left, right, up
+    U'|',   // down
+    U'.',   // left, down
+    U'.',   // right, down
+    U'-',   // left, right, down
+    U'|',   // up, down
+    U'|',   // left, up, down
+    U'|',   // right, up, down
+    U'+',   // left, right, up, down
+};
+
+// Returns the next tab stop after `x` on a line whose stops are `tab_width`
+// columns apart counting from `origin`, which `x` must not be left of.
+static auto NextTabStop(int x, int origin, int tab_width) -> int {
+  CARBON_DCHECK(x >= origin, "Column {0} is left of the origin {1}.", x,
+                origin);
+  return origin + ((x - origin) / tab_width + 1) * tab_width;
+}
+
+Buffer::Buffer(int columns, Charset charset, int tab_width)
+    : columns_(columns),
+      width_(columns),
+      tab_width_(tab_width),
+      metrics_(charset) {
+  CARBON_CHECK(columns > 0 && columns <= MaxColumns,
+               "Buffer width must be in [1, {0}], but was {1}.", MaxColumns,
+               columns);
+  CARBON_CHECK(tab_width > 0 && tab_width <= MaxTabWidth,
+               "Tab width must be in [1, {0}], but was {1}.", MaxTabWidth,
+               tab_width);
+}
+
+auto Buffer::height() const -> int {
+  return static_cast<int>(cells_.size()) / width_;
+}
+
+auto Buffer::EnsureRow(int y) -> void {
+  CARBON_CHECK(y >= 0 && y < MaxRows, "Row {0} is outside [0, {1}).", y,
+               MaxRows);
+  if (y < height()) {
+    return;
+  }
+  // Rows are added at the end and nothing already in the grid moves, so this
+  // asks for exactly the rows wanted and lets the vector amortize the growing.
+  cells_.resize(static_cast<size_t>(y + 1) * width_);
+}
+
+auto Buffer::EnsureColumn(int x) -> void {
+  CARBON_CHECK(x >= 0 && x < MaxColumns, "Column {0} is outside [0, {1}).", x,
+               MaxColumns);
+  if (x < width_) {
+    return;
+  }
+  // Widening moves every row, so it grows by halves rather than to exactly what
+  // was asked: a row drawn one code point at a time would otherwise copy the
+  // whole grid on every one of them. Growth stops at the bound, which is what
+  // holds the product of the two dimensions inside what a cell index can
+  // represent.
+  int width = std::min(std::max(x + 1, width_ + width_ / 2), MaxColumns);
+
+  int rows = height();
+  llvm::SmallVector<Cell, 0> new_cells(static_cast<size_t>(rows) * width);
+  for (int y : llvm::seq(rows)) {
+    llvm::copy(
+        llvm::ArrayRef(cells_).slice(static_cast<size_t>(y) * width_, width_),
+        new_cells.begin() + static_cast<size_t>(y) * width);
+  }
+  cells_ = std::move(new_cells);
+
+  // A mark's key is a cell index, which depends on the width, so each is
+  // recomputed for the new one.
+  llvm::DenseMap<int, std::string> new_combining_marks;
+  new_combining_marks.reserve(combining_marks_.size());
+  for (auto& [index, marks] : combining_marks_) {
+    new_combining_marks.insert(
+        {index / width_ * width + index % width_, std::move(marks)});
+  }
+  combining_marks_ = std::move(new_combining_marks);
+
+  width_ = width;
+}
+
+auto Buffer::ClearCells(int x, int y, int width) -> void {
+  CARBON_CHECK(
+      x >= 0 && width >= 0 && x + width <= width_ && y >= 0 && y < height(),
+      "Clearing [{0}, {1}) of row {2} reaches outside the {3}x{4} cells the "
+      "buffer holds.",
+      x, x + width, y, width_, height());
+
+  // A cleared range must not leave half of a double-width character behind, so
+  // it extends over either half that crosses its edges.
+  int begin = x;
+  if (begin > 0 && CellAt(begin, y).is_continuation) {
+    --begin;
+  }
+  int end = x + width;
+  if (end < width_ && CellAt(end, y).is_continuation) {
+    ++end;
+  }
+
+  for (int i = begin; i < end; ++i) {
+    CellAt(i, y) = Cell();
+    combining_marks_.erase(CellIndex(i, y));
+  }
+}
+
+auto Buffer::AttachCombiningMark(int x, int y, char32_t code_point) -> void {
+  // A mark has nowhere to go when no cell precedes it, so it is dropped.
+  if (x <= 0 || x > width_ || y < 0 || y >= height()) {
+    return;
+  }
+  // The left half of a double-width character is never itself a continuation,
+  // so stepping back from one always lands on a real character.
+  int base = x - 1;
+  if (CellAt(base, y).is_continuation) {
+    --base;
+  }
+  CARBON_CHECK(base >= 0, "A continuation cell at column zero has no base.");
+
+  Utf8Storage storage;
+  llvm::StringRef encoded = EncodeUtf8(code_point, storage);
+  std::string& marks = combining_marks_[CellIndex(base, y)];
+  if (marks.size() + encoded.size() > MaxCombiningBytes) {
+    return;
+  }
+  marks.append(encoded.data(), encoded.size());
+}
+
+auto Buffer::DrawCodePoint(int x, int y, char32_t code_point,
+                           const Style& style) -> DrawEnd {
+  CheckTextOrigin(x, y);
+  return {.x = PlaceCodePoint(x, y, code_point, style), .y = y};
+}
+
+auto Buffer::PlaceCodePoint(int x, int y, char32_t code_point,
+                            const Style& style) -> int {
+  CARBON_DCHECK(x >= 0 && y >= 0,
+                "Placing at ({0}, {1}), which no walk should reach.", x, y);
+
+  int width = metrics_.CodePointWidth(code_point);
+  if (width == 0) {
+    AttachCombiningMark(x, y, code_point);
+    return x;
+  }
+  code_point = metrics_.RenderedCodePoint(code_point);
+
+  // Both bounds are reached by what the text holds rather than by where the
+  // caller aimed -- a word overhanging the target width, or newlines running
+  // past the rows a grid can index -- so past either one nothing is drawn and
+  // the column still advances, which is what keeps measuring and drawing
+  // answering the same thing. A double-width character needs both its columns,
+  // so one that would only half fit is past the edge like any other: splitting
+  // it would leave the terminal rendering half a character.
+  if (y >= MaxRows || x > MaxColumns - width) {
+    return x + width;
+  }
+
+  EnsureColumn(x + width - 1);
+  EnsureRow(y);
+  ClearCells(x, y, width);
+
+  Cell& cell = CellAt(x, y);
+  cell.code_point = code_point;
+  cell.style = style;
+  // Nothing is wider than two columns, so the second is the only continuation
+  // there can be.
+  if (width > 1) {
+    Cell& continuation = CellAt(x + 1, y);
+    continuation.style = style;
+    continuation.is_continuation = true;
+  }
+  return x + width;
+}
+
+// Returns the glyphs a cell's directions are read from.
+static auto LineGlyphs(Charset charset) -> const std::array<char32_t, 16>& {
+  return charset == Charset::Utf8 ? Utf8LineGlyphs : AsciiLineGlyphs;
+}
+
+auto Buffer::DrawLine(int x, int y, uint8_t directions, const Style& style)
+    -> void {
+  CARBON_DCHECK(directions <= LineDirections,
+                "Direction bits {0} name no glyph.", directions);
+  EnsureColumn(x);
+  EnsureRow(y);
+
+  uint8_t existing = CellAt(x, y).lines;
+  if (existing == 0) {
+    // Whatever is here isn't a line. Clearing also removes either half of a
+    // double-width character the cell was part of.
+    ClearCells(x, y, 1);
+  }
+
+  Cell& cell = CellAt(x, y);
+  cell.lines = existing | directions | LineCell;
+  cell.code_point = LineGlyphs(metrics_.charset())[cell.lines & LineDirections];
+  cell.style = style;
+}
+
+// Checks that a line of `length` starting at `position` stays within `limit`,
+// which is the width for a horizontal line and `MaxRows` for a vertical one.
+//
+// Unlike text, a line has no reason to reach outside what it is being drawn
+// into: nothing about it is unbreakable, and a layout that put one there
+// computed the wrong extent.
+static auto CheckLineFits(int position, int length, int limit) -> void {
+  CARBON_CHECK(length >= 0 && position <= limit - length,
+               "A line of {0} at {1} runs outside the {2} available to it.",
+               length, position, limit);
+}
+
+auto Buffer::DrawHorizontalLine(int x, int y, int length, const Style& style,
+                                LineEnd start, LineEnd end) -> DrawEnd {
+  CheckOrigin(x, y);
+  CheckLineFits(x, length, columns_);
+  for (int i : llvm::seq(length)) {
+    // A cell in the middle of the line is entered from one side and left by the
+    // other. An end cell is only left towards the rest of the line, unless that
+    // end runs out through the cell's own side.
+    uint8_t directions =
+        (i > 0 || start == LineEnd::Edge ? LineLeft : 0) |
+        (i + 1 < length || end == LineEnd::Edge ? LineRight : 0);
+    DrawLine(x + i, y, directions, style);
+  }
+  return {.x = x + length, .y = y};
+}
+
+auto Buffer::DrawVerticalLine(int x, int y, int length, const Style& style,
+                              LineEnd start, LineEnd end) -> DrawEnd {
+  CheckOrigin(x, y);
+  CheckLineFits(y, length, MaxRows);
+  for (int i : llvm::seq(length)) {
+    uint8_t directions =
+        (i > 0 || start == LineEnd::Edge ? LineUp : 0) |
+        (i + 1 < length || end == LineEnd::Edge ? LineDown : 0);
+    DrawLine(x, y + i, directions, style);
+  }
+  return {.x = x, .y = y + length};
+}
+
+auto Buffer::DrawBox(int x, int y, int box_width, int box_height,
+                     const Style& style) -> DrawEnd {
+  CheckOrigin(x, y);
+  CheckLineFits(x, box_width, columns_);
+  CheckLineFits(y, box_height, MaxRows);
+  if (box_width == 0 || box_height == 0) {
+    return {.x = x, .y = y};
+  }
+  DrawHorizontalLine(x, y, box_width, style);
+  DrawHorizontalLine(x, y + box_height - 1, box_width, style);
+  DrawVerticalLine(x, y, box_height, style);
+  DrawVerticalLine(x + box_width - 1, y, box_height, style);
+  return {.x = x + box_width, .y = y + box_height};
+}
+
+template <typename PlaceFn>
+auto Buffer::WalkText(int x, int y, int margin, llvm::StringRef text,
+                      PlaceFn place) const -> DrawEnd {
+  CheckTextSize(text);
+  CARBON_CHECK(margin >= 0 && margin <= x && y >= 0 && y < MaxRows,
+               "Text at ({0}, {1}) with a margin of {2} is outside the {3} "
+               "rows a buffer covers, or left of its margin.",
+               x, y, margin, MaxRows);
+
+  int cur_x = x;
+  int cur_y = y;
+
+  while (!text.empty()) {
+    char32_t code_point = metrics_.TakeCodePoint(text);
+    if (code_point == '\n') {
+      cur_x = margin;
+      ++cur_y;
+      continue;
+    }
+    if (code_point == '\r') {
+      cur_x = margin;
+      continue;
+    }
+    if (code_point == '\t') {
+      int stop = NextTabStop(cur_x, margin, tab_width_);
+      for (; cur_x < stop; ++cur_x) {
+        place(cur_x, cur_y, U' ');
+      }
+      continue;
+    }
+
+    cur_x = place(cur_x, cur_y, code_point);
+  }
+
+  return {.x = cur_x, .y = cur_y};
+}
+
+auto Buffer::DrawText(int x, int y, int margin, llvm::StringRef text,
+                      const Style& style) -> DrawEnd {
+  return WalkText(x, y, margin, text,
+                  [&](int cur_x, int cur_y, char32_t code_point) {
+                    return PlaceCodePoint(cur_x, cur_y, code_point, style);
+                  });
+}
+
+auto Buffer::MeasureText(int x, int y, int margin, llvm::StringRef text) const
+    -> DrawEnd {
+  return WalkText(x, y, margin, text,
+                  [&](int cur_x, int /*cur_y*/, char32_t code_point) {
+                    return cur_x + metrics_.CodePointWidth(code_point);
+                  });
+}
+
+// Returns whether wrapped text can be broken at `c`.
+//
+// This is the one definition of where wrapping may introduce a break, so that
+// measuring what text wraps into and drawing it wrapped agree about it.
+// Carriage returns count so that a CRLF ending is whitespace rather than part
+// of the word before it; what becomes of the `\r` is then up to the drawing.
+static constexpr auto IsWrapBreak(char c) -> bool {
+  return c == ' ' || c == '\t' || c == '\r';
+}
+
+template <typename PlaceFn>
+auto Buffer::WalkWrappedText(int x, int y, int margin, int max_width,
+                             llvm::StringRef text, PlaceFn place) const
+    -> DrawEnd {
+  CheckTextSize(text);
+  // The block runs from the margin to `margin + max_width`, lies within the
+  // buffer, and holds the column the text starts in, which is every bound on
+  // the three of them read in one order.
+  CARBON_CHECK(llvm::is_sorted(std::array{0, margin, x, x + 1,
+                                          margin + max_width, columns_}) &&
+                   y >= 0 && y < MaxRows,
+               "A block of {0} columns at {1} holding text from ({2}, {3}) "
+               "does not fit the {4} columns and {5} rows a buffer covers.",
+               max_width, margin, x, y, columns_, MaxRows);
+
+  // The column a row runs out of room at. The block lies within the buffer's
+  // width, so this is a column like any other rather than a sum that has to be
+  // kept from overflowing.
+  int limit = margin + max_width;
+
+  int cur_x = x;
+  int cur_y = y;
+
+  // Splitting on bytes is safe because every character text can break at is
+  // ASCII, and UTF-8 never encodes anything else using an ASCII byte. Only
+  // words are decoded; whitespace is handled a byte at a time.
+  while (!text.empty()) {
+    if (text.front() == '\n') {
+      text = text.drop_front();
+      cur_x = margin;
+      ++cur_y;
+      continue;
+    }
+
+    if (IsWrapBreak(text.front())) {
+      llvm::StringRef breaks = text.take_while(IsWrapBreak);
+      text = text.drop_front(breaks.size());
+      for (char c : breaks) {
+        if (c == '\r') {
+          continue;
+        }
+        // Whitespace stops at the block's edge, leaving the word after it to
+        // wrap.
+        int next = std::min(
+            c == '\t' ? NextTabStop(cur_x, margin, tab_width_) : cur_x + 1,
+            limit);
+        while (cur_x < next) {
+          cur_x = place(cur_x, cur_y, U' ');
+        }
+      }
+
+      // A combining mark renders into the column before it, so one following
+      // whitespace belongs to that whitespace and goes with it. Left to begin
+      // the next word, it would move to another row whenever that word wrapped
+      // and attach to whatever preceded it there.
+      while (!text.empty()) {
+        llvm::StringRef rest = text;
+        char32_t code_point = metrics_.TakeCodePoint(rest);
+        if (metrics_.CodePointWidth(code_point) != 0) {
+          break;
+        }
+        text = rest;
+        cur_x = place(cur_x, cur_y, code_point);
+      }
+      continue;
+    }
+
+    llvm::StringRef word =
+        text.take_until([](char c) { return c == '\n' || IsWrapBreak(c); });
+    text = text.drop_front(word.size());
+
+    // Move a word that doesn't fit down to the next row, which minimizes the
+    // overhang when it doesn't fit there either. The word is drawn into the row
+    // this starts before anything else can reach it, so a wrapped row begins at
+    // the margin rather than with the whitespace the wrap came after.
+    if (cur_x > margin && cur_x + metrics_.Width(word) > limit) {
+      cur_x = margin;
+      ++cur_y;
+    }
+
+    while (!word.empty()) {
+      cur_x = place(cur_x, cur_y, metrics_.TakeCodePoint(word));
+    }
+  }
+
+  return {.x = cur_x, .y = cur_y};
+}
+
+auto Buffer::DrawWrappedText(int x, int y, int margin, int max_width,
+                             llvm::StringRef text, const Style& style)
+    -> DrawEnd {
+  return WalkWrappedText(x, y, margin, max_width, text,
+                         [&](int cur_x, int cur_y, char32_t code_point) {
+                           return PlaceCodePoint(cur_x, cur_y, code_point,
+                                                 style);
+                         });
+}
+
+auto Buffer::MeasureWrappedText(int x, int y, int margin, int max_width,
+                                llvm::StringRef text) const -> DrawEnd {
+  return WalkWrappedText(x, y, margin, max_width, text,
+                         [&](int cur_x, int /*cur_y*/, char32_t code_point) {
+                           return cur_x + metrics_.CodePointWidth(code_point);
+                         });
+}
+
+auto Buffer::MeasureWrapWidth(llvm::StringRef text) const -> int {
+  int width = 0;
+  while (!text.empty()) {
+    llvm::StringRef word =
+        text.take_until([](char c) { return c == '\n' || IsWrapBreak(c); });
+    width = std::max(width, metrics_.Width(word));
+    text = text.drop_front(std::max<size_t>(word.size(), 1));
+  }
+  return width;
+}
+
+auto Buffer::LastVisibleColumn(int y, ColorMode mode) const -> int {
+  // A style only paints a blank cell if it is rendered at all, so with color
+  // off a blank cell is padding whatever style it carries.
+  bool styles_render = mode != ColorMode::NoColor;
+  for (int x = width_ - 1; x >= 0; --x) {
+    const Cell& cell = CellAt(x, y);
+    if (cell.is_continuation || cell.code_point != ' ' ||
+        (styles_render && cell.style.IsVisibleOnBlank()) ||
+        (!combining_marks_.empty() &&
+         combining_marks_.contains(CellIndex(x, y)))) {
+      return x;
+    }
+  }
+  return -1;
+}
+
+auto Buffer::Render(OutputBufferRef out, ColorMode mode) const -> void {
+  Utf8Storage storage;
+
+  // The style a terminal starts in, and the one it is left in.
+  const Style default_style;
+
+  // Cells outlive this loop, so the active style is tracked by pointing at one
+  // rather than copying a whole style per cell. It carries across rows: a style
+  // is usually still in use on the row below, and turning it off and back on
+  // costs a reset and a fresh start for nothing.
+  const Style* active = &default_style;
+
+  int rows = height();
+  for (int y = 0; y < rows; ++y) {
+    int last = LastVisibleColumn(y, mode);
+    for (int x = 0; x <= last; ++x) {
+      const Cell& cell = CellAt(x, y);
+      if (cell.is_continuation) {
+        continue;
+      }
+
+      active->AppendTransitionTo(out, cell.style, mode);
+      active = &cell.style;
+      out.Append(EncodeUtf8(cell.code_point, storage));
+
+      // Almost nothing has combining marks, so the lookup is worth skipping
+      // outright rather than doing it for every cell on the screen.
+      if (!combining_marks_.empty()) {
+        auto marks = combining_marks_.find(CellIndex(x, y));
+        if (marks != combining_marks_.end()) {
+          out.Append(marks->second);
+        }
+      }
+    }
+
+    // A style is turned off before the newline in two cases. On the last row,
+    // so that nothing is left set for whatever is printed after this and the
+    // escape that turns it off still falls inside the rendering. And whenever
+    // it paints where there is no glyph, because a terminal fills the rest of
+    // the row with the background it is in when the row ends, so leaving one
+    // set would paint a stripe out to the right edge that nothing asked for.
+    if (y + 1 == rows || active->IsVisibleOnBlank()) {
+      active->AppendTransitionTo(out, default_style, mode);
+      active = &default_style;
+    }
+    out.Append("\n");
+  }
+}
+
+auto Buffer::WriteTo(Filesystem::WriteFileRef file, ColorMode mode) const
+    -> ErrorOr<Success, Filesystem::FdError> {
+  // Sized for the few short lines a diagnostic renders to. A full screen with
+  // color runs well past it and allocates once.
+  llvm::SmallString<1024> bytes;
+  Render(bytes, mode);
+  return file.WriteCompleteBuffer(llvm::ArrayRef<std::byte>(
+      reinterpret_cast<const std::byte*>(bytes.data()), bytes.size()));
+}
+
+}  // namespace Carbon::Terminal

common/terminal/buffer.h

@@ -0,0 +1,546 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_TERMINAL_BUFFER_H_
+#define CARBON_COMMON_TERMINAL_BUFFER_H_
+
+#include <algorithm>
+#include <cstdint>
+#include <string>
+
+#include "common/check.h"
+#include "common/filesystem.h"
+#include "common/terminal/capabilities.h"
+#include "common/terminal/color.h"
+#include "common/terminal/metrics.h"
+#include "common/terminal/output_buffer_ref.h"
+#include "common/terminal/style.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+
+namespace Carbon::Terminal {
+
+// Where a line stops within the cell at one of its ends.
+//
+// A line runs between points, and in a grid of cells the two points it can
+// name are a cell's center and a cell's outer edge. Which one an end is decides
+// what a line meeting it there becomes: a line ending at a center and another
+// leaving that center form a corner, while a line running out through an edge
+// carries on past whatever meets it, which is a tee.
+//
+// This is the distinction a vector graphics stroke draws between a butt cap and
+// a square cap, where the square cap extends the stroke by half its width past
+// the endpoint. Half a stroke here is half a cell.
+//
+// Unicode has a glyph for a line reaching only the middle of its cell (U+2574
+// through U+2577), so a `Center` end is drawn as one and the reader sees where
+// the line really stops rather than having to infer it from the junctions. With
+// `Charset::Ascii` there is nothing to draw half a line with, so both ends fill
+// their cell and only the junctions around them say which was which.
+enum class LineEnd : int8_t {
+  // The line stops at the center of its end cell. Lines meeting there corner.
+  Center,
+  // The line runs out through the outer edge of its end cell, joining whatever
+  // is beyond it. Lines meeting there tee.
+  Edge,
+};
+
+// A grid of styled cells staged for rendering to a terminal.
+//
+// Coordinates are 0-based with (0, 0) at the top left, `x` counting terminal
+// columns and `y` counting rows.
+//
+// A buffer renders once, top to bottom, the way a compiler writes diagnostics.
+// There is no cursor addressing and nothing is ever redrawn, so a rendered
+// buffer is just as valid in a file or a pipe as on a terminal.
+//
+// Every row is a line, ended by a newline of its own, so nothing is left for
+// the terminal to break. A break introduced to fit a width is an ordinary
+// newline like any other, which is what lets wrapped text carry an indent or
+// sit in a column beside a gutter: a terminal wrapping a row of its own accord
+// continues at column zero, under the gutter rather than beside it. It also
+// means text copied out of the output holds the lines that were displayed.
+//
+// The cost is that such a break is in whatever a reader copies, so wrapping
+// never puts one inside a word. A path or a URL stays whole and overhangs the
+// width when it doesn't fit, which is what keeps it selectable in one piece and
+// clickable where a terminal recognizes one. Wrapping only adds breaks as well:
+// the newlines already in a caller's text are kept as they are. A row is a row
+// once something is drawn into it, so a break the text ends with closes its
+// last line rather than opening an empty one after it.
+//
+// Staging into a grid lets layout position content directly, rather than
+// interleaving text, padding, and escape sequences as it goes. That separation
+// is what makes the two hard parts tractable: escape sequences are minimized
+// once, in `Render`, and the drawing APIs reason about columns on screen rather
+// than bytes in a stream.
+//
+// Which bytes make up a column depends on the charset, and the buffer handles
+// that rather than leaving it to callers, because getting it wrong misaligns
+// everything downstream of it:
+//
+// - Under `Charset::Ascii` no UTF-8 processing happens at all. Every byte is
+//   one column, exactly as a terminal decoding some single-byte encoding will
+//   treat it, and bytes outside printable ASCII are replaced with `?` because
+//   there is no telling what such a terminal would draw for them.
+// - Under `Charset::Utf8` bytes are decoded as UTF-8. Double-width characters
+//   occupy both of the columns they will really take, and drawing over either
+//   column erases the whole character instead of leaving half of one behind.
+//   Combining marks render into the column before them, so a base character
+//   and its marks stay in one cell. Carbon source is in Unicode normalization
+//   form C, which still spells out marks for characters that have no
+//   precomposed form, so this comes up in ordinary input. Anything with no
+//   printable rendering, including invalid UTF-8, becomes U+FFFD.
+//
+// A buffer is `columns()` wide, and that width is the whole point of it: it is
+// what wrapping fits text into, and it comes from the terminal where one was
+// measured and from `DefaultColumns` where none was. Rows are the direction
+// there is no bound in -- a buffer grows downward to whatever is drawn into it,
+// up to `MaxRows` -- so laying out is a question of how many rows something
+// takes, never of how wide the grid will turn out to be.
+//
+// The two ways of drawing text differ in whether what they draw is held to the
+// width. `DrawText` does not wrap, so text it is given has nowhere else to go:
+// it widens the buffer, and `width()` grows past `columns()`.
+// `DrawWrappedText` and line drawing are held to the width, since wrapping has
+// the next row and a line running outside it came from a wrong extent. A
+// drawing of either that starts or ends outside the width is a programming
+// error and is checked; a caller placing one already knows the width, since it
+// is what decided the layout.
+//
+// A wrapped block widens the buffer only by the words in it, never by where it
+// was told to start: a word it cannot break overhangs, for the reason above.
+//
+// Nothing is drawn left of the origin or past `MaxColumns` either way, and text
+// running off the bottom on its own newlines is clipped rather than checked.
+//
+// A combining mark renders into the cell before it, so one with no cell before
+// it -- at column zero, or on a row nothing has been drawn on -- has nowhere to
+// go and is dropped. That is data rather than a coordinate, which is why it is
+// dropped rather than checked: source files contain such text.
+//
+// TODO: None of this handles bidirectional text. A right-to-left run reorders
+// on screen, so the column a character occupies stops following from the
+// characters before it, which is the assumption every position here rests on:
+// that drawing advances left to right by the width of what was drawn. Getting
+// this right needs the reordering to happen before anything is placed, which
+// makes it a question about where the boundary between a client's layout and
+// this buffer should sit -- whether the buffer takes runs that are already in
+// visual order, or takes logical order and reorders as it draws, and what it
+// then means for a caller to name a column at all. Marking a span and drawing a
+// line under it are the hard cases, since a logically contiguous span need not
+// be contiguous on screen.
+class Buffer {
+ public:
+  // The bounds a buffer exists within.
+  //
+  // These are far past anything a terminal displays, and exist so that a cell
+  // index stays representable rather than to ration anything. Unlike
+  // `columns()`, every way of drawing is held to them: past them nothing is
+  // drawn and the column still advances, so measuring and drawing agree.
+  // Clipped rather than checked, since how far unwrapped text or an overhang
+  // runs is a fact about the text.
+  static constexpr int MaxColumns = 1 << 14;
+  static constexpr int MaxRows = 1 << 16;
+
+  // The most bytes of text one operation draws or measures.
+  //
+  // The column advances by the width of what was drawn whether or not a cell
+  // was written, so without this a long enough run would carry it past what an
+  // `int` holds and come back negative. Far more text than any terminal shows,
+  // and a caller with this much has built it rather than read it off a line.
+  static constexpr int MaxTextBytes = 1 << 24;
+
+  // The widest tab stops a buffer draws to.
+  //
+  // Far past any terminal, and small enough that even text made entirely of
+  // tabs measures into a column an `int` holds: a tab is the one character
+  // that occupies more columns than it does bytes, so this is what bounds
+  // `MaxTextBytes` of them.
+  static constexpr int MaxTabWidth = 64;
+
+  // Where a drawing ended: for text, the row it ended on and the column after
+  // its last code point there; for a line or a box, the cell past the end of
+  // what it drew.
+  //
+  // Everything that draws returns one, so that a caller placing something
+  // after a drawing advances from this rather than measuring the same text a
+  // second time. The `Measure` operations return one too, and answer for text
+  // that hasn't been drawn yet what drawing it would answer.
+  struct DrawEnd {
+    int x;
+    int y;
+
+    friend auto operator==(DrawEnd lhs, DrawEnd rhs) -> bool = default;
+  };
+
+  // Constructs an empty buffer holding `charset`, laying out for
+  // `DefaultColumns`.
+  explicit Buffer(Charset charset) : Buffer(DefaultColumns, charset) {}
+
+  // Constructs an empty buffer `columns` wide, which must be in
+  // [1, `MaxColumns`], and whose tabs advance to stops `tab_width` columns
+  // apart.
+  //
+  // The width is what everything drawn into the buffer is laid out for and
+  // checked against, not a starting size. The grid holds it from the start, so
+  // a row is only ever reallocated for something that overhangs it.
+  Buffer(int columns, Charset charset, int tab_width = DefaultTabWidth);
+
+  // Constructs an empty buffer holding `capabilities`'s charset and tab stops,
+  // laying out for its width, or for `DefaultColumns` where it has none.
+  //
+  // Both numbers are clamped rather than checked. They describe a terminal
+  // rather than coming from a caller -- `columns` by way of `COLUMNS`, which
+  // anyone can export as anything -- so a value a grid cannot hold is bad input
+  // rather than a mistake, and the nearest usable one lays out no worse than
+  // the fallback would.
+  explicit Buffer(const Capabilities& capabilities)
+      : Buffer(std::clamp(capabilities.columns.value_or(DefaultColumns), 1,
+                          MaxColumns),
+               capabilities.charset,
+               std::clamp(capabilities.tab_width, 1, MaxTabWidth)) {}
+
+  // Returns the width everything drawn into the buffer is laid out for.
+  auto columns() const -> int { return columns_; }
+
+  // Returns the columns the grid currently holds: `columns()` until unwrapped
+  // text or an overhanging word reached past it, and at least enough to hold
+  // what did after that.
+  auto width() const -> int { return width_; }
+
+  // Returns the number of rows the grid holds, which is one past the last row
+  // drawn into.
+  auto height() const -> int;
+
+  auto charset() const -> Charset { return metrics_.charset(); }
+
+  // Returns how text is measured for this buffer's charset.
+  //
+  // The buffer lays its cells out with this, so a caller deciding where to put
+  // something asks the same thing the drawing will.
+  auto metrics() const -> Metrics { return metrics_; }
+
+  // Returns where `DrawText` would end for these arguments, without drawing.
+  //
+  // Measuring and drawing walk the text with the same code, differing only in
+  // whether they write a cell, so a layout decision made from this can't
+  // disagree with what drawing then does.
+  //
+  // This is for text that a tab, a newline, or a carriage return makes
+  // positional. Text with none of them is as wide wherever it is drawn, and
+  // `Metrics::Width` answers for it without a buffer to draw into.
+  auto MeasureText(int x, int y, int margin, llvm::StringRef text) const
+      -> DrawEnd;
+
+  // Returns where the `DrawText` taking no margin would end, which draws `text`
+  // as text of its own beginning at (x, y).
+  auto MeasureText(int x, int y, llvm::StringRef text) const -> DrawEnd {
+    return MeasureText(x, y, x, text);
+  }
+
+  // Returns where `DrawWrappedText` would end for these arguments, without
+  // drawing.
+  //
+  // The block and the origin are checked as drawing checks them, so measuring
+  // answers only for arguments drawing would accept.
+  auto MeasureWrappedText(int x, int y, int margin, int max_width,
+                          llvm::StringRef text) const -> DrawEnd;
+
+  // Returns the fewest columns `text` wraps into without overhanging them,
+  // which is the width of its widest word since wrapping never breaks one.
+  //
+  // Wrapping into fewer columns still draws everything; the excess overhangs.
+  // So this is a layout preference rather than a minimum.
+  auto MeasureWrapWidth(llvm::StringRef text) const -> int;
+
+  // Draws `code_point` at (x, y), which must be a non-negative column and a row
+  // inside `MaxRows`, widening the buffer and adding rows as needed to reach
+  // it. One code point is unwrapped text, so it is not held to the width.
+  //
+  // Returns the column after it, which is `x` again for a combining mark since
+  // one renders into the column before it. A double-width character takes both
+  // its columns wherever it starts: half a character is not something a
+  // terminal can render, so the choice is between the whole of it and none.
+  auto DrawCodePoint(int x, int y, char32_t code_point, const Style& style)
+      -> DrawEnd;
+
+  // Draws a horizontal line across `length` columns starting at (x, y).
+  //
+  // By default the line runs between the centers of its first and last cells,
+  // which is what a line connecting two things is: `DrawBox` draws its four
+  // sides this way, and each pair meets at a corner. `LineEnd::Edge` instead
+  // runs that end out through the side of its cell, which is what a line
+  // bounding `length` whole columns of something is, and what makes a line
+  // meeting it there a tee. A line of one column between two centers is a
+  // point, and is drawn as one.
+  //
+  // Lines join wherever they overlap: a cell records which directions lines
+  // leave it in, and its glyph follows from those bits alone, so crossings,
+  // corners, and tees all appear without being asked for and whatever order
+  // the lines were drawn in. This is the only way to produce a junction, and
+  // it suffices because a junction in real line art always has the lines that
+  // imply it running through it. Only line drawing records directions, so text
+  // containing `-` or `+` is never redrawn as line art.
+  //
+  // A cell's style is whatever was drawn there last, so crossing lines of
+  // different styles do depend on order.
+  auto DrawHorizontalLine(int x, int y, int length, const Style& style,
+                          LineEnd start = LineEnd::Center,
+                          LineEnd end = LineEnd::Center) -> DrawEnd;
+
+  // Draws a vertical line down `length` rows starting at (x, y), with the same
+  // meaning for its ends. Returns the row after it, in the column it ran down.
+  auto DrawVerticalLine(int x, int y, int length, const Style& style,
+                        LineEnd start = LineEnd::Center,
+                        LineEnd end = LineEnd::Center) -> DrawEnd;
+
+  // Draws the outline of a box with its top-left corner at (x, y).
+  //
+  // Each side runs between the centers of the cells it ends in, so the four
+  // corners come out of the sides meeting there. A box with no interior is
+  // then the single line that bounds it, and one with no extent in either
+  // direction is a point, without either being a case of its own.
+  auto DrawBox(int x, int y, int box_width, int box_height, const Style& style)
+      -> DrawEnd;
+
+  // Draws `text` starting at (x, y), which must be a column at or right of
+  // `margin` and a row inside `MaxRows`, as part of text whose left edge is
+  // `margin`.
+  //
+  // Nothing here wraps, so text runs off the right of the width when it is
+  // longer than the room left, and the buffer widens to hold it. That is what
+  // this is for: text that must not be broken, such as a source line quoted as
+  // it was written. A caller that wants the text held to the width wants
+  // `DrawWrappedText`.
+  //
+  // Newlines return to column `margin` on the next row, carriage returns to
+  // column `margin` on the same row, and tabs advance to the next tab stop,
+  // with stops measured from `margin` so that a quoted source line keeps the
+  // tab alignment it had in the file wherever the quote is placed. Returns
+  // where it ended, which for text with a newline in it is on a later row than
+  // it started.
+  //
+  // The margin is what lets text with newlines in it be drawn as differently
+  // styled spans, each starting where the last ended and all naming the same
+  // margin, the way `DrawWrappedText` does for a block: a newline in the middle
+  // of such a run returns to the text's own left edge rather than to wherever
+  // the span it fell in happened to start.
+  auto DrawText(int x, int y, int margin, llvm::StringRef text,
+                const Style& style) -> DrawEnd;
+
+  // Draws `text` as text of its own beginning at (x, y), which is then both
+  // where it starts and the margin its later rows return to.
+  auto DrawText(int x, int y, llvm::StringRef text, const Style& style)
+      -> DrawEnd {
+    return DrawText(x, y, x, text, style);
+  }
+
+  // Draws `text` starting at (x, y), into the block of `max_width` columns
+  // beginning at `margin`.
+  //
+  // The block must lie within `columns()` and `x` within the block, so
+  // `0 <= margin <= x < margin + max_width <= columns()`. A block is a division
+  // of the width rather than something that can exceed it: what a caller wants
+  // when it has nothing to divide is `max_width` of `columns() - margin`, the
+  // whole of what is left.
+  //
+  // The block is what the text wraps within, and (x, y) is only where this run
+  // of it starts: rows after the first begin at `margin`, and how much room a
+  // row has is measured from there. A block whose spans are styled differently
+  // is drawn as one call per span, each starting where the last ended and all
+  // naming the same margin and width. Passing `x` as the margin draws a block
+  // in one call.
+  //
+  // Wrapping breaks at ASCII spaces, tabs, and carriage returns, and only
+  // there. A word here is whatever lies between two of them, so a URL is one
+  // word, and one too long for a row of its own is moved down to one and then
+  // overhangs it rather than being broken.
+  //
+  // Whitespace stops at the block's edge rather than running past it, so the
+  // spaces between two words stay on the row the first of them ended and the
+  // row the second wraps onto begins at the margin. Spaces the text opens with,
+  // or that follow a newline in it, are kept as they are, since those are
+  // indentation the caller wrote.
+  //
+  // Newlines are breaks the caller already made, and are kept as they are:
+  // wrapping only adds breaks to the text it is given. They break the line as a
+  // wrap does, continuing at `margin` on the next row, and carriage returns are
+  // dropped so that CRLF endings break exactly once.
+  //
+  // A tab is both a break opportunity and a jump to the next tab stop, with
+  // stops measured from `margin` rather than from `x`. The margin is the one
+  // column every row of the block begins at, so the stops are the same on each
+  // of them and a tabbed column stays a column however the text wraps; stops
+  // from `x` would move with the span that happened to be drawn first. A tab
+  // that would reach past the block stops at its edge, like the spaces do,
+  // leaving the word after it to wrap.
+  //
+  // `DrawText` is the way to draw text that should not wrap at all, and differs
+  // in more than that: it keeps every space, and returns to the margin on a
+  // carriage return rather than dropping it.
+  //
+  // Returns where it ended.
+  //
+  // TODO: There is no mode that reflows, treating the newlines in `text` as
+  // breaks to be chosen again rather than kept. Text that arrives wrapped to
+  // some other width keeps that wrapping, which is wrong for it wherever that
+  // width isn't the one it is being drawn into. Add one when there is a caller
+  // with such text, since which breaks a reflow may discard -- every newline,
+  // or only those a previous wrapping introduced -- is a question about where
+  // that text came from.
+  auto DrawWrappedText(int x, int y, int margin, int max_width,
+                       llvm::StringRef text, const Style& style) -> DrawEnd;
+
+  // Renders the grid, appending the bytes that draw it to `out`.
+  //
+  // Each row ends in a newline, with trailing blank cells dropped so output
+  // carries no invisible padding. The rendering ends with the style turned off
+  // so nothing bleeds into what is printed next, and a style that paints blank
+  // cells is turned off at each row's end so a background does not run to the
+  // right edge. Color is chosen here rather than at construction because it
+  // affects only how cells are serialized, while the charset decides how
+  // content is laid out into them.
+  auto Render(OutputBufferRef out, ColorMode mode) const -> void;
+
+  // Renders the grid and writes it to `file`.
+  //
+  // The whole grid goes out in one `write` where the destination accepts it,
+  // which is what gives the output whatever atomicity the descriptor offers
+  // against other writers: a terminal or a pipe interleaves at write
+  // boundaries, so one call per rendered buffer is the most that can be had
+  // without a lock.
+  auto WriteTo(Filesystem::WriteFileRef file, ColorMode mode) const
+      -> ErrorOr<Success, Filesystem::FdError>;
+
+ private:
+  // The directions in which drawn lines leave a cell, and whether the cell
+  // holds line art at all. A cell's glyph is a function of the directions
+  // alone.
+  enum LineDirection : uint8_t {
+    LineLeft = 1 << 0,
+    LineRight = 1 << 1,
+    LineUp = 1 << 2,
+    LineDown = 1 << 3,
+    LineDirections = 0b1111,
+    // Set on every cell line drawing writes. A cell can hold line art and no
+    // directions -- a line between one center and itself is a point -- and
+    // without this such a cell would be indistinguishable from one holding
+    // text, so nothing drawn later would join it.
+    LineCell = 1 << 4,
+  };
+
+  struct Cell {
+    // The code point rendered here. For a cell with `lines` set, this is
+    // derived from those bits and the charset.
+    char32_t code_point = ' ';
+
+    Style style;
+
+    // Which directions drawn lines leave this cell in, with `LineCell` set,
+    // or zero for a cell holding text.
+    uint8_t lines = 0;
+
+    // Whether this cell is the right half of a double-width character, and so
+    // renders nothing of its own.
+    bool is_continuation = false;
+  };
+
+  // Checks that `text` is short enough to measure without overflowing a column.
+  static auto CheckTextSize(llvm::StringRef text) -> void {
+    CARBON_CHECK(text.size() <= MaxTextBytes,
+                 "Laying out {0} bytes of text is past the {1} one operation "
+                 "handles.",
+                 text.size(), MaxTextBytes);
+  }
+
+  auto CellIndex(int x, int y) const -> int { return y * width_ + x; }
+  auto CellAt(int x, int y) -> Cell& { return cells_[CellIndex(x, y)]; }
+  auto CellAt(int x, int y) const -> const Cell& {
+    return cells_[CellIndex(x, y)];
+  }
+
+  // Checks that (x, y) is somewhere unwrapped text may start, which the width
+  // does not decide.
+  //
+  // The text walks check this themselves, together with the bounds particular
+  // to each: they are inlined into every text operation, and one check there
+  // costs measurably less than two.
+  auto CheckTextOrigin(int x, int y) const -> void {
+    CARBON_CHECK(x >= 0 && y >= 0 && y < MaxRows,
+                 "Drawing text at ({0}, {1}) is outside the {2} rows a buffer "
+                 "covers.",
+                 x, y, MaxRows);
+  }
+
+  // Checks that (x, y) is somewhere a drawing held to the width may start.
+  auto CheckOrigin(int x, int y) const -> void {
+    CARBON_CHECK(
+        x >= 0 && x < columns_ && y >= 0 && y < MaxRows,
+        "Drawing at ({0}, {1}) is outside the {2} columns and {3} rows "
+        "a buffer covers.",
+        x, y, columns_, MaxRows);
+  }
+
+  // Places `code_point` at (x, y) without checking it against the target width
+  // or `MaxRows`, which text reaches on its own by overhanging or by carrying
+  // newlines. Past either, nothing is drawn and the column still advances. The
+  // coordinates must be non-negative, which follows from the origin the walk
+  // was checked at.
+  auto PlaceCodePoint(int x, int y, char32_t code_point, const Style& style)
+      -> int;
+
+  // The walks behind the text operations, over which drawing and measuring are
+  // the same code. `place` is called with each code point and where it goes,
+  // and returns the column after it: `PlaceCodePoint` when drawing, and the
+  // width alone when measuring.
+  template <typename PlaceFn>
+  auto WalkText(int x, int y, int margin, llvm::StringRef text,
+                PlaceFn place) const -> DrawEnd;
+  template <typename PlaceFn>
+  auto WalkWrappedText(int x, int y, int margin, int max_width,
+                       llvm::StringRef text, PlaceFn place) const -> DrawEnd;
+
+  // Adds rows until row `y` exists.
+  auto EnsureRow(int y) -> void;
+
+  // Widens the grid until column `x` exists, reflowing the rows it already
+  // holds, which are stored back to back. Only something overhanging the target
+  // width reaches past it, so this runs for nothing else.
+  auto EnsureColumn(int x) -> void;
+
+  // Resets the cells in row `y` spanning columns [x, x + width), along with
+  // either half of a double-width character that straddles the range's edges.
+  auto ClearCells(int x, int y, int width) -> void;
+
+  // Appends `code_point` to the marks rendered with the cell before column `x`.
+  auto AttachCombiningMark(int x, int y, char32_t code_point) -> void;
+
+  // Adds `directions` to the lines through (x, y) and updates its glyph.
+  auto DrawLine(int x, int y, uint8_t directions, const Style& style) -> void;
+
+  // Returns the last column in row `y` that renders anything under `mode`, or
+  // -1 when the row renders nothing.
+  auto LastVisibleColumn(int y, ColorMode mode) const -> int;
+
+  // The width laid out for, and the width the grid holds. They differ only
+  // where something overhung the first.
+  int columns_;
+  int width_;
+
+  int tab_width_;
+  Metrics metrics_;
+
+  llvm::SmallVector<Cell, 0> cells_;
+
+  // Combining marks, as UTF-8, for the few cells that have any, keyed by cell
+  // index. Kept out of `Cell` so that the common case of no marks costs
+  // nothing per cell. Always empty under `Charset::Ascii`.
+  llvm::DenseMap<int, std::string> combining_marks_;
+};
+
+}  // namespace Carbon::Terminal
+
+#endif  // CARBON_COMMON_TERMINAL_BUFFER_H_

common/terminal/capabilities.cpp

@@ -0,0 +1,243 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/capabilities.h"
+
+#include <sys/ioctl.h>
+#include <unistd.h>
+
+#include <cstdlib>
+
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringSwitch.h"
+
+namespace Carbon::Terminal {
+
+// Returns the value of `name` in the process environment, empty when unset.
+static auto GetEnv(const char* name) -> llvm::StringRef {
+  const char* value = std::getenv(name);
+  return value ? llvm::StringRef(value) : llvm::StringRef();
+}
+
+auto ColorEnvironment::FromProcess() -> ColorEnvironment {
+  return {.no_color = GetEnv("NO_COLOR"),
+          .clicolor_force = GetEnv("CLICOLOR_FORCE"),
+          .force_color = GetEnv("FORCE_COLOR"),
+          .clicolor = GetEnv("CLICOLOR"),
+          .colorterm = GetEnv("COLORTERM"),
+          .term_program = GetEnv("TERM_PROGRAM"),
+          .term = GetEnv("TERM")};
+}
+
+// Returns whether the environment and the stream call for color, ignoring any
+// explicit preference. See `ChooseColorMode` for the precedence this
+// implements and where it comes from.
+static auto EnvironmentEnablesColor(const ColorEnvironment& env,
+                                    bool is_terminal) -> bool {
+  if (!env.no_color.empty()) {
+    return false;
+  }
+  // The forcing variables use `0` to decline to force, and `FORCE_COLOR` takes
+  // it further as a request to disable.
+  if (env.force_color == "0") {
+    return false;
+  }
+  if (!env.force_color.empty() ||
+      (!env.clicolor_force.empty() && env.clicolor_force != "0")) {
+    return true;
+  }
+  if (env.clicolor == "0") {
+    return false;
+  }
+  if (!is_terminal) {
+    return false;
+  }
+
+  // `dumb` says outright that escape sequences won't render, which outranks
+  // anything below claiming they will.
+  if (env.term == "dumb") {
+    return false;
+  }
+
+  // Any of these identifies the terminal as something that renders escapes. A
+  // terminal that none of them describe can't be assumed to.
+  //
+  // `COLORTERM` and `TERM_PROGRAM` stand on their own rather than refining
+  // `TERM`: `TERM` names a terminfo entry, while these name the emulator and
+  // the color it handles.
+  return !env.term.empty() || !env.colorterm.empty() ||
+         !env.term_program.empty();
+}
+
+// Returns the richest color escapes the terminal is believed to accept.
+//
+// Every signal here is a heuristic: there is no way to ask a terminal what it
+// supports without writing to it and parsing a reply, which would be far too
+// invasive for a compiler. Guessing too high garbles color on a terminal that
+// can't keep up, and guessing too low only makes output plainer, so unknown
+// terminals get the conservative answer.
+static auto DetectColorDepth(const ColorEnvironment& env) -> ColorMode {
+  // `FORCE_COLOR`'s levels name a depth outright.
+  if (auto mode = llvm::StringSwitch<std::optional<ColorMode>>(env.force_color)
+                      .Case("1", ColorMode::Ansi16)
+                      .Case("2", ColorMode::Ansi256)
+                      .Case("3", ColorMode::Truecolor)
+                      .Default(std::nullopt)) {
+    return *mode;
+  }
+
+  // The convention documented at
+  // https://github.com/termstandard/colors#checking-for-colorterm.
+  if (env.colorterm == "truecolor" || env.colorterm == "24bit") {
+    return ColorMode::Truecolor;
+  }
+
+  // `TERM_PROGRAM` identifies the emulator regardless of how `TERM` is set,
+  // which matters because several of these ship a conservative `TERM` while
+  // rendering far more than it claims.
+  {
+    if (auto mode =
+            llvm::StringSwitch<std::optional<ColorMode>>(env.term_program)
+                .Case("vscode", ColorMode::Truecolor)
+                .Case("iTerm.app", ColorMode::Truecolor)
+                .Case("WarpTerminal", ColorMode::Truecolor)
+                .Case("Hyper", ColorMode::Truecolor)
+                .Case("Tabby", ColorMode::Truecolor)
+                .Case("Terminus", ColorMode::Truecolor)
+                // Apple's Terminal renders only the 256-color palette.
+                .Case("Apple_Terminal", ColorMode::Ansi256)
+                .Default(std::nullopt)) {
+      return *mode;
+    }
+  }
+
+  // The enumerated terminals that stand in for a terminfo lookup.
+  {
+    if (auto mode = llvm::StringSwitch<std::optional<ColorMode>>(env.term)
+                        .Case("xterm-kitty", ColorMode::Truecolor)
+                        .Case("alacritty", ColorMode::Truecolor)
+                        .Case("wezterm", ColorMode::Truecolor)
+                        .Case("ghostty", ColorMode::Truecolor)
+                        .StartsWith("foot", ColorMode::Truecolor)
+                        .StartsWith("contour", ColorMode::Truecolor)
+                        .StartsWith("vte", ColorMode::Truecolor)
+                        .EndsWith("-direct", ColorMode::Truecolor)
+                        .EndsWith("-truecolor", ColorMode::Truecolor)
+                        .EndsWith("-256color", ColorMode::Ansi256)
+                        .EndsWith("-256", ColorMode::Ansi256)
+                        .Default(std::nullopt)) {
+      return *mode;
+    }
+  }
+
+  // Color is called for, but nothing said how much of it works.
+  return ColorMode::Ansi16;
+}
+
+auto ChooseColorMode(Preference preference, const ColorEnvironment& env,
+                     bool is_terminal) -> ColorMode {
+  switch (preference) {
+    case Preference::Never:
+      return ColorMode::NoColor;
+    case Preference::Always:
+      break;
+    case Preference::Auto:
+      if (!EnvironmentEnablesColor(env, is_terminal)) {
+        return ColorMode::NoColor;
+      }
+      break;
+  }
+  return DetectColorDepth(env);
+}
+
+auto ChooseCharset(Preference preference, llvm::StringRef locale) -> Charset {
+  switch (preference) {
+    case Preference::Never:
+      return Charset::Ascii;
+    case Preference::Always:
+      return Charset::Utf8;
+    case Preference::Auto:
+      break;
+  }
+
+  // Locale names spell the encoding several ways: `en_US.UTF-8`, `C.utf8`, and
+  // bare `UTF-8` all appear in the wild.
+  return locale.contains_insensitive("utf-8") ||
+                 locale.contains_insensitive("utf8")
+             ? Charset::Utf8
+             : Charset::Ascii;
+}
+
+auto ChooseBackground(BackgroundPreference preference,
+                      llvm::StringRef colorfgbg) -> Background {
+  switch (preference) {
+    case BackgroundPreference::Dark:
+      return Background::Dark;
+    case BackgroundPreference::Light:
+      return Background::Light;
+    case BackgroundPreference::Auto:
+      break;
+  }
+
+  // The background is the last field, since some terminals write a third one
+  // between the foreground and it.
+  llvm::StringRef background = colorfgbg.rsplit(';').second;
+  unsigned index = 0;
+  if (!llvm::to_integer(background, index) || index > 15) {
+    // Anything else, including the `default` some terminals write and the
+    // variable being unset, says nothing.
+    return Background::Dark;
+  }
+  // The first eight palette entries are the dark half, except that the eighth
+  // is white and the ninth is the dark gray that follows it.
+  return (index <= 6 || index == 8) ? Background::Dark : Background::Light;
+}
+
+// Returns the locale that determines the terminal's character encoding,
+// following the precedence POSIX defines for `LC_CTYPE`.
+static auto GetLocale() -> llvm::StringRef {
+  for (const char* name : {"LC_ALL", "LC_CTYPE", "LANG"}) {
+    if (llvm::StringRef value = GetEnv(name); !value.empty()) {
+      return value;
+    }
+  }
+  return "";
+}
+
+// Returns the terminal's width in columns, or nullopt when there is nothing to
+// ask.
+//
+// `COLUMNS` comes first: when it is exported, the user has deliberately
+// overridden the real width.
+static auto GetColumns(int fd) -> std::optional<int> {
+  int columns = 0;
+  if (llvm::to_integer(GetEnv("COLUMNS"), columns) && columns > 0) {
+    return columns;
+  }
+
+  struct winsize size = {};
+  if (ioctl(fd, TIOCGWINSZ, &size) == 0 && size.ws_col > 0) {
+    return size.ws_col;
+  }
+  return std::nullopt;
+}
+
+auto Capabilities::Detect(Filesystem::WriteFileRef file,
+                          Preferences preferences) -> Capabilities {
+  int fd = file.unix_fd();
+
+  Capabilities capabilities;
+  capabilities.is_terminal = isatty(fd) != 0;
+  capabilities.color_mode =
+      ChooseColorMode(preferences.color, ColorEnvironment::FromProcess(),
+                      capabilities.is_terminal);
+  capabilities.charset = ChooseCharset(preferences.utf8, GetLocale());
+  capabilities.background =
+      ChooseBackground(preferences.background, GetEnv("COLORFGBG"));
+  capabilities.columns = GetColumns(fd);
+
+  return capabilities;
+}
+
+}  // namespace Carbon::Terminal

common/terminal/capabilities.h

@@ -0,0 +1,253 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_TERMINAL_CAPABILITIES_H_
+#define CARBON_COMMON_TERMINAL_CAPABILITIES_H_
+
+#include <cstdint>
+#include <optional>
+
+#include "common/filesystem.h"
+#include "common/terminal/color.h"
+#include "llvm/ADT/StringRef.h"
+
+namespace Carbon::Terminal {
+
+// The encoding the terminal decodes output with.
+//
+// This decides far more than which characters can be drawn: rendering has to
+// count the columns a run of bytes will occupy, and that count only follows
+// from the code points those bytes encode if the terminal agrees about the
+// encoding. Disagreeing misaligns the entire line rather than drawing a single
+// character wrong.
+//
+// No other encoding is modeled. A terminal decoding something else, an ISO 8859
+// part for example, is treated as `Ascii`. That is correct output for any of
+// them, as they all encode printable ASCII as itself, and rendering in one
+// natively would mean carrying its conversion and column-width tables to gain
+// nothing but nicer line drawing, which `Ascii` already has a fallback for. So
+// `Utf8` is used only where the environment says outright that the terminal
+// decodes UTF-8, and `Ascii` does no UTF-8 processing at all.
+enum class Charset : int8_t {
+  // Every byte is one column, and lines are drawn from `-`, `|`, and `+`.
+  //
+  // Bytes outside printable ASCII are replaced rather than passed through,
+  // because a terminal decoding some single-byte encoding will render them as
+  // something, and there is no way to know what.
+  Ascii,
+  // Bytes are decoded as UTF-8, giving double-width characters two columns and
+  // combining marks none, and lines are drawn with box-drawing characters.
+  Utf8,
+};
+
+// Whether to use one of the terminal features detection decides about, where
+// an explicit request overrides what detection would conclude.
+//
+// This is the tri-state that a `--color=never` style flag parses into. It says
+// nothing about which feature is being requested; `Preferences` holds one of
+// these per feature.
+enum class Preference : int8_t {
+  // Decide from the environment and the stream.
+  Auto,
+  // Never use the feature, whatever the environment says.
+  Never,
+  // Use the feature even when the stream isn't a terminal. This is what a
+  // caller wants when piping into a pager, capturing output for later replay,
+  // or writing a test.
+  Always,
+};
+
+// What the terminal draws its text on.
+//
+// Nothing about the rendering depends on the exact color, only on which side of
+// the middle it sits: a color chosen to read on one is hard to read on the
+// other.
+enum class Background : int8_t {
+  Dark,
+  Light,
+};
+
+// An explicit statement of what the terminal draws its text on, where `Auto`
+// leaves it to detection.
+//
+// This is a tri-state like `Preference`, but its two settings name the answer
+// rather than turning a feature on and off, so it is an enum of its own.
+enum class BackgroundPreference : int8_t {
+  Auto,
+  Dark,
+  Light,
+};
+
+// An explicit preference for each feature detection decides about, normally
+// parsed from command line flags.
+struct Preferences {
+  Preference color = Preference::Auto;
+  Preference utf8 = Preference::Auto;
+  BackgroundPreference background = BackgroundPreference::Auto;
+};
+
+// The environment variables that control whether and how color is used.
+//
+// An unset variable and one set to the empty string mean the same thing
+// throughout: no opinion. Values point into the process environment and are
+// invalidated by anything that modifies it.
+struct ColorEnvironment {
+  // Reads the variables from the process environment.
+  static auto FromProcess() -> ColorEnvironment;
+
+  llvm::StringRef no_color;
+  llvm::StringRef clicolor_force;
+  llvm::StringRef force_color;
+  llvm::StringRef clicolor;
+  llvm::StringRef colorterm;
+  llvm::StringRef term_program;
+  llvm::StringRef term;
+};
+
+// Returns the color mode to render with.
+//
+// Whether to use color at all is decided first, from highest priority to
+// lowest:
+//
+// - An explicit `Never` or `Always` preference.
+// - `NO_COLOR` set to any non-empty value disables color: see
+//   https://no-color.org.
+// - `FORCE_COLOR=0` disables color, following the Node convention.
+// - Any other non-empty `FORCE_COLOR`, or a non-empty `CLICOLOR_FORCE` other
+//   than `0`, enables color even when the stream isn't a terminal.
+// - `CLICOLOR=0` disables color.
+// - `TERM=dumb` disables color, being an explicit statement that escape
+//   sequences won't render.
+// - Otherwise color is used only when the stream is a terminal and something
+//   identifies that terminal: `TERM` set to anything else, or `COLORTERM` or
+//   `TERM_PROGRAM` set at all. The latter two stand on their own rather than
+//   refining `TERM`, because the emulator sets them itself and they are
+//   specifically about color, while `TERM` is left unset by anything not
+//   launched from a shell.
+//
+// How much color to use is then guessed from `FORCE_COLOR`'s level,
+// `COLORTERM`, `TERM_PROGRAM`, and `TERM`, falling back to `Ansi16` when color
+// is called for but nothing says how much of it works. Apart from
+// `FORCE_COLOR`, which is a request rather than a description, none of these
+// enable color on their own, so a rich `COLORTERM` inherited by a redirected
+// stream can't put escape sequences into it.
+//
+// Depth comes from enumerating known terminals rather than from terminfo,
+// which trades a list to maintain here for not depending on databases that are
+// routinely absent from the containers and CI images this runs in. An
+// unrecognized terminal gets the conservative answer.
+//
+// This is separated from `Capabilities::Detect` so that the policy can be
+// tested without touching the process environment.
+auto ChooseColorMode(Preference preference, const ColorEnvironment& env,
+                     bool is_terminal) -> ColorMode;
+
+// Returns the encoding to render with, where `locale` is the value of the
+// first set variable among `LC_ALL`, `LC_CTYPE`, and `LANG`.
+//
+// Only a locale that names UTF-8 gets `Utf8`. Guessing wrong in that direction
+// costs alignment on every line that isn't pure ASCII, while guessing wrong
+// the other way only makes output plainer.
+auto ChooseCharset(Preference preference, llvm::StringRef locale) -> Charset;
+
+// Returns what the terminal draws its text on, where `colorfgbg` is the value
+// of `COLORFGBG`.
+//
+// That variable is the only thing a process can read without talking to the
+// terminal. `rxvt` and its derivatives set it, as do a few others, to the
+// foreground and background palette indices separated by `;` -- sometimes with
+// a third field between them -- so the background is the last of them. An index
+// of 0 through 6 or 8 is a dark one, 7 and 9 through 15 a light one, and
+// anything outside that range says nothing.
+//
+// It is missing far more often than it is present, and stale when the user
+// changes their theme without restarting, so anything it doesn't answer is
+// treated as dark. Guessing wrong that way costs contrast; guessing wrong the
+// other way puts light text on a light background.
+auto ChooseBackground(BackgroundPreference preference,
+                      llvm::StringRef colorfgbg) -> Background;
+
+// The width to lay out for when nothing says how wide the output is.
+//
+// Layout always has a width to fit, because the alternative is output laid out
+// as if nothing bounded it, which a terminal then wraps at column zero --
+// breaking every indent and gutter it was given, and in the middle of whatever
+// word it lands on. The cost of guessing is asymmetric: a viewer wider than
+// this sees slack on the right, while one narrower sees the wrapping done
+// twice, ours and then its own.
+//
+// Eighty is the traditional terminal width, and narrower ones are rare enough
+// that fitting them would cost more in wasted width everywhere else.
+inline constexpr int DefaultColumns = 80;
+
+// The columns between tab stops, absent anything saying otherwise.
+//
+// Eight is the interval terminfo records as `it#8` for all but a handful of
+// legacy entries. Nothing measures a terminal's stops, so unlike its width this
+// stands in for no measurement: `Capabilities` carries it as a plain value
+// rather than as one a caller can tell apart from an absence.
+inline constexpr int DefaultTabWidth = 8;
+
+// What the terminal behind a stream can render, and how wide it is.
+//
+// Detect this once per stream at startup and pass it down; the fields come from
+// environment queries and system calls that shouldn't be repeated per
+// diagnostic.
+struct Capabilities {
+  // Detects the capabilities of the terminal behind `file`, honoring
+  // `preferences`.
+  //
+  // Detection reads the descriptor directly, because `isatty` and `TIOCGWINSZ`
+  // are what answer the question and no stream abstraction exposes them.
+  // LLVM's `raw_ostream::has_colors()` is not a substitute for the enablement
+  // rule above, which recognizes terminals its `TERM` list doesn't.
+  //
+  // An `OSC 11` query would ask the terminal what it draws on, which is the
+  // only accurate answer and what `vim`, `delta`, and `bat` do. It isn't one a
+  // non-interactive tool can use: the reply has to be waited for, and drawing
+  // with it only when it arrives in time would leave the colors depending on
+  // that. Reading it also consumes whatever was typed ahead for the next shell
+  // command, along with the input a compile may be taking from stdin.
+  // `COLORFGBG` is the passive stand-in that costs none of this.
+  static auto Detect(Filesystem::WriteFileRef file,
+                     Preferences preferences = {}) -> Capabilities;
+
+  // The richest color escapes the terminal is believed to understand.
+  ColorMode color_mode = ColorMode::NoColor;
+
+  // The encoding the terminal decodes output with.
+  Charset charset = Charset::Ascii;
+
+  // What the terminal draws its text on.
+  Background background = Background::Dark;
+
+  // Whether the stream is attached to a terminal at all. Note that color can
+  // still be in use when this is false, if the environment forces it.
+  bool is_terminal = false;
+
+  // The terminal's width, or none when nothing says how wide the output is.
+  // Positive whenever it is set, so layout can divide by it freely.
+  //
+  // This says what was measured, and nothing is invented to fill it in: an
+  // absence is a real answer about a pipe nobody described. Laying out still
+  // needs a width, and `DefaultColumns` is what a layout falls back to, so
+  // whether this is set decides whether output is fitted to the terminal in
+  // front of it or to a width chosen to be safe wherever it ends up.
+  std::optional<int> columns;
+
+  // The columns between the terminal's tab stops, which is what a tab in text
+  // advances to the next of.
+  //
+  // TODO: Nothing sets this away from `DefaultTabWidth`. A terminal's stops are
+  // mutable at runtime -- `hts` sets one and `tbc` clears them -- so the only
+  // report of the live ones is `DECRQPSR`, which few emulators outside `xterm`
+  // answer, or a `DSR-CPR` round trip after writing a tab, which nearly all do.
+  // Either means putting the descriptor in raw mode and reading a reply with a
+  // timeout. Add it when a terminal that disagrees with eight is worth that.
+  int tab_width = DefaultTabWidth;
+};
+
+}  // namespace Carbon::Terminal
+
+#endif  // CARBON_COMMON_TERMINAL_CAPABILITIES_H_

common/terminal/capabilities_test.cpp

@@ -0,0 +1,340 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/capabilities.h"
+
+#include <gtest/gtest.h>
+
+#include <utility>
+
+#include "common/filesystem.h"
+
+namespace Carbon::Terminal {
+namespace {
+
+// Detection policy is a pure function of the environment and whether the
+// stream is a terminal, so these tests build the environment directly instead
+// of mutating the process environment, which would leak between tests and race
+// with anything else running.
+auto OnTerminal(const ColorEnvironment& env) -> ColorMode {
+  return ChooseColorMode(Preference::Auto, env, /*is_terminal=*/true);
+}
+
+auto OffTerminal(const ColorEnvironment& env) -> ColorMode {
+  return ChooseColorMode(Preference::Auto, env, /*is_terminal=*/false);
+}
+
+// A terminal that supports color, for tests varying one other variable.
+auto ColorTerminal() -> ColorEnvironment { return {.term = "xterm-256color"}; }
+
+TEST(CapabilitiesTest, ColorNeedsATerminal) {
+  EXPECT_EQ(OnTerminal(ColorTerminal()), ColorMode::Ansi256);
+
+  // Writing to a file or a pipe must stay plain, or every redirected build log
+  // fills with escape sequences.
+  EXPECT_EQ(OffTerminal(ColorTerminal()), ColorMode::NoColor);
+
+  // A terminal that nothing says anything about can't be assumed to render
+  // escapes.
+  EXPECT_EQ(OnTerminal({}), ColorMode::NoColor);
+  EXPECT_EQ(OnTerminal({.term = ""}), ColorMode::NoColor);
+  EXPECT_EQ(OnTerminal({.term = "dumb"}), ColorMode::NoColor);
+}
+
+TEST(CapabilitiesTest, ColorFromTheEmulatorWithoutTerm) {
+  // `TERM` is unset for anything not launched from a shell, but the emulator
+  // sets `COLORTERM` and `TERM_PROGRAM` itself, and both are specifically
+  // about color. Either one identifies the terminal on its own, at whatever
+  // depth it names.
+  EXPECT_EQ(OnTerminal({.colorterm = "truecolor"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.colorterm = "yes"}), ColorMode::Ansi16);
+  EXPECT_EQ(OnTerminal({.term_program = "vscode"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term_program = "unknown"}), ColorMode::Ansi16);
+
+  // An empty value says nothing at all.
+  EXPECT_EQ(OnTerminal({.colorterm = "", .term_program = ""}),
+            ColorMode::NoColor);
+
+  // `dumb` outranks them: it states outright that escapes won't render.
+  EXPECT_EQ(OnTerminal({.colorterm = "truecolor", .term = "dumb"}),
+            ColorMode::NoColor);
+
+  // And none of them enable color off a terminal.
+  EXPECT_EQ(OffTerminal({.colorterm = "truecolor"}), ColorMode::NoColor);
+  EXPECT_EQ(OffTerminal({.term_program = "vscode"}), ColorMode::NoColor);
+}
+
+TEST(CapabilitiesTest, ExplicitPreferenceWins) {
+  ColorEnvironment forcing = {.force_color = "3", .term = "xterm-256color"};
+  EXPECT_EQ(ChooseColorMode(Preference::Never, forcing, /*is_terminal=*/true),
+            ColorMode::NoColor);
+
+  ColorEnvironment disabling = {.no_color = "1", .term = "dumb"};
+  EXPECT_EQ(ChooseColorMode(Preference::Always, disabling,
+                            /*is_terminal=*/false),
+            ColorMode::Ansi16);
+
+  // Forcing color on without any hint of what the terminal handles gets the
+  // depth every color terminal supports.
+  EXPECT_EQ(ChooseColorMode(Preference::Always, {}, /*is_terminal=*/false),
+            ColorMode::Ansi16);
+  EXPECT_EQ(ChooseColorMode(Preference::Always, ColorTerminal(),
+                            /*is_terminal=*/false),
+            ColorMode::Ansi256);
+}
+
+TEST(CapabilitiesTest, NoColor) {
+  // https://no-color.org: any non-empty value disables color, whatever it is.
+  EXPECT_EQ(OnTerminal({.no_color = "1", .term = "xterm-256color"}),
+            ColorMode::NoColor);
+  EXPECT_EQ(OnTerminal({.no_color = "0", .term = "xterm-256color"}),
+            ColorMode::NoColor);
+
+  // Being set to the empty string carries no meaning, so it must not disable
+  // color: an empty variable inherited from a wrapper script would otherwise
+  // silently turn color off everywhere.
+  EXPECT_EQ(OnTerminal({.no_color = "", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+
+  // It outranks the forcing variables.
+  EXPECT_EQ(OnTerminal({.no_color = "1", .force_color = "3"}),
+            ColorMode::NoColor);
+  EXPECT_EQ(OnTerminal({.no_color = "1", .clicolor_force = "1"}),
+            ColorMode::NoColor);
+}
+
+TEST(CapabilitiesTest, ForceColor) {
+  // Color even without a terminal, at the depth the level names.
+  EXPECT_EQ(OffTerminal({.force_color = "1"}), ColorMode::Ansi16);
+  EXPECT_EQ(OffTerminal({.force_color = "2"}), ColorMode::Ansi256);
+  EXPECT_EQ(OffTerminal({.force_color = "3"}), ColorMode::Truecolor);
+
+  // The level overrides what the terminal claims.
+  EXPECT_EQ(OnTerminal({.force_color = "1", .colorterm = "truecolor"}),
+            ColorMode::Ansi16);
+
+  // Any other non-empty value enables color without naming a depth.
+  EXPECT_EQ(OffTerminal({.force_color = "true"}), ColorMode::Ansi16);
+  EXPECT_EQ(OffTerminal({.force_color = "true", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+
+  // Zero disables color outright, even on a capable terminal.
+  EXPECT_EQ(OnTerminal({.force_color = "0", .term = "xterm-256color"}),
+            ColorMode::NoColor);
+}
+
+TEST(CapabilitiesTest, EmptyValuesCarryNoOpinion) {
+  // An empty variable means the same as an unset one throughout, so a wrapper
+  // script that exports one without a value changes nothing.
+  EXPECT_EQ(OffTerminal({.force_color = ""}), ColorMode::NoColor);
+  EXPECT_EQ(OffTerminal({.clicolor_force = ""}), ColorMode::NoColor);
+  EXPECT_EQ(OnTerminal({.no_color = "", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+  EXPECT_EQ(OnTerminal({.clicolor = "", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+  EXPECT_EQ(OnTerminal({.force_color = "", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+}
+
+TEST(CapabilitiesTest, CliColor) {
+  // The BSD convention: `CLICOLOR_FORCE` enables color off a terminal, and
+  // `CLICOLOR=0` disables it on one.
+  EXPECT_EQ(OffTerminal({.clicolor_force = "1"}), ColorMode::Ansi16);
+  EXPECT_EQ(OffTerminal({.clicolor_force = "1", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+  // `0` means "don't force", not "disable", so a terminal still gets color.
+  EXPECT_EQ(OnTerminal({.clicolor_force = "0", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+  EXPECT_EQ(OffTerminal({.clicolor_force = "0"}), ColorMode::NoColor);
+
+  EXPECT_EQ(OnTerminal({.clicolor = "0", .term = "xterm-256color"}),
+            ColorMode::NoColor);
+  EXPECT_EQ(OnTerminal({.clicolor = "1", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+
+  // Forcing beats disabling.
+  EXPECT_EQ(OnTerminal({.clicolor_force = "1", .clicolor = "0"}),
+            ColorMode::Ansi16);
+}
+
+TEST(CapabilitiesTest, ColorDepthFromColorterm) {
+  EXPECT_EQ(OnTerminal({.colorterm = "truecolor", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.colorterm = "24bit", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.colorterm = "", .term = "xterm"}), ColorMode::Ansi16);
+
+  // `COLORTERM` can't enable color off a terminal, so a rich value inherited
+  // by a redirected stream can't smuggle escapes into it.
+  EXPECT_EQ(OffTerminal({.colorterm = "truecolor", .term = "xterm"}),
+            ColorMode::NoColor);
+}
+
+TEST(CapabilitiesTest, ColorDepthFromTermProgram) {
+  EXPECT_EQ(OnTerminal({.term_program = "vscode", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term_program = "iTerm.app", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term_program = "WarpTerminal", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term_program = "Hyper", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term_program = "Tabby", .term = "xterm"}),
+            ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term_program = "Terminus", .term = "xterm"}),
+            ColorMode::Truecolor);
+  // Apple's Terminal renders only the 256-color palette.
+  EXPECT_EQ(OnTerminal({.term_program = "Apple_Terminal", .term = "xterm"}),
+            ColorMode::Ansi256);
+  EXPECT_EQ(OnTerminal({.term_program = "unknown", .term = "xterm-256color"}),
+            ColorMode::Ansi256);
+}
+
+TEST(CapabilitiesTest, ColorDepthFromTerm) {
+  EXPECT_EQ(OnTerminal({.term = "xterm-kitty"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "alacritty"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "wezterm"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "foot-extra"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "xterm-direct"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "ghostty"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "contour-latest"}), ColorMode::Truecolor);
+  EXPECT_EQ(OnTerminal({.term = "xterm-truecolor"}), ColorMode::Truecolor);
+
+  EXPECT_EQ(OnTerminal({.term = "xterm-256color"}), ColorMode::Ansi256);
+  EXPECT_EQ(OnTerminal({.term = "screen-256color"}), ColorMode::Ansi256);
+  EXPECT_EQ(OnTerminal({.term = "putty-256"}), ColorMode::Ansi256);
+
+  // A terminal matching both a truecolor and a 256-color pattern takes the
+  // richer one, so the order these are tried in is load-bearing.
+  EXPECT_EQ(OnTerminal({.term = "vte-256color"}), ColorMode::Truecolor);
+
+  // Known to render color, but with nothing saying how much.
+  EXPECT_EQ(OnTerminal({.term = "xterm"}), ColorMode::Ansi16);
+  EXPECT_EQ(OnTerminal({.term = "linux"}), ColorMode::Ansi16);
+}
+
+TEST(CapabilitiesTest, Charset) {
+  EXPECT_EQ(ChooseCharset(Preference::Auto, "en_US.UTF-8"), Charset::Utf8);
+  EXPECT_EQ(ChooseCharset(Preference::Auto, "C.utf8"), Charset::Utf8);
+  EXPECT_EQ(ChooseCharset(Preference::Auto, "en_US.utf-8"), Charset::Utf8);
+
+  // Drawing box characters into a terminal decoding something else turns them
+  // into several bytes of mojibake and destroys the alignment they were for.
+  EXPECT_EQ(ChooseCharset(Preference::Auto, "C"), Charset::Ascii);
+  EXPECT_EQ(ChooseCharset(Preference::Auto, "POSIX"), Charset::Ascii);
+  EXPECT_EQ(ChooseCharset(Preference::Auto, "en_US.ISO-8859-1"),
+            Charset::Ascii);
+  EXPECT_EQ(ChooseCharset(Preference::Auto, ""), Charset::Ascii);
+
+  EXPECT_EQ(ChooseCharset(Preference::Never, "en_US.UTF-8"), Charset::Ascii);
+  EXPECT_EQ(ChooseCharset(Preference::Always, "C"), Charset::Utf8);
+}
+
+TEST(CapabilitiesTest, BackgroundFromColorFgBg) {
+  // `fg;bg`, which is what `rxvt` and its derivatives write.
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "15;0"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "0;15"),
+            Background::Light);
+  // The eighth entry is white and the ninth the dark gray after it, so the
+  // halves are not simply the low and high eight.
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "0;7"),
+            Background::Light);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "15;8"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "15;6"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "0;9"),
+            Background::Light);
+  // Some terminals write a third field between the two.
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "15;default;0"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "0;default;15"),
+            Background::Light);
+}
+
+TEST(CapabilitiesTest, BackgroundWithNothingToGoOn) {
+  // Unset, unparsable, and out of range all say nothing, and what nothing
+  // gets is the assumption that costs contrast rather than legibility.
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, ""), Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "default;default"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "0;99"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "15"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Auto, "15;"),
+            Background::Dark);
+}
+
+TEST(CapabilitiesTest, BackgroundPreferenceWins) {
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Dark, "0;15"),
+            Background::Dark);
+  EXPECT_EQ(ChooseBackground(BackgroundPreference::Light, "15;0"),
+            Background::Light);
+}
+
+TEST(CapabilitiesTest, Defaults) {
+  // The defaults describe a plain-text sink, which is what a file or a pipe
+  // gets and what tests should use unless exercising something richer.
+  Capabilities capabilities;
+  EXPECT_EQ(capabilities.color_mode, ColorMode::NoColor);
+  EXPECT_EQ(capabilities.charset, Charset::Ascii);
+  EXPECT_EQ(capabilities.background, Background::Dark);
+  EXPECT_FALSE(capabilities.is_terminal);
+  EXPECT_FALSE(capabilities.columns.has_value());
+}
+
+TEST(CapabilitiesTest, Detect) {
+  // Detection reads the process environment and the descriptor it is handed, so
+  // only what neither can change is pinned here. What the policy decides from
+  // given inputs is tested above, against `ChooseColorMode` and `ChooseCharset`
+  // directly.
+  //
+  // It detects against a file rather than the process's own streams: those are
+  // a pipe under the test runner but a terminal under a debugger, and an
+  // exported `FORCE_COLOR` turns color on for either.
+  auto dir = Filesystem::MakeTmpDir();
+  ASSERT_TRUE(dir.ok()) << dir.error();
+  auto file = dir->OpenWriteOnly("out", Filesystem::CreationOptions::CreateNew);
+  ASSERT_TRUE(file.ok()) << file.error();
+
+  Capabilities capabilities = Capabilities::Detect(*file);
+  // A file is never a terminal.
+  EXPECT_FALSE(capabilities.is_terminal);
+  // `COLUMNS` reaches detection from the environment, so whether a width is
+  // found depends on it, but one that is found is usable.
+  if (capabilities.columns) {
+    EXPECT_GT(*capabilities.columns, 0);
+  }
+  EXPECT_GT(capabilities.tab_width, 0);
+
+  // A preference decides on its own, whatever the environment holds. Color
+  // forced on picks a depth from the environment, so only that it is on can be
+  // pinned here.
+  EXPECT_EQ(Capabilities::Detect(
+                *file, {.color = Preference::Never, .utf8 = Preference::Never})
+                .color_mode,
+            ColorMode::NoColor);
+  EXPECT_NE(
+      Capabilities::Detect(*file, {.color = Preference::Always}).color_mode,
+      ColorMode::NoColor);
+  EXPECT_EQ(Capabilities::Detect(*file, {.utf8 = Preference::Always}).charset,
+            Charset::Utf8);
+  EXPECT_EQ(Capabilities::Detect(*file, {.utf8 = Preference::Never}).charset,
+            Charset::Ascii);
+  EXPECT_EQ(
+      Capabilities::Detect(*file, {.background = BackgroundPreference::Light})
+          .background,
+      Background::Light);
+  EXPECT_EQ(
+      Capabilities::Detect(*file, {.background = BackgroundPreference::Dark})
+          .background,
+      Background::Dark);
+
+  (*std::move(file)).Close().Check();
+}
+
+}  // namespace
+}  // namespace Carbon::Terminal

common/terminal/color.cpp

@@ -0,0 +1,239 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/color.h"
+
+#include <algorithm>
+#include <array>
+
+#include "common/check.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Format.h"
+
+namespace Carbon::Terminal {
+
+static constexpr int AnsiColorCount = 16;
+
+// Reference values for the 16 ANSI colors.
+//
+// Nothing standardizes these: a terminal draws them from the user's palette,
+// which is exactly what makes them worth using. But downsampling an RGB color
+// still needs some notion of where each named color sits, so these use the
+// xterm defaults, which terminals vary from but stay recognizably near.
+static constexpr std::array<Color::RgbValue, AnsiColorCount> AnsiColorRgbs = {{
+    {.r = 0, .g = 0, .b = 0},        // Black
+    {.r = 205, .g = 0, .b = 0},      // Red
+    {.r = 0, .g = 205, .b = 0},      // Green
+    {.r = 205, .g = 205, .b = 0},    // Yellow
+    {.r = 0, .g = 0, .b = 238},      // Blue
+    {.r = 205, .g = 0, .b = 205},    // Magenta
+    {.r = 0, .g = 205, .b = 205},    // Cyan
+    {.r = 229, .g = 229, .b = 229},  // White
+    {.r = 127, .g = 127, .b = 127},  // BrightBlack
+    {.r = 255, .g = 0, .b = 0},      // BrightRed
+    {.r = 0, .g = 255, .b = 0},      // BrightGreen
+    {.r = 255, .g = 255, .b = 0},    // BrightYellow
+    {.r = 92, .g = 92, .b = 255},    // BrightBlue
+    {.r = 255, .g = 0, .b = 255},    // BrightMagenta
+    {.r = 0, .g = 255, .b = 255},    // BrightCyan
+    {.r = 255, .g = 255, .b = 255},  // BrightWhite
+}};
+
+static constexpr std::array<llvm::StringRef, AnsiColorCount> AnsiColorNames = {
+    "Black",       "Red",           "Green",       "Yellow",
+    "Blue",        "Magenta",       "Cyan",        "White",
+    "BrightBlack", "BrightRed",     "BrightGreen", "BrightYellow",
+    "BrightBlue",  "BrightMagenta", "BrightCyan",  "BrightWhite"};
+
+// Returns the "redmean" distance between two colors, squared and scaled by 256
+// to keep it in integer arithmetic.
+//
+// Treating the channels as orthogonal axes is cheaper but sits a long way from
+// perceived difference, and downsampling is exactly where that shows: a color
+// picked for a diagnostic lands on whichever of a small fixed set the
+// arithmetic says is closest, and a plain Euclidean fit underweights green,
+// where the eye is most sensitive. Redmean weights the
+// channels by where the pair sits on the red axis, which tracks perception far
+// better for a couple of extra multiplies:
+// https://en.wikipedia.org/wiki/Color_difference#sRGB
+//
+// The formula ends in a square root, which is dropped because only the ordering
+// is used. Scaling by 256 turns the two fractional weights into integers; the
+// result peaks just under 150 million, well inside the range.
+static auto DistanceSquared(Color::RgbValue lhs, Color::RgbValue rhs) -> int {
+  int red_mean = (static_cast<int>(lhs.r) + static_cast<int>(rhs.r)) / 2;
+  int dr = static_cast<int>(lhs.r) - static_cast<int>(rhs.r);
+  int dg = static_cast<int>(lhs.g) - static_cast<int>(rhs.g);
+  int db = static_cast<int>(lhs.b) - static_cast<int>(rhs.b);
+  return (512 + red_mean) * dr * dr + 1024 * dg * dg +
+         (767 - red_mean) * db * db;
+}
+
+// Returns the ANSI color whose reference value is nearest to `rgb`.
+static auto NearestAnsiColor(Color::RgbValue rgb) -> AnsiColor {
+  int best_index = 0;
+  int best_distance = DistanceSquared(rgb, AnsiColorRgbs[0]);
+  for (int i = 1; i < AnsiColorCount; ++i) {
+    int distance = DistanceSquared(rgb, AnsiColorRgbs[i]);
+    if (distance < best_distance) {
+      best_distance = distance;
+      best_index = i;
+    }
+  }
+  return static_cast<AnsiColor>(best_index);
+}
+
+// The channel values of the 6x6x6 color cube at palette indices 16 through
+// 231. The first step is much larger than the rest, so a channel can't be
+// rounded to the nearest level by dividing.
+static constexpr std::array<uint8_t, 6> CubeLevels = {0,   95,  135,
+                                                      175, 215, 255};
+
+// The midpoints between adjacent entries of `CubeLevels`, which are where the
+// nearest level changes.
+static constexpr std::array<uint8_t, 5> CubeLevelMidpoints = {48, 115, 155, 195,
+                                                              235};
+
+static_assert(
+    [] {
+      for (size_t i = 0; i < CubeLevelMidpoints.size(); ++i) {
+        // Rounded up, so that a value exactly between two levels takes the
+        // higher one.
+        if (CubeLevelMidpoints[i] !=
+            (CubeLevels[i] + CubeLevels[i + 1] + 1) / 2) {
+          return false;
+        }
+      }
+      return true;
+    }(),
+    "Midpoints must stay in step with the levels they separate.");
+
+// Returns the index into `CubeLevels` of the level nearest `value`.
+static auto NearestCubeLevel(uint8_t value) -> int {
+  int level = 0;
+  while (level < static_cast<int>(CubeLevelMidpoints.size()) &&
+         value >= CubeLevelMidpoints[level]) {
+    ++level;
+  }
+  return level;
+}
+
+// Returns the 256-color palette index whose color is nearest to `rgb`.
+static auto NearestPaletteIndex(Color::RgbValue rgb) -> uint8_t {
+  // Only the color cube and the gray ramp are considered. Indices 0 through 15
+  // alias the ANSI colors, whose appearance comes from the user's palette, so
+  // an exact RGB request must never be answered with one.
+  int r_level = NearestCubeLevel(rgb.r);
+  int g_level = NearestCubeLevel(rgb.g);
+  int b_level = NearestCubeLevel(rgb.b);
+  Color::RgbValue cube = {.r = CubeLevels[r_level],
+                          .g = CubeLevels[g_level],
+                          .b = CubeLevels[b_level]};
+
+  // The gray ramp at indices 232 through 255 runs from 8 to 238 in steps of
+  // 10, and is finer than the cube's gray diagonal for near-neutral colors.
+  int average = (static_cast<int>(rgb.r) + static_cast<int>(rgb.g) +
+                 static_cast<int>(rgb.b)) /
+                3;
+  int gray_step = std::clamp((average - 8 + 5) / 10, 0, 23);
+  auto gray_value = static_cast<uint8_t>(8 + 10 * gray_step);
+  Color::RgbValue gray = {.r = gray_value, .g = gray_value, .b = gray_value};
+
+  if (DistanceSquared(rgb, gray) < DistanceSquared(rgb, cube)) {
+    return 232 + gray_step;
+  }
+  return 16 + 36 * r_level + 6 * g_level + b_level;
+}
+
+// Returns the SGR parameter selecting `color` for `target`.
+//
+// The original ANSI codes cover the first eight colors, and the later "bright"
+// codes cover the rest at a fixed offset.
+static auto AnsiSgrCode(AnsiColor color, ColorTarget target) -> uint8_t {
+  CARBON_CHECK(target != ColorTarget::Underline,
+               "Underline color has no direct ANSI form.");
+  int index = static_cast<int>(color);
+  int base = target == ColorTarget::Background ? 40 : 30;
+  if (index >= 8) {
+    // Bright foregrounds are 90-97 and bright backgrounds 100-107.
+    base += 60;
+  }
+  return base + (index % 8);
+}
+
+// Returns the SGR parameter introducing an extended color for `target`, which
+// is followed by either `;5;<index>` or `;2;<r>;<g>;<b>`.
+static auto ExtendedSgrCode(ColorTarget target) -> uint8_t {
+  switch (target) {
+    case ColorTarget::Foreground:
+      return 38;
+    case ColorTarget::Background:
+      return 48;
+    case ColorTarget::Underline:
+      return 58;
+  }
+}
+
+auto Color::AppendEscape(OutputBufferRef out, ColorMode mode,
+                         ColorTarget target) const -> void {
+  if (mode == ColorMode::NoColor) {
+    return;
+  }
+
+  // Underline colors are only expressible through the extended-color escape,
+  // which `Ansi16` doesn't use.
+  if (target == ColorTarget::Underline && mode == ColorMode::Ansi16) {
+    return;
+  }
+
+  CARBON_CHECK(is_set(), "Only a color that is set can be selected.");
+
+  if (kind_ == Kind::Ansi) {
+    if (target == ColorTarget::Underline) {
+      // Named underline colors go through the palette form of the extended
+      // escape, as there is no direct code for them.
+      out.Append("\x1b[58;5;", static_cast<uint8_t>(ansi()), "m");
+    } else {
+      out.Append("\x1b[", AnsiSgrCode(ansi(), target), "m");
+    }
+    return;
+  }
+
+  switch (mode) {
+    case ColorMode::Truecolor:
+      out.Append("\x1b[", ExtendedSgrCode(target), ";2;", channels_.r, ";",
+                 channels_.g, ";", channels_.b, "m");
+      break;
+
+    case ColorMode::Ansi256:
+      out.Append("\x1b[", ExtendedSgrCode(target), ";5;",
+                 NearestPaletteIndex(channels_), "m");
+      break;
+
+    case ColorMode::Ansi16:
+      out.Append("\x1b[", AnsiSgrCode(NearestAnsiColor(channels_), target),
+                 "m");
+      break;
+
+    case ColorMode::NoColor:
+      CARBON_FATAL("Returned above without emitting anything.");
+  }
+}
+
+auto Color::Print(llvm::raw_ostream& out) const -> void {
+  switch (kind_) {
+    case Kind::None:
+      out << "None";
+      return;
+    case Kind::Ansi:
+      out << AnsiColorNames[static_cast<int>(ansi())];
+      return;
+    case Kind::Rgb:
+      out << llvm::format("#%02x%02x%02x", channels_.r, channels_.g,
+                          channels_.b);
+      return;
+  }
+}
+
+}  // namespace Carbon::Terminal

common/terminal/color.h

@@ -0,0 +1,167 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_TERMINAL_COLOR_H_
+#define CARBON_COMMON_TERMINAL_COLOR_H_
+
+#include <cstdint>
+
+#include "common/check.h"
+#include "common/ostream.h"
+#include "common/terminal/output_buffer_ref.h"
+
+namespace Carbon::Terminal {
+
+// The color escape sequences a terminal understands.
+//
+// Colors, like the other text attributes, are selected with Select Graphic
+// Rendition (SGR) escape sequences. The sequence and the codes for the first
+// eight colors come from ECMA-48, published in parallel as ANSI X3.64;
+// terminal emulators added the bright variants, the 256-color palette, and the
+// 24-bit form:
+// https://ecma-international.org/publications-and-standards/standards/ecma-48/
+//
+// These form a ladder: each mode can express everything the modes before it
+// can. Colors that the active mode can't express exactly are downsampled to
+// the nearest color it can, so callers author in the richest form and let
+// rendering degrade on its own.
+enum class ColorMode : int8_t {
+  // Emit no escape sequences at all, producing plain text.
+  NoColor,
+  // The 16 colors with SGR codes of their own.
+  Ansi16,
+  // The 256-color palette: the 16 ANSI colors, a 6x6x6 RGB cube, and a 24-step
+  // gray ramp.
+  Ansi256,
+  // Direct 24-bit RGB, commonly called "truecolor".
+  Truecolor,
+};
+
+// The 16 colors with SGR codes of their own.
+//
+// Terminals render these through the user's configured palette, which makes
+// them the right choice for output that should blend with the user's theme.
+// The tradeoff is that their rendered appearance is outside our control: a
+// user's "red" may be any color at all.
+enum class AnsiColor : uint8_t {
+  Black,
+  Red,
+  Green,
+  Yellow,
+  Blue,
+  Magenta,
+  Cyan,
+  White,
+  BrightBlack,
+  BrightRed,
+  BrightGreen,
+  BrightYellow,
+  BrightBlue,
+  BrightMagenta,
+  BrightCyan,
+  BrightWhite,
+};
+
+// Which part of a cell's rendering a color applies to.
+enum class ColorTarget : int8_t {
+  Foreground,
+  Background,
+  // The color of the underline itself, independent of the foreground. Only
+  // `Ansi256` and richer modes can express this.
+  Underline,
+};
+
+// A color to render with: one of the 16 named ANSI colors, a 24-bit RGB value,
+// or no color at all.
+//
+// RGB colors render exactly where the terminal supports them, and are
+// downsampled where it doesn't. Downsampling to `Ansi16` measures distance
+// against fixed reference values, but the terminal renders the result from the
+// user's palette, so a downsampled color can land far from the original.
+// Prefer `AnsiColor` wherever output should track the user's theme, and RGB
+// only where an exact color matters.
+//
+// A default-constructed color selects nothing. That is how a `Style` spells
+// leaving one of its colors to the terminal, so this is a value with an empty
+// state rather than something wrapped in an `optional` to get one.
+class Color : public Printable<Color> {
+ public:
+  // Whether a color names a palette entry, gives channel values directly, or
+  // selects nothing.
+  enum class Kind : uint8_t {
+    None,
+    Ansi,
+    Rgb,
+  };
+
+  // The channel values of a 24-bit color.
+  struct RgbValue {
+    uint8_t r;
+    uint8_t g;
+    uint8_t b;
+
+    friend auto operator==(RgbValue lhs, RgbValue rhs) -> bool = default;
+  };
+
+  constexpr Color() = default;
+
+  // Colors convert implicitly from `AnsiColor` so that call sites can read as
+  // `style.Foreground(AnsiColor::Red)`.
+  //
+  // NOLINTNEXTLINE(google-explicit-constructor)
+  constexpr Color(AnsiColor ansi)
+      : kind_(Kind::Ansi), channels_{.r = static_cast<uint8_t>(ansi)} {}
+
+  constexpr Color(uint8_t r, uint8_t g, uint8_t b)
+      : kind_(Kind::Rgb), channels_{.r = r, .g = g, .b = b} {}
+
+  auto kind() const -> Kind { return kind_; }
+
+  // Returns whether this selects a color at all.
+  auto is_set() const -> bool { return kind_ != Kind::None; }
+
+  // Returns the named color. Valid only when `kind()` is `Ansi`.
+  auto ansi() const -> AnsiColor {
+    CARBON_CHECK(kind_ == Kind::Ansi,
+                 "Only a named color has a palette index.");
+    return static_cast<AnsiColor>(channels_.r);
+  }
+
+  // Returns the channel values. Valid only when `kind()` is `Rgb`.
+  auto rgb() const -> RgbValue {
+    CARBON_CHECK(kind_ == Kind::Rgb, "Only an RGB color has channel values.");
+    return channels_;
+  }
+
+  // Appends the escape sequence selecting this color for `target`, which
+  // requires that one is set.
+  //
+  // Appends nothing when `mode` is `NoColor`, or when `target` is `Underline`
+  // and `mode` is `Ansi16`, which has no way to express an underline color.
+  auto AppendEscape(OutputBufferRef out, ColorMode mode,
+                    ColorTarget target) const -> void;
+
+  auto Print(llvm::raw_ostream& out) const -> void;
+
+  // Written out rather than defaulted because the `Printable` base has no
+  // comparison of its own, which would leave a defaulted one deleted.
+  friend auto operator==(Color lhs, Color rhs) -> bool {
+    return lhs.kind_ == rhs.kind_ && lhs.channels_ == rhs.channels_;
+  }
+
+ private:
+  Kind kind_ = Kind::None;
+
+  // The palette index in `r` with the rest zero for `Ansi`, the channel values
+  // for `Rgb`, and all zero for `None`.
+  //
+  // Overlapping the two in a union would leave the bytes past a palette index
+  // unwritten. Every byte carrying part of the value is what lets a whole
+  // `Style` be compared as bytes, and an index fits in a channel anyway.
+  RgbValue channels_ = {.r = 0, .g = 0, .b = 0};
+};
+
+}  // namespace Carbon::Terminal
+
+#endif  // CARBON_COMMON_TERMINAL_COLOR_H_

common/terminal/color_test.cpp

@@ -0,0 +1,171 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/color.h"
+
+#include <gtest/gtest.h>
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/StringExtras.h"
+
+namespace Carbon::Terminal {
+namespace {
+
+auto Escape(Color color, ColorMode mode,
+            ColorTarget target = ColorTarget::Foreground) -> std::string {
+  llvm::SmallString<32> escape;
+  color.AppendEscape(escape, mode, target);
+  return std::string(escape);
+}
+
+TEST(ColorTest, AnsiEscapes) {
+  // Named colors use their own SGR codes rather than the extended forms, in
+  // every mode that has color at all, so the terminal renders them from the
+  // user's palette.
+  for (ColorMode mode :
+       {ColorMode::Ansi16, ColorMode::Ansi256, ColorMode::Truecolor}) {
+    EXPECT_EQ(Escape(AnsiColor::Red, mode), "\x1b[31m");
+    EXPECT_EQ(Escape(AnsiColor::Black, mode), "\x1b[30m");
+    EXPECT_EQ(Escape(AnsiColor::BrightCyan, mode), "\x1b[96m");
+    EXPECT_EQ(Escape(AnsiColor::Red, mode, ColorTarget::Background),
+              "\x1b[41m");
+    EXPECT_EQ(Escape(AnsiColor::BrightWhite, mode, ColorTarget::Background),
+              "\x1b[107m");
+  }
+
+  EXPECT_EQ(Escape(AnsiColor::Red, ColorMode::NoColor), "");
+}
+
+TEST(ColorTest, RgbEscapes) {
+  Color red(255, 0, 0);
+  EXPECT_EQ(Escape(red, ColorMode::Truecolor), "\x1b[38;2;255;0;0m");
+  EXPECT_EQ(Escape(red, ColorMode::Truecolor, ColorTarget::Background),
+            "\x1b[48;2;255;0;0m");
+  EXPECT_EQ(Escape(red, ColorMode::Ansi256), "\x1b[38;5;196m");
+  EXPECT_EQ(Escape(red, ColorMode::Ansi16), "\x1b[91m");
+  EXPECT_EQ(Escape(red, ColorMode::NoColor), "");
+}
+
+TEST(ColorTest, UnderlineEscapes) {
+  // Underline colors only exist in the extended-color escapes, so in `Ansi16`
+  // the terminal draws the underline in the foreground color.
+  EXPECT_EQ(
+      Escape(AnsiColor::Red, ColorMode::Truecolor, ColorTarget::Underline),
+      "\x1b[58;5;1m");
+  EXPECT_EQ(Escape(AnsiColor::Red, ColorMode::Ansi256, ColorTarget::Underline),
+            "\x1b[58;5;1m");
+  EXPECT_EQ(Escape(AnsiColor::Red, ColorMode::Ansi16, ColorTarget::Underline),
+            "");
+
+  Color green(0, 255, 0);
+  EXPECT_EQ(Escape(green, ColorMode::Truecolor, ColorTarget::Underline),
+            "\x1b[58;2;0;255;0m");
+  EXPECT_EQ(Escape(green, ColorMode::Ansi256, ColorTarget::Underline),
+            "\x1b[58;5;46m");
+  EXPECT_EQ(Escape(green, ColorMode::Ansi16, ColorTarget::Underline), "");
+}
+
+TEST(ColorTest, DownsampleToAnsi16) {
+  // The reference value of each ANSI color must come back as that color, or
+  // downsampling would shift colors that were already expressible. Spelling
+  // the values out here rather than reading them back from the same table the
+  // implementation uses is what makes this catch a wrong table.
+  struct Expected {
+    Color color;
+    llvm::StringRef escape;
+  };
+  Expected cases[] = {
+      {Color(0, 0, 0), "\x1b[30m"},       {Color(205, 0, 0), "\x1b[31m"},
+      {Color(0, 205, 0), "\x1b[32m"},     {Color(205, 205, 0), "\x1b[33m"},
+      {Color(0, 0, 238), "\x1b[34m"},     {Color(205, 0, 205), "\x1b[35m"},
+      {Color(0, 205, 205), "\x1b[36m"},   {Color(229, 229, 229), "\x1b[37m"},
+      {Color(127, 127, 127), "\x1b[90m"}, {Color(255, 0, 0), "\x1b[91m"},
+      {Color(0, 255, 0), "\x1b[92m"},     {Color(255, 255, 0), "\x1b[93m"},
+      {Color(92, 92, 255), "\x1b[94m"},   {Color(255, 0, 255), "\x1b[95m"},
+      {Color(0, 255, 255), "\x1b[96m"},   {Color(255, 255, 255), "\x1b[97m"},
+  };
+  for (const Expected& expected : cases) {
+    EXPECT_EQ(Escape(expected.color, ColorMode::Ansi16), expected.escape)
+        << expected.color;
+  }
+
+  // Colors between the reference values land on the nearest one.
+  EXPECT_EQ(Escape(Color(250, 10, 10), ColorMode::Ansi16), "\x1b[91m");
+  EXPECT_EQ(Escape(Color(10, 10, 10), ColorMode::Ansi16), "\x1b[30m");
+  EXPECT_EQ(Escape(Color(120, 120, 120), ColorMode::Ansi16), "\x1b[90m");
+}
+
+TEST(ColorTest, DownsampleToPalette) {
+  // The corners of the 6x6x6 cube are exactly representable.
+  EXPECT_EQ(Escape(Color(0, 0, 0), ColorMode::Ansi256), "\x1b[38;5;16m");
+  EXPECT_EQ(Escape(Color(255, 255, 255), ColorMode::Ansi256), "\x1b[38;5;231m");
+  EXPECT_EQ(Escape(Color(255, 0, 0), ColorMode::Ansi256), "\x1b[38;5;196m");
+  EXPECT_EQ(Escape(Color(0, 0, 255), ColorMode::Ansi256), "\x1b[38;5;21m");
+
+  // The cube's levels are unevenly spaced, so rounding has to account for that
+  // rather than divide: 95 and 135 are adjacent levels only 40 apart.
+  EXPECT_EQ(Escape(Color(95, 0, 0), ColorMode::Ansi256), "\x1b[38;5;52m");
+  EXPECT_EQ(Escape(Color(130, 0, 0), ColorMode::Ansi256), "\x1b[38;5;88m");
+
+  // Near-neutral colors land on the gray ramp, which is far finer than the
+  // cube's diagonal, except at the ends where the cube wins.
+  EXPECT_EQ(Escape(Color(8, 8, 8), ColorMode::Ansi256), "\x1b[38;5;232m");
+  EXPECT_EQ(Escape(Color(128, 128, 128), ColorMode::Ansi256), "\x1b[38;5;244m");
+  EXPECT_EQ(Escape(Color(238, 238, 238), ColorMode::Ansi256), "\x1b[38;5;255m");
+}
+
+TEST(ColorTest, DownsampleAvoidsPaletteEntries) {
+  // Indices 0 through 15 render from the user's palette, so an exact RGB
+  // request must never be answered with one.
+  for (int r = 0; r < 256; r += 17) {
+    for (int g = 0; g < 256; g += 17) {
+      for (int b = 0; b < 256; b += 17) {
+        Color color(r, g, b);
+        std::string escape = Escape(color, ColorMode::Ansi256);
+        int index = 0;
+        ASSERT_TRUE(llvm::to_integer(
+            llvm::StringRef(escape).drop_front(7).drop_back(1), index))
+            << color;
+        EXPECT_GE(index, 16) << color;
+      }
+    }
+  }
+}
+
+TEST(ColorTest, Equality) {
+  EXPECT_EQ(Color(AnsiColor::Red), Color(AnsiColor::Red));
+  EXPECT_NE(Color(AnsiColor::Red), Color(AnsiColor::Blue));
+  EXPECT_EQ(Color(1, 2, 3), Color(1, 2, 3));
+  EXPECT_NE(Color(1, 2, 3), Color(1, 2, 4));
+
+  // A named color and its reference value are different colors: the terminal
+  // renders one from the palette and the other exactly.
+  EXPECT_NE(Color(AnsiColor::BrightRed), Color(255, 0, 0));
+
+  // A palette index occupies the same byte as the red channel, so these pairs
+  // hold identical channel bytes and are told apart only by their kind.
+  EXPECT_NE(Color(AnsiColor::Red), Color(1, 0, 0));
+  EXPECT_NE(Color(AnsiColor::Black), Color(0, 0, 0));
+  EXPECT_NE(Color(AnsiColor::Black), Color());
+  EXPECT_NE(Color(0, 0, 0), Color());
+  EXPECT_EQ(Color(), Color());
+}
+
+TEST(ColorTest, Unset) {
+  EXPECT_FALSE(Color().is_set());
+  EXPECT_EQ(Color().kind(), Color::Kind::None);
+
+  // Black is a color like any other, however little of it there is.
+  EXPECT_TRUE(Color(AnsiColor::Black).is_set());
+  EXPECT_TRUE(Color(0, 0, 0).is_set());
+}
+
+TEST(ColorTest, Print) {
+  EXPECT_EQ(PrintToString(Color(AnsiColor::BrightMagenta)), "BrightMagenta");
+  EXPECT_EQ(PrintToString(Color(0x12, 0xab, 0xff)), "#12abff");
+  EXPECT_EQ(PrintToString(Color()), "None");
+}
+
+}  // namespace
+}  // namespace Carbon::Terminal

common/terminal/metrics.cpp

@@ -0,0 +1,190 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/metrics.h"
+
+#include "common/check.h"
+#include "llvm/Support/ConvertUTF.h"
+#include "llvm/Support/Unicode.h"
+
+namespace Carbon::Terminal {
+
+// Stands in for anything a UTF-8 terminal has no rendering for: invalid UTF-8,
+// control characters, and unassigned code points.
+static constexpr char32_t Utf8Replacement = U'�';
+
+// Returns whether an ASCII terminal renders `code_point` as itself, in one
+// column.
+static auto IsPrintableAscii(char32_t code_point) -> bool {
+  return code_point >= 0x20 && code_point < 0x7f;
+}
+
+// Spelled out rather than handed to a general converter, which walks a range
+// and checks bounds this already knows. Box-drawing characters go through here
+// for every cell of every line drawn.
+//
+// TODO: Offer this to LLVM, whose `ConvertCodePointToUTF8` is the general
+// converter this replaces. Encoding one code point at a time is what anything
+// writing UTF-8 out of a grid does, so this belongs beside it rather than
+// here; drop this once it is there.
+auto EncodeUtf8(char32_t code_point, Utf8Storage& storage) -> llvm::StringRef {
+  // Most of what gets rendered is ASCII, and encoding it is a single byte.
+  if (code_point < 0x80) {
+    storage[0] = static_cast<char>(code_point);
+    return llvm::StringRef(storage.data(), 1);
+  }
+
+  // Surrogates have no encoding of their own, and nothing past the last code
+  // point has one at all.
+  if (code_point > 0x10ffff || (code_point >= 0xd800 && code_point < 0xe000)) {
+    code_point = Utf8Replacement;
+  }
+
+  auto trailing = [code_point](int shift) {
+    return static_cast<char>(0b1000'0000 | ((code_point >> shift) & 0b11'1111));
+  };
+  if (code_point < 0x800) {
+    storage[0] = static_cast<char>(0b1100'0000 | (code_point >> 6));
+    storage[1] = trailing(0);
+    return llvm::StringRef(storage.data(), 2);
+  }
+  if (code_point < 0x10000) {
+    storage[0] = static_cast<char>(0b1110'0000 | (code_point >> 12));
+    storage[1] = trailing(6);
+    storage[2] = trailing(0);
+    return llvm::StringRef(storage.data(), 3);
+  }
+  storage[0] = static_cast<char>(0b1111'0000 | (code_point >> 18));
+  storage[1] = trailing(12);
+  storage[2] = trailing(6);
+  storage[3] = trailing(0);
+  return llvm::StringRef(storage.data(), 4);
+}
+
+// Returns the columns `code_point` occupies on a UTF-8 terminal: zero for a
+// combining mark, one or two for one with a glyph of its own, and a
+// negative value when there is no printable rendering for it.
+//
+// TODO: This encodes a code point only for LLVM to decode it again.
+// `llvm::sys::unicode::charWidth` computes exactly this and is what
+// `columnWidthUTF8` calls once per code point, but it is file-local to LLVM's
+// `Unicode.cpp`. Exposing it there would let this call it directly. LLVM's own
+// contract already says a string's width is the sum of its code points', so
+// there is nothing in the way of it.
+static auto Utf8CodePointWidth(char32_t code_point) -> int {
+  // Printable ASCII is one column, and is most of what gets measured. The
+  // general path parses a UTF-8 sequence and searches several code point
+  // range tables, which is far more than this needs.
+  if (IsPrintableAscii(code_point)) {
+    return 1;
+  }
+
+  Utf8Storage storage;
+  return llvm::sys::unicode::columnWidthUTF8(EncodeUtf8(code_point, storage));
+}
+
+// Removes the first UTF-8 sequence from `text` and returns the code point it
+// encodes.
+static auto TakeUtf8CodePoint(llvm::StringRef& text) -> char32_t {
+  const auto* begin = reinterpret_cast<const llvm::UTF8*>(text.data());
+  const auto* pos = begin;
+  llvm::UTF32 code_point = 0;
+  if (llvm::convertUTF8Sequence(&pos, begin + text.size(), &code_point,
+                                llvm::strictConversion) != llvm::conversionOK) {
+    text = text.drop_front(1);
+    return Utf8Replacement;
+  }
+  text = text.drop_front(pos - begin);
+  return code_point;
+}
+
+auto Metrics::TakeCodePoint(llvm::StringRef& text) const -> char32_t {
+  CARBON_CHECK(!text.empty(), "No code point to take.");
+  if (charset_ == Charset::Ascii) {
+    auto byte = static_cast<unsigned char>(text.front());
+    text = text.drop_front();
+    return byte;
+  }
+  return TakeUtf8CodePoint(text);
+}
+
+auto Metrics::CodePointWidth(char32_t code_point) const -> int {
+  if (charset_ == Charset::Ascii) {
+    return 1;
+  }
+  int width = Utf8CodePointWidth(code_point);
+  // A code point with no rendering is drawn as the replacement character, which
+  // takes one column.
+  return width < 0 ? 1 : width;
+}
+
+auto Metrics::RenderedCodePoint(char32_t code_point) const -> char32_t {
+  // Printable ASCII is most of what gets drawn, and settling it here keeps it
+  // out of the range tables the general answer searches.
+  if (IsPrintableAscii(code_point)) {
+    return code_point;
+  }
+
+  // Fallback if we can't use unicode.
+  if (charset_ == Charset::Ascii) {
+    return U'?';
+  }
+
+  // Which code points have no rendering is what a negative width names as well,
+  // asked directly rather than through a width that has to encode one to
+  // answer.
+  return llvm::sys::unicode::isPrintable(static_cast<int>(code_point))
+             ? code_point
+             : Utf8Replacement;
+}
+
+auto Metrics::Width(llvm::StringRef text) const -> int {
+  // Checked rather than debug-checked: text with one of these in it measures as
+  // though each took one column, which is not what drawing does, and measuring
+  // wrong is invisible in the output. The scan is one more linear pass over
+  // text that is walked linearly anyway.
+  CARBON_CHECK(
+      text.find_first_of("\t\n\r") == llvm::StringRef::npos,
+      "Width is only for text whose width is its code points', but got `{0}`.",
+      text);
+  if (charset_ == Charset::Ascii) {
+    return static_cast<int>(text.size());
+  }
+
+  // Text that is valid UTF-8 throughout and printable throughout is the common
+  // case, and LLVM measures a whole run of it in one pass. It answers with a
+  // negative value rather than a width when the text holds anything it can't
+  // measure, which is what the walk below is for: each such code point still
+  // takes the one column the replacement character drawn for it will.
+  int width = llvm::sys::unicode::columnWidthUTF8(text);
+  if (width >= 0) {
+    return width;
+  }
+
+  width = 0;
+  while (!text.empty()) {
+    width += CodePointWidth(TakeUtf8CodePoint(text));
+  }
+  return width;
+}
+
+auto Metrics::TakeColumns(llvm::StringRef& text, int columns) const
+    -> llvm::StringRef {
+  llvm::StringRef rest = text;
+  int taken = 0;
+  while (!rest.empty()) {
+    llvm::StringRef next = rest;
+    int width = CodePointWidth(TakeCodePoint(next));
+    if (taken + width > columns) {
+      break;
+    }
+    taken += width;
+    rest = next;
+  }
+  llvm::StringRef prefix = text.drop_back(rest.size());
+  text = rest;
+  return prefix;
+}
+
+}  // namespace Carbon::Terminal

common/terminal/metrics.h

@@ -0,0 +1,114 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_TERMINAL_METRICS_H_
+#define CARBON_COMMON_TERMINAL_METRICS_H_
+
+#include <array>
+#include <cstddef>
+
+#include "common/terminal/capabilities.h"
+#include "llvm/ADT/StringRef.h"
+
+namespace Carbon::Terminal {
+
+// The most bytes one code point encodes to in UTF-8, and storage for one.
+inline constexpr size_t MaxUtf8Bytes = 4;
+using Utf8Storage = std::array<char, MaxUtf8Bytes>;
+
+// Encodes `code_point` as UTF-8 into `storage`, returning the bytes written.
+//
+// Code points with no valid encoding, including surrogates and anything past
+// U+10FFFF, become the replacement character.
+auto EncodeUtf8(char32_t code_point, Utf8Storage& storage) -> llvm::StringRef;
+
+// How many columns a terminal spends on text, given the charset it decodes
+// with.
+//
+// Which bytes make up a column depends on the charset, so every question about
+// the size of text is a question about the charset as well, and this is what
+// answers both at once. `Buffer` holds one and lays its cells out with it;
+// anything deciding where to put something asks one directly rather than
+// keeping its own idea of how wide a string is.
+//
+// Nothing here converts between a byte offset and a column: which byte a column
+// lands on depends on the encoding, and which column a byte lands in depends on
+// the width of everything before it. `TakeColumns` hands back the text it cut
+// rather than an offset into it, so a caller never holds one count where the
+// other belongs.
+//
+// TODO: Every width here is a sum over code points taken in logical order,
+// which is only the width on screen for left-to-right text. Bidirectional text
+// reorders, so a run's width still adds up but `TakeColumns` has no meaning:
+// the prefix occupying the first N columns need not be a prefix of the string.
+// Settle this together with the question `Buffer`'s own TODO describes, since
+// both turn on what a client hands over.
+class Metrics {
+ public:
+  explicit constexpr Metrics(Charset charset) : charset_(charset) {}
+
+  constexpr auto charset() const -> Charset { return charset_; }
+
+  // Removes the next code point from `text`, which must not be empty, and
+  // returns it: one byte under `Charset::Ascii`, and one decoded code point
+  // under `Charset::Utf8`.
+  //
+  // A byte that doesn't start a valid sequence yields the replacement
+  // character and is consumed on its own, so decoding resynchronizes at the
+  // next byte rather than discarding the rest of the text.
+  auto TakeCodePoint(llvm::StringRef& text) const -> char32_t;
+
+  // Returns the columns `code_point` occupies once drawn, which is what drawing
+  // it advances by.
+  //
+  // Under `Charset::Ascii` every code point is one column. Under
+  // `Charset::Utf8` a combining mark is zero, since it renders into the column
+  // before it, and anything with no printable rendering is one, since it is
+  // drawn as a replacement character.
+  //
+  // A combining mark is the only thing zero is ever the answer for, which is
+  // what lets `Buffer` read a zero as one: a code point to fold into the cell
+  // before it rather than give a cell of its own. A code point that takes no
+  // column without combining with anything, such as U+200C ZERO WIDTH
+  // NON-JOINER, has no printable rendering here and takes the column its
+  // replacement character does. Terminals disagree about those -- some give
+  // them a column and some don't -- so drawing one as itself would leave the
+  // columns counted here and the columns painted disagreeing from there on.
+  auto CodePointWidth(char32_t code_point) const -> int;
+
+  // Returns the code point to render for `code_point`, which is a replacement
+  // character where it has no dependable rendering of its own.
+  //
+  // Under `Charset::Ascii` that is anything outside printable ASCII, because a
+  // terminal decoding some single-byte encoding will draw such a byte as
+  // something and there is no way to know what. Under `Charset::Utf8` it is
+  // anything with no printable rendering at all, which includes the surrogates
+  // and so covers everything UTF-8 has no encoding for as well.
+  auto RenderedCodePoint(char32_t code_point) const -> char32_t;
+
+  // Returns the columns `text` occupies once drawn.
+  //
+  // `text` must hold no character that drawing gives a width other than its
+  // code points', so no tab, newline, or carriage return. Those are positional
+  // -- what a tab advances by depends on where the text began -- which makes
+  // them questions about a drawing rather than about the text, and `Buffer`
+  // answers those.
+  auto Width(llvm::StringRef text) const -> int;
+
+  // Removes and returns the longest prefix of `text` that occupies at most
+  // `columns` columns.
+  //
+  // A code point that would straddle the end stops the walk before it, so a cut
+  // never lands inside one and the prefix is never wider than asked for -- it
+  // can be one column narrower, where a double-width character sits on the
+  // boundary.
+  auto TakeColumns(llvm::StringRef& text, int columns) const -> llvm::StringRef;
+
+ private:
+  Charset charset_;
+};
+
+}  // namespace Carbon::Terminal
+
+#endif  // CARBON_COMMON_TERMINAL_METRICS_H_

common/terminal/metrics_test.cpp

@@ -0,0 +1,161 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/metrics.h"
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+
+#include <cstdint>
+
+#include "llvm/ADT/StringRef.h"
+
+namespace Carbon::Terminal {
+namespace {
+
+// "e" followed by U+0301 COMBINING ACUTE ACCENT, which is one column because
+// the mark renders into the column the "e" is in.
+static constexpr llvm::StringLiteral AcuteE = "é";
+
+TEST(MetricsTest, Width) {
+  Metrics utf8(Charset::Utf8);
+  EXPECT_EQ(utf8.Width(""), 0);
+  EXPECT_EQ(utf8.Width("hello"), 5);
+  EXPECT_EQ(utf8.Width("中中"), 4);
+  EXPECT_EQ(utf8.Width("a中b"), 4);
+  EXPECT_EQ(utf8.Width(AcuteE), 1);
+
+  // Every byte is a column when the terminal isn't decoding UTF-8.
+  Metrics ascii(Charset::Ascii);
+  EXPECT_EQ(ascii.Width("hello"), 5);
+  EXPECT_EQ(ascii.Width("中中"), 6);
+  EXPECT_EQ(ascii.Width(AcuteE), 3);
+}
+
+TEST(MetricsTest, CodePointWidth) {
+  Metrics utf8(Charset::Utf8);
+  EXPECT_EQ(utf8.CodePointWidth(U'a'), 1);
+  EXPECT_EQ(utf8.CodePointWidth(U'中'), 2);
+  // A combining mark renders into the column before it.
+  EXPECT_EQ(utf8.CodePointWidth(U'́'), 0);
+  // Something with no rendering is drawn as a replacement, which is a column.
+  EXPECT_EQ(utf8.CodePointWidth(U''), 1);
+
+  Metrics ascii(Charset::Ascii);
+  EXPECT_EQ(ascii.CodePointWidth(U'a'), 1);
+  EXPECT_EQ(ascii.CodePointWidth(U'中'), 1);
+  EXPECT_EQ(ascii.CodePointWidth(U'́'), 1);
+}
+
+TEST(MetricsTest, OnlyCombiningMarksAreZeroColumns) {
+  // Drawing reads a width of zero as "renders into the cell before this one",
+  // so a code point that takes no column without combining with anything has to
+  // measure as something else. Terminals disagree about these -- Terminal.app
+  // gives U+200C a column and VS Code's terminal gives it none -- so each is
+  // drawn as a replacement character, which takes exactly one.
+  Metrics utf8(Charset::Utf8);
+  for (char32_t code_point : {U'\u200b', U'\u200c', U'\u200d', U'\ufeff'}) {
+    EXPECT_EQ(utf8.CodePointWidth(code_point), 1)
+        << static_cast<uint32_t>(code_point);
+    EXPECT_EQ(utf8.RenderedCodePoint(code_point), U'�')
+        << static_cast<uint32_t>(code_point);
+  }
+}
+
+TEST(MetricsTest, RenderedCodePoint) {
+  Metrics utf8(Charset::Utf8);
+  EXPECT_EQ(utf8.RenderedCodePoint(U'a'), U'a');
+  EXPECT_EQ(utf8.RenderedCodePoint(U'中'), U'中');
+  EXPECT_EQ(utf8.RenderedCodePoint(U''), U'�');
+
+  // Code points that UTF-8 has no encoding for have no rendering either.
+  EXPECT_EQ(utf8.RenderedCodePoint(static_cast<char32_t>(0xd800)), U'�');
+  EXPECT_EQ(utf8.RenderedCodePoint(static_cast<char32_t>(0x110000)), U'�');
+
+  // An ASCII terminal is only given what it draws as itself, because there is
+  // no telling what it would draw for anything else.
+  Metrics ascii(Charset::Ascii);
+  EXPECT_EQ(ascii.RenderedCodePoint(U'a'), U'a');
+  EXPECT_EQ(ascii.RenderedCodePoint(U'中'), U'?');
+  EXPECT_EQ(ascii.RenderedCodePoint(U''), U'?');
+}
+
+TEST(MetricsTest, TakeColumns) {
+  Metrics utf8(Charset::Utf8);
+  llvm::StringRef text = "abcde";
+  EXPECT_EQ(utf8.TakeColumns(text, 3), "abc");
+  EXPECT_EQ(text, "de");
+
+  // Taking more than there is takes all of it.
+  EXPECT_EQ(utf8.TakeColumns(text, 10), "de");
+  EXPECT_EQ(text, "");
+
+  // Taking nothing takes nothing, and a negative width is no different.
+  text = "abcde";
+  EXPECT_EQ(utf8.TakeColumns(text, 0), "");
+  EXPECT_EQ(utf8.TakeColumns(text, -1), "");
+  EXPECT_EQ(text, "abcde");
+}
+
+TEST(MetricsTest, TakeColumnsKeepsWideCharactersWhole) {
+  Metrics utf8(Charset::Utf8);
+  // A character that would straddle the end stops the walk before it, so the
+  // prefix comes back a column short rather than half a character wide.
+  llvm::StringRef text = "中中中";
+  llvm::StringRef prefix = utf8.TakeColumns(text, 3);
+  EXPECT_EQ(prefix, "中");
+  EXPECT_EQ(utf8.Width(prefix), 2);
+  EXPECT_EQ(text, "中中");
+
+  // A request landing on a character boundary takes the whole prefix.
+  text = "中中中";
+  EXPECT_EQ(utf8.TakeColumns(text, 4), "中中");
+  EXPECT_EQ(text, "中");
+}
+
+TEST(MetricsTest, TakeColumnsUnderAscii) {
+  // Every byte is a column, so a multi-byte character is cut like any other
+  // run of bytes.
+  Metrics ascii(Charset::Ascii);
+  llvm::StringRef text = "中";
+  EXPECT_EQ(ascii.TakeColumns(text, 2).size(), 2U);
+  EXPECT_EQ(text.size(), 1U);
+}
+
+TEST(MetricsTest, TakeCodePointResynchronizesOnInvalidUtf8) {
+  Metrics utf8(Charset::Utf8);
+  // A byte that starts no valid sequence is consumed on its own, so the text
+  // after it is still decoded rather than being discarded.
+  llvm::StringRef text =
+      "\xff"
+      "a";
+  EXPECT_EQ(utf8.TakeCodePoint(text), U'�');
+  EXPECT_EQ(utf8.TakeCodePoint(text), U'a');
+  EXPECT_TRUE(text.empty());
+}
+
+TEST(MetricsTest, EncodeUtf8) {
+  Utf8Storage storage;
+  EXPECT_EQ(EncodeUtf8(U'a', storage), "a");
+  EXPECT_EQ(EncodeUtf8(U'é', storage), "é");
+  EXPECT_EQ(EncodeUtf8(U'中', storage), "中");
+  EXPECT_EQ(EncodeUtf8(U'\U0001f525', storage), "\U0001f525");
+
+  // A code point with no encoding of its own becomes the replacement.
+  EXPECT_EQ(EncodeUtf8(static_cast<char32_t>(0xd800), storage), "�");
+  EXPECT_EQ(EncodeUtf8(static_cast<char32_t>(0x110000), storage), "�");
+}
+
+TEST(MetricsDeathTest, WidthRejectsPositionalCharacters) {
+  // A tab's width is a fact about a drawing rather than about the text, so
+  // answering for one here would be answering a question this can't see the
+  // inputs to.
+  Metrics metrics(Charset::Utf8);
+  EXPECT_DEATH((void)metrics.Width("a\tb"), "Width is only for text whose");
+  EXPECT_DEATH((void)metrics.Width("a\nb"), "Width is only for text whose");
+  EXPECT_DEATH((void)metrics.Width("a\rb"), "Width is only for text whose");
+}
+
+}  // namespace
+}  // namespace Carbon::Terminal

common/terminal/output_buffer_ref.h

@@ -0,0 +1,163 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_TERMINAL_OUTPUT_BUFFER_REF_H_
+#define CARBON_COMMON_TERMINAL_OUTPUT_BUFFER_REF_H_
+
+#include <array>
+#include <concepts>
+#include <cstdint>
+#include <cstring>
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+
+namespace Carbon::Terminal {
+
+// A reference to the buffer a terminal rendering is assembled into.
+//
+// This owns nothing. It refers to a buffer the caller holds, which must outlive
+// it, and converts implicitly from one so that rendering goes into storage the
+// caller already has.
+//
+// Rendering assembles bytes here rather than streaming them: a stream call per
+// literal and per number costs measurably more than handing over a finished
+// sequence, and code that wants a stream prints the buffer once it is complete.
+//
+// Appending is shaped around what terminal output is made of, which is a great
+// many short escape sequences, each a handful of literal bytes around a number
+// that never exceeds 255. Taking a whole sequence at a time grows the buffer
+// once per sequence rather than once per byte, and that difference is much of
+// what rendering costs.
+class OutputBufferRef {
+ public:
+  // Implicit, so that call sites pass the buffer they already hold rather than
+  // naming this type.
+  //
+  // NOLINTNEXTLINE(google-explicit-constructor)
+  OutputBufferRef(llvm::SmallVectorImpl<char>& bytes) : bytes_(&bytes) {}
+
+  // Appends `pieces`, each of which is either text, appended as it is, or a
+  // `uint8_t`, appended in decimal.
+  //
+  // No other type is accepted, so the two can never be taken for each other,
+  // and nothing needs one: the literal bytes of an escape sequence are always
+  // text, and every number one carries is a channel value, a palette index, or
+  // an SGR code, none of which exceed 255.
+  //
+  // No piece may point into the buffer, which appending can reallocate.
+  template <typename... PieceT>
+  auto Append(const PieceT&... pieces) -> void {
+    if constexpr (sizeof...(pieces) == 1) {
+      // A lone piece has nothing to assemble, and the buffer's own append is
+      // already the single growth and single copy this is after.
+      (AppendPiece(pieces), ...);
+    } else {
+      // Growing to the bound before writing keeps how far the buffer grows
+      // independent of the piece values, so computing one can't hold that up.
+      // Only the trim afterwards depends on how many digits a number took.
+      size_t begin = bytes_->size();
+      bytes_->resize_for_overwrite(begin + (AppendedSize(pieces) + ... + 0));
+      char* data = bytes_->data();
+      char* cursor = data + begin;
+      ((cursor = WritePiece(cursor, pieces)), ...);
+      bytes_->truncate(cursor - data);
+    }
+  }
+
+ private:
+  // The room a number needs: its three digits, plus one more because it is
+  // written as a single four-byte store whose last byte is discarded.
+  static constexpr size_t NumberBytes = 4;
+
+  // The decimal text of a number, and how many digits it took. The digits are
+  // at the front and the length in the byte after them, so a whole entry is one
+  // store and the length says how far of it to keep.
+  struct NumberText {
+    std::array<char, NumberBytes - 1> digits;
+    uint8_t length;
+  };
+  static_assert(sizeof(NumberText) == NumberBytes,
+                "A number is written by storing a whole entry at once.");
+
+  // The text of every value a number piece can hold. A kilobyte of table, in
+  // exchange for a lookup where computing the digits would branch on the value
+  // three times.
+  static constexpr std::array<NumberText, 256> NumberTexts = [] {
+    std::array<NumberText, 256> texts = {};
+    for (int value = 0; value < 256; ++value) {
+      NumberText& text = texts[value];
+      text.length = 1 + (value >= 10) + (value >= 100);
+      int rest = value;
+      for (int digit = text.length; digit > 0; --digit) {
+        text.digits[digit - 1] = static_cast<char>('0' + rest % 10);
+        rest /= 10;
+      }
+    }
+    return texts;
+  }();
+
+  // Returns the most bytes a piece can append. A number contributes the bound
+  // above rather than the digits it will take, so the bound for a sequence
+  // doesn't depend on any of the values in it.
+  template <size_t N>
+  static constexpr auto AppendedSize(const char (& /*piece*/)[N]) -> size_t {
+    return N - 1;
+  }
+  static constexpr auto AppendedSize(llvm::StringRef piece) -> size_t {
+    return piece.size();
+  }
+  template <std::same_as<uint8_t> T>
+  static constexpr auto AppendedSize(T /*piece*/) -> size_t {
+    return NumberBytes;
+  }
+
+  // Writes a piece at `out` and returns the position past it. There must be
+  // `AppendedSize(piece)` bytes of room, as nothing here checks.
+  template <size_t N>
+  static auto WritePiece(char* out, const char (&piece)[N]) -> char* {
+    std::memcpy(out, piece, N - 1);
+    return out + N - 1;
+  }
+  static auto WritePiece(char* out, llvm::StringRef piece) -> char* {
+    // An empty `StringRef` may hold a null pointer, which `memcpy` doesn't
+    // accept even for an empty copy.
+    if (!piece.empty()) {
+      std::memcpy(out, piece.data(), piece.size());
+    }
+    return out + piece.size();
+  }
+  template <std::same_as<uint8_t> T>
+  static auto WritePiece(char* out, T piece) -> char* {
+    // One load and one store, with no branch on the value. Escape sequences
+    // carry color channels and palette indices, which are spread across the
+    // whole range, so a branch per digit is one the processor can't predict,
+    // and there are four numbers in a truecolor escape. The store always covers
+    // four bytes, which is why a number reserves that many, and the cursor
+    // advances only over the digits that count.
+    const NumberText& text = NumberTexts[piece];
+    std::memcpy(out, &text, sizeof(text));
+    return out + text.length;
+  }
+
+  // Appends a piece on its own, growing the buffer to fit it.
+  template <size_t N>
+  auto AppendPiece(const char (&piece)[N]) -> void {
+    bytes_->append(piece, piece + N - 1);
+  }
+  auto AppendPiece(llvm::StringRef piece) -> void {
+    bytes_->append(piece.begin(), piece.end());
+  }
+  template <std::same_as<uint8_t> T>
+  auto AppendPiece(T piece) -> void {
+    std::array<char, NumberBytes> digits;
+    bytes_->append(digits.data(), WritePiece(digits.data(), piece));
+  }
+
+  llvm::SmallVectorImpl<char>* bytes_;
+};
+
+}  // namespace Carbon::Terminal
+
+#endif  // CARBON_COMMON_TERMINAL_OUTPUT_BUFFER_REF_H_

common/terminal/output_buffer_ref_test.cpp

@@ -0,0 +1,99 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/terminal/output_buffer_ref.h"
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+
+#include "llvm/ADT/SmallString.h"
+
+namespace Carbon::Terminal {
+namespace {
+
+using ::testing::Eq;
+
+// A single piece and several pieces are appended by different code, so both
+// appear throughout these tests rather than in one case of their own.
+
+TEST(OutputBufferRefTest, Text) {
+  llvm::SmallString<16> bytes;
+  OutputBufferRef out = bytes;
+  out.Append("one");
+  out.Append(llvm::StringRef(" two"));
+  out.Append(std::string(" three"));
+  out.Append(" four", llvm::StringRef(" five"));
+  EXPECT_THAT(bytes, Eq("one two three four five"));
+}
+
+TEST(OutputBufferRefTest, EmptyPieces) {
+  llvm::SmallString<16> bytes;
+  OutputBufferRef out = bytes;
+  out.Append();
+  out.Append("");
+  out.Append("", llvm::StringRef(), "kept", llvm::StringRef(""));
+  EXPECT_THAT(bytes, Eq("kept"));
+}
+
+TEST(OutputBufferRefTest, NumbersUseEveryDigitCount) {
+  llvm::SmallString<16> bytes;
+  OutputBufferRef out = bytes;
+  for (uint8_t value : {0, 9, 10, 99, 100, 255}) {
+    out.Append(value);
+    out.Append(" ", value, " ");
+  }
+  EXPECT_THAT(bytes, Eq("0 0 9 9 10 10 99 99 100 100 255 255 "));
+}
+
+// A number always writes fewer bytes than it reserves, so pieces after one in
+// the same call are what catch a misplaced write.
+TEST(OutputBufferRefTest, NumbersFollowedByMorePieces) {
+  llvm::SmallString<32> bytes;
+  OutputBufferRef out = bytes;
+  out.Append("\x1b[", static_cast<uint8_t>(38), ";2;", static_cast<uint8_t>(1),
+             ";", static_cast<uint8_t>(22), ";", static_cast<uint8_t>(255),
+             "m");
+  EXPECT_THAT(bytes, Eq("\x1b[38;2;1;22;255m"));
+}
+
+TEST(OutputBufferRefTest, AppendsAfterExistingContents) {
+  llvm::SmallString<16> bytes = llvm::StringRef("before:");
+  OutputBufferRef out = bytes;
+  out.Append(static_cast<uint8_t>(7));
+  EXPECT_THAT(bytes, Eq("before:7"));
+}
+
+// Appending has to work the same however the buffer is laid out, and a number
+// leaves the buffer grown further than it wrote, so reallocation is where a
+// size mistake would show up.
+TEST(OutputBufferRefTest, AppendsPastInlineCapacity) {
+  llvm::SmallString<8> bytes;
+  OutputBufferRef out = bytes;
+  std::string expected;
+  for (int i = 0; i < 100; ++i) {
+    out.Append("x", static_cast<uint8_t>(i));
+    expected += "x" + std::to_string(i);
+  }
+  EXPECT_THAT(bytes, Eq(expected));
+  EXPECT_THAT(bytes.size(), Eq(expected.size()));
+}
+
+// References to one buffer all append to it, and none of them own it, so the
+// buffer keeps everything written through any of them.
+TEST(OutputBufferRefTest, ReferencesShareTheirBuffer) {
+  llvm::SmallString<16> bytes;
+  OutputBufferRef first = bytes;
+  first.Append("a");
+  {
+    OutputBufferRef second = bytes;
+    second.Append("b");
+  }
+  OutputBufferRef copy = first;
+  copy.Append("c");
+  first.Append("d");
+  EXPECT_THAT(bytes, Eq("abcd"));
+}
+
+}  // namespace
+}  // namespace Carbon::Terminal