If multiple files lower the same variable, use the same GlobalVariable. (#8101)

We can lower the same variable from multiple `SemIR::File`s if we reach
it through a specific for a generic in a different file. When this
happens, reuse the same global variable in lowering. This matches how we
handle the same problem for functions.

Assisted-by: Claude via Antigravity

.agents/skills/diagnostics/SKILL.md

@@ -2,7 +2,8 @@
 name: Diagnostics
 description:
     Instructions for declaring, formatting, emitting, testing, and styling
-    diagnostic messages (errors, warnings, notes) in the Carbon toolchain.
+    diagnostic messages (errors, warnings, and the labels that explain them) in
+    the Carbon toolchain.
 ---
 
 # Diagnostics in the Carbon Toolchain
@@ -15,9 +16,9 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 The Carbon compiler features a highly-engineered, context-aware diagnostics
 framework designed to deliver precise, readable, and highly targetable
-diagnostic output (errors, warnings, notes). This document establishes strict
-rules for declaring, formatting, emitting, testing, and styling compiler
-diagnostics.
+diagnostic output (errors, warnings, and the labels that explain them). This
+document establishes strict rules for declaring, formatting, emitting, testing,
+and styling compiler diagnostics.
 
 ---
 
@@ -39,8 +40,8 @@ Diagnostics are handled via three decoupled core components:
     [kind.def](../../../toolchain/diagnostics/kind.def).
 2.  **Emitters**: Specialized formatting pipelines (parameterized on custom
     phase location types `LocT` like `Token` or `LocId`) that convert raw tokens
-    to standardized physical source locations (file, line, column, and text
-    snippet).
+    to standardized physical source locations (file, line, column, and the text
+    of the line).
 3.  **Consumers**: Pipelines that process, track, filter, and sort diagnostics.
     The default `SortingConsumer` buffers and stable-sorts diagnostics based on
     their `last_byte_offset` matching compiler traversal order to ensure perfect
@@ -86,7 +87,7 @@ declaration (`CARBON_DIAGNOSTIC` or `CARBON_DIAGNOSTIC_ON_SCOPE`).
 -   **File Scope**: If the diagnostic is shared among multiple functions inside
     the _same_ file, declare it at **file scope** inside the anonymous namespace
     of the `.cpp` file.
--   **Global Scope**: If a diagnostic (such as a shared helper note) is reused
+-   **Global Scope**: If a diagnostic (such as a shared helper label) is reused
     _across different physical files_, define it in a shared header (e.g.
     context/check helpers) and mark it `extern` where applicable, ensuring the
     macro is only invoked once.
@@ -155,40 +156,66 @@ Custom structures can define how they serialize inside diagnostics using the
 
 ### Fluent Builder Pattern
 
-For compound diagnostics requiring multiple sub-notes, carets, or custom code
-overrides, use `Build` to chain actions fluently:
+A diagnostic is one message plus the labels that mark the source explaining it.
+Use `Build` to state the message and `Attach` to mark each place the reader has
+to look, chained fluently:
 
 ```cpp
+CARBON_DIAGNOSTIC_LABEL(ModifierPrevious, Info, "`{0}` previously appeared here",
+                        Lex::TokenKind);
+
 context.emitter()
     .Build(second_node, ModifierRepeated, context.token_kind(second_node))
-    .Note(first_node, ModifierPrevious, context.token_kind(first_node))
-    .OverrideSnippet("custom snippet...")
+    .Attach(first_node, ModifierPrevious, context.token_kind(first_node))
     .Emit();
 ```
 
+Labels are declared with `CARBON_DIAGNOSTIC_LABEL` where they are attached, and
+are not registered in `kind.def`; `check_diagnostics.py` checks that each one is
+attached and covered. A label is either `Primary` -- the range the message
+itself is about -- or `Info`, for somewhere else the reader must look; those two
+are the only categories, because anything not read against the code it names is
+not a label. `Attach(loc)` with no label marks a range wordlessly, which is what
+to use when a label would only restate the message.
+
+Every diagnostic should attach at least one label; a message with nothing
+attached has nothing to show. See
+[diagnostics.md](../../../toolchain/docs/diagnostics.md) for which range to pick
+and when a label earns words.
+
 > [!SAFETY] Emitter builders are marked `[[nodiscard]]`. To prevent a developer
 > from creating a builder but failing to terminal-chain `.Emit()`, the builder
 > uses an rvalue overload `Emit() &&` that triggers a compile-time
 > `static_assert(false)`. You must save the builder to an lvalue or execute the
-> chain exactly as `emitter.Build(...).Note(...).Emit()`.
+> chain exactly as `emitter.Build(...).Attach(...).Emit()`.
 
 ### RAII Context & Annotation Scopes
 
-Manage large checking structures requiring blanket note context using RAII block
+Manage large checking structures requiring blanket context using RAII block
 scopes:
 
--   `ContextScope`: Automatically converts any diagnostics emitted within its
-    scope into sub-notes under a high-level operation descriptor:
+-   `ContextScope`: Attaches a context to every diagnostic emitted within its
+    scope, describing the higher-level operation that failed because of it. A
+    context is declared with `CARBON_DIAGNOSTIC_CONTEXT`, or
+    `CARBON_DIAGNOSTIC_SOFT_CONTEXT` for one dropped when the diagnostic
+    already has a context:
 
     ```cpp
-    ContextScope context_scope(&context.emitter(), [&](ContextBuilder& builder) {
-      builder.Context(eval_loc, InCallToEvalFn);
-    });
-    // any checker error emitted here will automatically append the 'InCallToEvalFn' note
+    Diagnostics::ContextScope diagnostic_context(
+        &context.emitter(), [&](auto& builder) {
+          CARBON_DIAGNOSTIC_CONTEXT(
+              QualifiedDeclInIncompleteClassScope,
+              "cannot declare a member of incomplete class {0}", SemIR::TypeId);
+          builder.Attach(loc_id, QualifiedDeclInIncompleteClassScope,
+                         context.classes().Get(class_id).self_type_id);
+        });
     ```
 
--   `AnnotationScope`: RAII block scope that automatically attaches blanket note
-    annotations to all scoped diagnostics.
+    A context is not a label: its text stands alone as a sentence, which is why
+    it leads the diagnostic rather than following it.
+
+-   `AnnotationScope`: RAII block scope that attaches labels to every diagnostic
+    emitted within it, using the same `Builder` as a direct emission.
 
 ---
 
@@ -257,5 +284,15 @@ Carbon strictly enforces testing coverage at build-time.
     ```
 
 3.  **Build Enforcement**: Failing to provide a diagnostic test check matcher
-    triggers a build compilation error on the target test
+    triggers a failure on the target test
     `//toolchain/diagnostics:coverage_test`.
+4.  **Label Enforcement**: A label is declared where it is attached rather than
+    in a central registry, so `coverage_test` has no list of labels to compare a
+    matched name against and leaves any name that isn't a kind alone.
+    [check_diagnostics.py](../../../toolchain/diagnostics/check_diagnostics.py)
+    is what checks them: it reads the declarations and the testdata directly, so
+    it catches a label with no test, a name matched on that is neither a kind
+    nor a label, and a kind registered in `kind.def` that nothing declares. It
+    runs as a pre-commit hook, which CI runs too, rather than as a bazel test --
+    reading every source file is what it needs and what a test in the sandbox
+    cannot do.

.bazelrc

@@ -125,6 +125,11 @@ common:linux --define=pfm=1
 # Disables `actions.declare_symlink`. Done for cross-environment support.
 common --allow_unresolved_symlinks=false
 
+# Starting with rules_python 2.0, Python binaries use venvs created with
+# `actions.declare_symlink`, which we disable above. Have it create them at
+# runtime instead.
+common --@rules_python//python/config_settings:venvs_use_declare_symlink=no
+
 # Removes the leading `/proc/self/cwd/` from file paths in the debug info. Some
 # tools like VS Code don't understand `/proc/self/cwd` in places like terminal
 # stack dumps, but do understand paths relative to the workspace root.

.clang-format

@@ -7,6 +7,7 @@ BasedOnStyle: Google
 AllowShortBlocksOnASingleLine: 'false'
 AllowShortIfStatementsOnASingleLine: Never
 AllowShortLoopsOnASingleLine: 'false'
+AttributeMacros: ['CARBON_PRESERVE_NONE']
 DerivePointerAlignment: 'false'
 ExperimentalAutoDetectBinPacking: 'false'
 FixNamespaceComments: 'true'

.github/actions/build-setup-common/action.yml

@@ -13,7 +13,7 @@ runs:
   steps:
     # Setup Python and related tools with uv.
     - name: Set up uv and Python
-      uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+      uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
       with:
         enable-cache: true
         version: '0.11.15'
@@ -135,8 +135,9 @@ runs:
         # more reliably succeed without retrying "permanent" errors.
         #
         # Disable lockfile updates, because some actions want to see
-        # differences.
-        ./scripts/run_bazel.py --attempts=5 --retry-all-errors \
-          mod --lockfile_mode=off deps
+        # differences. This avoids `bazel mod deps`, which evaluates every
+        # module extension; without the lockfile, that includes expensive ones
+        # that builds don't use, like the Rust crates newer protobuf versions
+        # depend on.
         ./scripts/run_bazel.py --attempts=5 --retry-all-errors \
           cquery --lockfile_mode=off //... | wc -l

.github/actions/build-setup-macos/action.yml

@@ -28,7 +28,7 @@ runs:
     # minimum supported LLVM version.
     - name: Cache Homebrew
       id: cache-homebrew-macos
-      uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
+      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
       with:
         # Cover all the critical parts of Homebrew here. Homebrew on Arm macOS
         # uses its own prefix making this easy to cover, but we need a few

.github/actions/build-setup-ubuntu/action.yml

@@ -13,7 +13,7 @@ runs:
     # everything provides a little flexibility to get space while trying
     # to shrink the build.
     - name: Free up disk space
-      uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
+      uses: jlumbroso/free-disk-space@ceedf095f4ec1a097402bc6bd80831f2e1a6fde6 # v2.0.0
       with:
         android: true
         dotnet: true
@@ -22,6 +22,18 @@ runs:
         # to save time.
         large-packages: false
 
+    # The image's Android environment variables still point into the SDK and
+    # NDK deleted above, which confuses tools that look for them, such as
+    # rules_android. `$GITHUB_ENV` can't unset variables, so set them to empty,
+    # which rules_android treats as unset.
+    - name: Clear Android environment variables
+      shell: bash
+      run: |
+        for var in ANDROID_HOME ANDROID_SDK_ROOT ANDROID_NDK ANDROID_NDK_HOME \
+            ANDROID_NDK_LATEST_HOME ANDROID_NDK_ROOT; do
+          echo "$var=" >> "$GITHUB_ENV"
+        done
+
     # Select the LLVM release - by the cache key and the download.
     - name: Select LLVM release
       shell: bash
@@ -37,7 +49,7 @@ runs:
     # reliability.
     - name: Cache LLVM and Clang installation
       id: cache-llvm-ubuntu
-      uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
+      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
       with:
         path: ~/llvm
         key: LLVM-${{ env.LLVM_RELEASE }}-Cache-ubuntu-${{ runner.arch }}

.github/actions/test-setup/action.yml

@@ -29,7 +29,7 @@ runs:
     # action run for the merge queue. We filter steps based on the paths here,
     # and condition steps on the output.
     - id: filter
-      uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+      uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
       with:
         filters: |
           has_code:

.github/dependabot.yml

@@ -0,0 +1,61 @@
+# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+# Exceptions. See /LICENSE for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+# Dependabot version updates for our GitHub Actions, pre-commit hooks, and Bazel
+# modules. See .github/workflows/README.md for how these PRs are handled.
+#
+# Updates keep our existing pinning: actions stay pinned to full commit SHAs
+# with a `# vX.Y.Z` comment, and pre-commit hooks stay frozen to commit SHAs
+# with a `# frozen: vX.Y.Z` comment.
+#
+# Security updates for other ecosystems, such as npm and pip, are enabled in the
+# repository settings and don't need an entry here.
+
+version: 2
+
+updates:
+  - package-ecosystem: github-actions
+    # `/` only covers `.github/workflows`; list composite actions separately.
+    directories: ['/', '/.github/actions/*']
+    schedule:
+      interval: weekly
+    # Give new releases a week to be vetted before we adopt them.
+    cooldown:
+      default-days: 7
+    # Use our existing label instead of Dependabot's per-ecosystem labels.
+    labels: ['dependencies']
+    groups:
+      # Batch minor and patch updates, but put major updates in a separate PR so
+      # they get a closer look.
+      github-actions:
+        patterns: ['*']
+        update-types: ['minor', 'patch']
+      github-actions-major:
+        patterns: ['*']
+        update-types: ['major']
+
+  - package-ecosystem: pre-commit
+    directory: '/'
+    schedule:
+      interval: monthly
+    cooldown:
+      default-days: 7
+    labels: ['dependencies']
+    groups:
+      pre-commit:
+        patterns: ['*']
+
+  # Dependabot regenerates `MODULE.bazel.lock` on a single platform, so the test
+  # workflow may still ask for platform-specific lockfile updates. LLVM is
+  # pinned with `git_override`, which Dependabot doesn't update.
+  - package-ecosystem: bazel
+    directory: '/'
+    schedule:
+      interval: monthly
+    cooldown:
+      default-days: 7
+    labels: ['dependencies']
+    groups:
+      bazel:
+        patterns: ['*']

.github/workflows/auto_label_prs.yaml

@@ -4,6 +4,7 @@
 
 name: 'Auto label PRs'
 on:
+  # zizmor: ignore[dangerous-triggers] - {necessary to mutate the PR}
   pull_request_target:
     types: [opened, ready_for_review]
 
@@ -19,7 +20,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           disable-sudo: true
           egress-policy: block
@@ -28,7 +29,7 @@ jobs:
             api.github.com:443
 
       - id: filter
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
         with:
           filters: |
             documentation:
@@ -113,7 +114,7 @@ jobs:
       # and it applies orthogonally to the others.
       - id: bot
         if:
-          contains(fromJSON('["CarbonInfraBot", "dependabot"]'),
+          contains(fromJSON('["CarbonInfraBot", "dependabot[bot]"]'),
           github.event.pull_request.user.login)
         run: |
           gh pr edit "${PR}" --add-label "bot" --repo carbon-language/carbon-lang

.github/workflows/check_dependent_pr.yaml

@@ -21,7 +21,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           disable-sudo: true
           egress-policy: block
@@ -33,12 +33,12 @@ jobs:
       # Note: pull_request_target checks out the base branch by default.
       # This is safe as it avoids running untrusted code from the PR branch.
       - name: Checkout code
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
       - name: Set up uv
-        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+        uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           version: '0.11.15'

.github/workflows/clangd_tidy.yaml

@@ -26,7 +26,7 @@ jobs:
 
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: block
           # When adding endpoints, see README.md.
@@ -41,29 +41,34 @@ jobs:
             download.gnome.org:443
             downloads.sourceforge.net:443
             files.pythonhosted.org:443
+            formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
+            pkg-containers.githubusercontent.com:443
             pypi.org:443
             raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
             releases.astral.sh:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
-      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
       - id: filter
-        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
         with:
           predicate-quantifier: 'every'
           filters: |

.github/workflows/dependabot_pr_description.yaml

@@ -0,0 +1,87 @@
+# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+# Exceptions. See /LICENSE for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+# Replaces Dependabot's PR description with a short markdown list of the updated
+# dependencies. We squash-merge using the PR description as the commit message,
+# and Dependabot's descriptions are mostly HTML: release notes, commit lists,
+# and command help. The original description remains in the PR's edit history.
+name: 'Dependabot PR description'
+
+on:
+  pull_request:
+    types: [opened, reopened, synchronize, edited]
+
+# Dependabot can push and edit the description in quick succession.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
+  cancel-in-progress: true
+
+permissions: {}
+
+jobs:
+  rewrite:
+    if: github.event.pull_request.user.login == 'dependabot[bot]'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read # For dependabot/fetch-metadata to read the PR's commit.
+      pull-requests: write # For `gh pr edit`.
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
+        with:
+          disable-sudo: true
+          egress-policy: block
+          # prettier-ignore
+          allowed-endpoints: >
+            api.github.com:443
+
+      - id: metadata
+        uses: dependabot/fetch-metadata@25dd0e34f4fe68f24cc83900b1fe3fe149efef98 # v3.1.0
+
+      - name: Replace the description
+        env:
+          DEPENDENCIES: ${{ steps.metadata.outputs.updated-dependencies-json }}
+          GH_TOKEN: ${{ github.token }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          PR_URL: ${{ github.event.pull_request.html_url }}
+        run: |
+          # Only replace a description that Dependabot wrote: either nobody has
+          # edited it, or Dependabot last did, such as when rebasing. This
+          # leaves human edits and our own earlier rewrites alone. Don't look
+          # for markers in the text, because Dependabot truncates descriptions
+          # that are too long.
+          # shellcheck disable=SC2016 # The `$`s are GraphQL variables.
+          editor="$(gh api graphql \
+            -f owner="${GITHUB_REPOSITORY%/*}" \
+            -f name="${GITHUB_REPOSITORY#*/}" \
+            -F number="$PR_NUMBER" \
+            -f query='
+              query($owner: String!, $name: String!, $number: Int!) {
+                repository(owner: $owner, name: $name) {
+                  pullRequest(number: $number) { editor { login } }
+                }
+              }' \
+            --jq '.data.repository.pullRequest.editor.login // ""')"
+          if [[ -n "$editor" && "$editor" != "dependabot" ]]; then
+            echo "The description was last edited by $editor, so leaving it."
+            exit 0
+          fi
+
+          # One short line per dependency. GitHub hard-wraps squash commit
+          # messages at 72 columns, so avoid tables and long links, and drop
+          # the URL prefix from pre-commit hook names.
+          body="$(jq -r '
+            unique_by([.dependencyName, .prevVersion, .newVersion])[]
+            | "- `\(.dependencyName | ltrimstr("https://github.com/"))` "
+              + (.prevVersion | ltrimstr("v")) + " -> "
+              + (.newVersion | ltrimstr("v"))
+              + if .updateType == "version-update:semver-major"
+                then " (major)" else "" end
+          ' <<<"$DEPENDENCIES")"
+          if [[ -z "$body" ]]; then
+            echo "Found no updated dependencies, so leaving it alone."
+            exit 0
+          fi
+
+          gh pr edit "$PR_URL" --repo "$GITHUB_REPOSITORY" --body "$body"

.github/workflows/discord_wiki.yaml

@@ -15,7 +15,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: audit
 

.github/workflows/gh_pages_ci.yaml

@@ -22,17 +22,17 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: audit
 
       - name: Checkout
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
       - name: Set up uv
-        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+        uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           version: '0.11.15'
@@ -40,7 +40,7 @@ jobs:
       - name: Prebuild actions
         run: ./website/prebuild.py
       - name: Setup Ruby
-        uses: ruby/setup-ruby@6ca151fd1bfcfd6fe0c4eb6837eb0584d0134a0c # v1.290.0
+        uses: ruby/setup-ruby@7f797815d8adceb1c412e7abefdd1ab4abc47a5f # v1.335.0
         with:
           # Runs 'bundle install' and caches installed gems automatically.
           bundler-cache: true

.github/workflows/gh_pages_deploy.yaml

@@ -27,17 +27,17 @@ jobs:
       contents: read
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: audit
 
       - name: Checkout
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
       - name: Set up uv
-        uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+        uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3
         with:
           enable-cache: true
           version: '0.11.15'
@@ -45,9 +45,9 @@ jobs:
       - name: Prebuild actions
         run: ./website/prebuild.py
       - name: Setup Pages
-        uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
+        uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
       - name: Setup Ruby
-        uses: ruby/setup-ruby@6ca151fd1bfcfd6fe0c4eb6837eb0584d0134a0c # v1.290.0
+        uses: ruby/setup-ruby@7f797815d8adceb1c412e7abefdd1ab4abc47a5f # v1.335.0
         with:
           # Runs 'bundle install' and caches installed gems automatically.
           bundler-cache: true
@@ -65,7 +65,7 @@ jobs:
       - name: Upload artifact
         # Automatically uploads an artifact from the './_site' directory by
         # default.
-        uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
+        uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
 
   deploy:
     environment:
@@ -79,10 +79,10 @@ jobs:
       id-token: write
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: audit
 
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5
+        uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1

.github/workflows/nightly_release.yaml

@@ -37,7 +37,7 @@ jobs:
     runs-on: ubuntu-22.04
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: block
           # When adding endpoints, see README.md.
@@ -52,24 +52,29 @@ jobs:
             download.gnome.org:443
             downloads.sourceforge.net:443
             files.pythonhosted.org:443
+            formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
+            pkg-containers.githubusercontent.com:443
             pypi.org:443
             raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
       - name: Checkout branch
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 

.github/workflows/prek.yaml

@@ -18,7 +18,7 @@ jobs:
     runs-on: ubuntu-22.04
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: block
           # When adding endpoints, see README.md.
@@ -33,24 +33,29 @@ jobs:
             download.gnome.org:443
             downloads.sourceforge.net:443
             files.pythonhosted.org:443
+            formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
+            pkg-containers.githubusercontent.com:443
             pypi.org:443
             raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
             releases.astral.sh:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
-      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
@@ -60,21 +65,21 @@ jobs:
           matrix_runner: ubuntu-22.04
           remote_cache_upload: '--remote_upload_local_results=false'
 
-      - uses: j178/prek-action@01345c78b7de7d79edf368729212760396ba9345 # v2
+      - uses: j178/prek-action@44924fe9ef4a6f88253c4b6b1bf11c689d3ce53d # v3.1.0
 
       # We want to automatically create github suggestions for prek file
       # changes for a pull request. But `pull_request` actions never have write
-      # permissions to the repository, so we create the suggestions in a separate
-      # privileged `workflow_run` action in prek_suggestions.yaml. Here,
-      # we upload the diffs and event configuration to an artifact for use by
-      # that action.
+      # permissions to the repository, so we create the suggestions (or, for
+      # Dependabot PRs, commit the fixes) in a separate privileged `workflow_run`
+      # action in prek_suggestions.yaml. Here, we upload the diffs and event
+      # configuration to an artifact for use by that action.
       - name: Collect prek output
         if: failure()
         run: |
           mkdir -p prek-output
           git diff > prek-output/diff
           cp $GITHUB_EVENT_PATH prek-output/event
-      - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
+      - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
         if: failure()
         with:
           name: prek output

.github/workflows/prek_suggestions.yaml

@@ -11,9 +11,18 @@ name: 'Add prek suggestions'
 # we use a separate privileged `workflow_run` action here to pick up its results
 # and convert them into suggestion comments.
 #
+# For Dependabot PRs, prek's fixes usually come from an updated hook and are in
+# files outside the PR's diff, where GitHub doesn't allow suggestions. So
+# instead, we commit the fixes directly to Dependabot's branch.
+#
+# A privileged `workflow_run` action is inherently risky, because it could be
+# tricked into running untrusted code from a PR. So neither job here runs
+# anything from the PR, and both treat prek's output as untrusted.
+#
 # This action is only run from the workflow file on the trunk branch. Changes to
 # this file will not take effect until they are merged to trunk.
 on:
+  # zizmor: ignore[dangerous-triggers] - {necessary to act on prek's results}
   workflow_run:
     workflows: [prek]
     types:
@@ -25,14 +34,16 @@ permissions:
 
 jobs:
   pull-request-suggestions:
-    # Only generate suggestions if prek for a PR failed.
+    # Only generate suggestions if prek for a PR failed. Dependabot's PRs are
+    # handled by `dependabot-fixes` below.
     if: |
       github.event.workflow_run.conclusion == 'failure' &&
-      github.event.workflow_run.event == 'pull_request'
+      github.event.workflow_run.event == 'pull_request' &&
+      !startsWith(github.event.workflow_run.head_branch, 'dependabot/')
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           disable-sudo: true
           egress-policy: block
@@ -42,17 +53,18 @@ jobs:
             github.com:443
             objects.githubusercontent.com:443
             raw.githubusercontent.com:443
+            release-assets.githubusercontent.com:443
 
-      - uses: reviewdog/action-setup@3f401fe1d58fe77e10d665ab713057375e39b887 # v1.3.0
+      - uses: reviewdog/action-setup@d8a7baabd7f3e8544ee4dbde3ee41d0011c3a93f # v1.5.0
         with:
-          reviewdog_version: latest
+          reviewdog_version: v0.21.2
 
-      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
       - name: Download prek output
-        uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
+        uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
         with:
           name: prek output
           github-token: ${{ secrets.GITHUB_TOKEN }}
@@ -68,3 +80,101 @@ jobs:
           cat ./diff | \
           GITHUB_EVENT_PATH=./event \
             reviewdog -f=diff -f.diff.strip=1 -reporter=github-pr-review
+
+  dependabot-fixes:
+    # Only handle prek failures from Dependabot's own pushes to its branches in
+    # this repository. This also keeps us from reacting to our own fix commits.
+    if: |
+      github.event.workflow_run.conclusion == 'failure' &&
+      github.event.workflow_run.event == 'pull_request' &&
+      github.event.workflow_run.actor.login == 'dependabot[bot]' &&
+      github.event.workflow_run.head_repository.full_name == github.repository &&
+      startsWith(github.event.workflow_run.head_branch, 'dependabot/')
+    runs-on: ubuntu-latest
+    permissions:
+      actions: read # To download the artifact from the prek run.
+      contents: read # For actions/checkout.
+      pull-requests: read # To check the PR before pushing to it.
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
+        with:
+          disable-sudo: true
+          egress-policy: block
+          # prettier-ignore
+          allowed-endpoints: >
+            api.github.com:443
+            github.com:443
+
+      - name: Download prek output
+        uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
+        with:
+          name: prek output
+          path: ${{ runner.temp }}/prek-output
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          run-id: ${{ github.event.workflow_run.id }}
+
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          ref: ${{ github.event.workflow_run.head_sha }}
+          persist-credentials: false
+
+      # The diff comes from running newly updated hooks, so treat it as
+      # untrusted: never run anything from it, and only accept edits to existing
+      # files outside of `.github`.
+      - name: Commit prek's fixes
+        env:
+          BRANCH: ${{ github.event.workflow_run.head_branch }}
+          DIFF: ${{ runner.temp }}/prek-output/diff
+          GH_TOKEN: ${{ github.token }}
+          HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
+          PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
+          # A CarbonInfraBot token that can push to this repository. A push with
+          # `GITHUB_TOKEN` wouldn't trigger CI for the new commit.
+          PUSH_TOKEN: ${{ secrets.CARBON_INFRA_BOT_DEPENDABOT_FIXES }}
+        run: |
+          # Check the PR itself, rather than trusting the artifact.
+          if [[ -z "$PR_NUMBER" ]]; then
+            echo "No PR is associated with this run, so skipping."
+            exit 0
+          fi
+          pr="$(gh api "repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER" \
+            --jq '[.user.login, .head.ref, .head.sha] | join(" ")')"
+          if [[ "$pr" != "dependabot[bot] $BRANCH $HEAD_SHA" ]]; then
+            echo "PR #$PR_NUMBER isn't Dependabot's or has moved on; skipping."
+            exit 0
+          fi
+
+          if [[ ! -s "$DIFF" ]]; then
+            echo "prek didn't change any files, so there's nothing to commit."
+            exit 0
+          fi
+
+          # Have git describe the diff: `--numstat` gives a tab-separated line
+          # per file with its added and removed line counts ("-" for binary
+          # files), and `--summary` adds a line for anything that isn't an edit,
+          # such as creating, deleting, or renaming a file. Only accept text
+          # edits to files outside of `.github`.
+          changes="$(git apply --numstat --summary "$DIFF")"
+          rejected="$(awk -F'\t' '
+            !($1 ~ /^[0-9]+$/ && $2 ~ /^[0-9]+$/ && $3 !~ /^\.github\//)
+          ' <<<"$changes")"
+          if [[ -n "$rejected" ]]; then
+            echo "::error::Not applying prek's diff because of:"
+            echo "$rejected"
+            exit 1
+          fi
+          if ! git apply "$DIFF"; then
+            echo "::error::prek's diff doesn't apply; run prek on the branch."
+            exit 1
+          fi
+          if [[ -z "$PUSH_TOKEN" ]]; then
+            echo "::error::CARBON_INFRA_BOT_DEPENDABOT_FIXES isn't available."
+            exit 1
+          fi
+
+          git -c user.name=CarbonInfraBot \
+            -c user.email=64870956+CarbonInfraBot@users.noreply.github.com \
+            commit --all --quiet --message='Apply fixes from updated prek hooks'
+          remote="https://x-access-token:${PUSH_TOKEN}@github.com"
+          git push "$remote/$GITHUB_REPOSITORY.git" "HEAD:refs/heads/$BRANCH"

.github/workflows/sync_repos.yaml

@@ -25,13 +25,13 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: audit
 
       # Checkout our main repository.
       - name: Checkout the main repository
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 

.github/workflows/tests.yaml

@@ -29,7 +29,7 @@ jobs:
       matrix:
         # Test a recent version of each supported OS, covering both x86-64 and
         # AArch64: Linux on x86-64 and AArch64, and macOS on AArch64.
-        runner: ['ubuntu-22.04', 'ubuntu-22.04-arm', 'macos-14']
+        runner: ['ubuntu-22.04', 'ubuntu-22.04-arm', 'macos-26']
         # Create a synthetic matrix dimension with the event name for filtering.
         event: ['${{ github.event_name }}']
         config:
@@ -40,7 +40,7 @@ jobs:
           - name: 'ASan'
             flags: '--config=asan'
         exclude:
-          - runner: 'macos-14'
+          - runner: 'macos-26'
             config: { name: 'ASan', flags: '--config=asan' }
           - event: 'pull_request'
             config: { name: 'ASan', flags: '--config=asan' }
@@ -49,7 +49,7 @@ jobs:
     runs-on: ${{ matrix.runner }}
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           egress-policy: block
           # When adding endpoints, see README.md.
@@ -64,24 +64,29 @@ jobs:
             download.gnome.org:443
             downloads.sourceforge.net:443
             files.pythonhosted.org:443
+            formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
             oauth2.googleapis.com:443
             objects.githubusercontent.com:443
+            pkg-containers.githubusercontent.com:443
             pypi.org:443
             raw.githubusercontent.com:443
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
             releases.astral.sh:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443
 
-      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
 
@@ -93,7 +98,11 @@ jobs:
             ${{ github.event_name == 'pull_request' &&
             github.event.pull_request.base.sha ||
             github.event.merge_group.base_sha }}
-          remote_cache_key: ${{ secrets.CARBON_BUILDS_GITHUB }}
+          # The remote cache key is only used outside of PRs, so only make it
+          # available on those events. PRs get an empty string.
+          remote_cache_key:
+            ${{ (github.event_name != 'pull_request' &&
+            secrets.CARBON_BUILDS_GITHUB) || '' }}
           targets_file: ${{ runner.temp }}/targets
 
       # Build and run just the tests impacted by the PR or merge group.

.github/workflows/triage_inactive.yaml

@@ -16,7 +16,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
+        uses: step-security/harden-runner@46b06cea07f9e64621fdf066ba77ac6915163d43 # v2.22.3
         with:
           disable-sudo: true
           egress-policy: block
@@ -24,7 +24,7 @@ jobs:
           allowed-endpoints: >
             api.github.com:443
 
-      - uses: actions/stale@28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9.0.0
+      - uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
         with:
           stale-issue-message: >
             We triage inactive PRs and issues in order to make it easier to find

.pre-commit-config.yaml

@@ -4,6 +4,7 @@
 
 # Update versions with:
 #   pre-commit autoupdate --freeze && pre-commit run -a
+# Dependabot also proposes monthly updates; see .github/dependabot.yml.
 # See https://pre-commit.com for more information
 # See https://pre-commit.com/hooks.html for more hooks
 
@@ -32,7 +33,7 @@ repos:
   # Run markdown linting early so that doc style and table-of-contents see the
   # linted state.
   - repo: https://github.com/rvben/rumdl-pre-commit
-    rev: v0.2.58
+    rev: b0cff962ee693b9e612c097d22e71e0cba127447 # frozen: v0.3.2
     hooks:
       - id: rumdl
         args: [--fix]
@@ -51,7 +52,7 @@ repos:
   # Re-run markdown linting to fix any issues caused by doc style and TOC. This
   # is very fast, so it shouldn't be problematic to run twice.
   - repo: https://github.com/rvben/rumdl-pre-commit
-    rev: v0.2.58
+    rev: b0cff962ee693b9e612c097d22e71e0cba127447 # frozen: v0.3.2
     hooks:
       - id: rumdl
         args: [--fix]
@@ -67,7 +68,7 @@ repos:
 
   # Formatters should be run late so that they can re-format any prior changes.
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: 0c7b6c989466a93942def1f84baf36ddfcd60c83 # frozen: v0.15.14
+    rev: 25bcf95026b96e2dd3c0aec3090c3d6ef6ca2e6c # frozen: v0.17.3
     hooks:
       - id: ruff-check
         args: [--fix]
@@ -173,8 +174,9 @@ repos:
           (?x)^(
               toolchain/.*\.cpp|
               toolchain/.*\.h|
+              toolchain/.*/testdata/.*|
               toolchain/diagnostics/check_diagnostics\.py|
-              toolchain/diagnostics/diagnostic_kind\.def
+              toolchain/diagnostics/kind\.def
           )$
         pass_filenames: false
 
@@ -188,7 +190,7 @@ repos:
         files: ^.*/BUILD$
         pass_filenames: false
   - repo: https://github.com/codespell-project/codespell
-    rev: 2ccb47ff45ad361a21071a7eedda4c37e6ae8c5a # frozen: v2.4.2
+    rev: 57b21406f092110c18776e39b0bda50d37c945c8 # frozen: v2.4.3
     hooks:
       - id: codespell
         args: ['-I', '.codespell_ignore', '--uri-ignore-words-list', '*']

MODULE.bazel

@@ -20,23 +20,28 @@ of:
         `MODULE.bazel.lock`; create a PR with those changes in order to include
         the host platform.
 
+`bazel mod deps` evaluates every module extension, but reuses results recorded
+in `MODULE.bazel.lock`. Re-evaluating some extensions can be expensive; for
+example, the Rust crates newer protobuf versions depend on require downloading
+a Rust toolchain.
+
 For updates, run `scripts/query_module_versions.py` to list the latest package
 versions.
 """
 
 module(name = "carbon")
 
-bazel_dep(name = "abseil-cpp", version = "20260107.1")
-bazel_dep(name = "bazel_skylib", version = "1.9.0")
+bazel_dep(name = "abseil-cpp", version = "20260817.0")
+bazel_dep(name = "bazel_skylib", version = "1.9.2")
 bazel_dep(name = "boost.unordered", version = "1.90.0.bcr.1")
-bazel_dep(name = "google_benchmark", version = "1.9.5")
-bazel_dep(name = "googletest", version = "1.17.0.bcr.2")
+bazel_dep(name = "google_benchmark", version = "1.10.0")
+bazel_dep(name = "googletest", version = "1.18.0.bcr.1")
 bazel_dep(name = "libpfm", version = "4.13.0")
 bazel_dep(name = "re2", version = "2025-11-05.bcr.1")
-bazel_dep(name = "rules_cc", version = "0.2.25")
-bazel_dep(name = "rules_pkg", version = "1.2.0")
+bazel_dep(name = "rules_cc", version = "0.2.27")
+bazel_dep(name = "rules_pkg", version = "1.3.0")
 bazel_dep(name = "rules_shell", version = "0.8.0")
-bazel_dep(name = "tcmalloc", version = "0.0.0-20250927-12f2552")
+bazel_dep(name = "tcmalloc", version = "0.0.0-20260818-15db23d")
 bazel_dep(name = "tree-sitter-bazel", version = "0.26.5")
 
 bazel_dep(name = "wolfd_bazel_compile_commands", version = "0.5.2", dev_dependency = True)
@@ -67,11 +72,11 @@ register_toolchains("@bazel_cc_toolchain//:all")
 register_toolchains("//toolchain/install:all")
 
 # Required for llvm-project.
-bazel_dep(name = "libxml2", version = "2.15.2")
-bazel_dep(name = "platforms", version = "1.0.0")
-bazel_dep(name = "protobuf", version = "34.0.bcr.1", repo_name = "com_google_protobuf")
-bazel_dep(name = "zlib-ng", version = "2.3.3", repo_name = "llvm_zlib")
-bazel_dep(name = "zstd", version = "1.5.7.bcr.1", repo_name = "llvm_zstd")
+bazel_dep(name = "libxml2", version = "2.15.3")
+bazel_dep(name = "platforms", version = "1.1.0")
+bazel_dep(name = "protobuf", version = "36.2", repo_name = "com_google_protobuf")
+bazel_dep(name = "zlib-ng", version = "2.3.3.bcr.2", repo_name = "llvm_zlib")
+bazel_dep(name = "zstd", version = "1.5.7.bcr.3", repo_name = "llvm_zstd")
 bazel_dep(name = "xz", version = "5.4.5.bcr.9", repo_name = "llvm_xz")
 
 ###############################################################################
@@ -85,17 +90,14 @@ git_override(
     build_file_content = "# empty",
     # We pin to specific upstream commits and try to track top-of-tree
     # reasonably closely rather than pinning to a specific release.
-    # HEAD as of 2026-09-25.
-    commit = "6ec87e7ec4b3ca6f665a02e05d0e5fe9c37cd6cc",
+    # HEAD as of 2026-11-02.
+    commit = "c744694c6ae1d12bbf7d0447b4bc91ee94a1904c",
     patch_cmds = ["echo \"module(name='llvm-raw')\" > MODULE.bazel"],
     patch_strip = 1,
     patches = [
         "//bazel/llvm_project:0001_Patch_for_mallinfo2_when_using_Bazel_build_system.patch",
-        "//bazel/llvm_project:0002_Added_Bazel_build_for_compiler_rt_fuzzer.patch",
-        "//bazel/llvm_project:0004_Introduce_basic_sources_exporting_for_libunwind.patch",
-        "//bazel/llvm_project:0005_Introduce_basic_sources_exporting_for_libcxx_and_libcxxabi.patch",
-        "//bazel/llvm_project:0006_Add_more_libc_math_excludes.patch",
         "//bazel/llvm_project:0012_Drop_dependencies_on_linux_uapi_and_pyyaml.patch",
+        "//bazel/llvm_project:0013_Exclude_additional_libc_math_builtins.patch",
     ],
     remote = "https://github.com/llvm/llvm-project.git",
 )
@@ -111,7 +113,7 @@ use_repo(llvm_project, "llvm-project")
 # Python
 ###############################################################################
 
-bazel_dep(name = "rules_python", version = "1.9.0")
+bazel_dep(name = "rules_python", version = "2.4.5")
 
 python = use_extension("@rules_python//python/extensions:python.bzl", "python")
 python.toolchain(

bazel/llvm_project/0002_Added_Bazel_build_for_compiler_rt_fuzzer.patch

@@ -1,39 +0,0 @@
-From 20d0f1fb854d9b7ce135d08575112fc58c4c7ce7 Mon Sep 17 00:00:00 2001
-From: jonmeow <jperkins@google.com>
-Date: Thu, 14 Sep 2023 20:31:11 +0000
-Subject: [PATCH] Add libfuzzer target to compiler-rt.
-
----
- .../compiler-rt/BUILD.bazel                     | 17 +++++++++++++++++
- 1 file changed, 17 insertions(+)
-
-diff --git a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-index 90264449de76..115da4cb77f6 100644
---- a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-+++ b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-@@ -57,6 +57,23 @@ cc_library(
-     ],
- )
-
-+cc_library(
-+    name = "FuzzerMain",
-+    srcs = glob(
-+        ["lib/fuzzer/Fuzzer*.cpp"],
-+    ),
-+    hdrs = glob([
-+        "lib/fuzzer/Fuzzer*.h",
-+        "lib/fuzzer/Fuzzer*.def",
-+    ]),
-+    copts = [
-+        # Not using no-sanitize=address per https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow#false-positives
-+        "-fno-sanitize=memory,thread,undefined",
-+        "-fsanitize-coverage=0",
-+    ],
-+    includes = ["lib/fuzzer"],
-+)
-+
- cc_library(
-     name = "orc_rt_common_headers",
-     hdrs = [
---
-2.42.0

bazel/llvm_project/0004_Introduce_basic_sources_exporting_for_libunwind.patch

@@ -1,64 +0,0 @@
-Commit ID: 354e38c89f28e2cc284e655a9cde707f457dc02c
-Change ID: sxspxmonsuvqzuvxvrvorlumwpwromsv
-Author   : Chandler Carruth <chandlerc@gmail.com> (2025-09-25 22:55:26)
-Committer: Chandler Carruth <chandlerc@gmail.com> (2026-02-14 03:46:06)
-
-    Introduce basic sources exporting for libunwind
-
-    This exports the source files directly so that they can be used to build
-    this runtime library on demand.
-
-diff --git a/utils/bazel/llvm-project-overlay/libunwind/BUILD.bazel b/utils/bazel/llvm-project-overlay/libunwind/BUILD.bazel
-index c9fdc819c0..7d734c5a06 100644
---- a/utils/bazel/llvm-project-overlay/libunwind/BUILD.bazel
-+++ b/utils/bazel/llvm-project-overlay/libunwind/BUILD.bazel
-@@ -21,3 +21,19 @@
-     ],
-     strip_include_prefix = "include",
- )
-+
-+filegroup(
-+    name = "libunwind_hdrs",
-+    srcs = glob(["include/**/*.h"]),
-+)
-+
-+filegroup(
-+    name = "libunwind_srcs",
-+    srcs = glob([
-+        "src/*.cpp",
-+        "src/*.hpp",
-+        "src/*.c",
-+        "src/*.h",
-+        "src/*.S",
-+    ]),
-+)
-diff --git a/utils/bazel/llvm-project-overlay/libunwind/libunwind_library.bzl b/utils/bazel/llvm-project-overlay/libunwind/libunwind_library.bzl
-new file mode 100644
-index 0000000000..25675d3070
---- /dev/null
-+++ b/utils/bazel/llvm-project-overlay/libunwind/libunwind_library.bzl
-@@ -0,0 +1,24 @@
-+# This file is licensed under the Apache License v2.0 with LLVM Exceptions.
-+# See https://llvm.org/LICENSE.txt for license information.
-+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-+
-+"""Starlark variables and macros for building libunwind.
-+
-+Variables provide base line information for how to build libunwind source files.
-+These can be used to generate non-Bazel builds of the library.
-+
-+Macros provide a convenient way to construct Bazel `cc_library` rules for
-+libunwind.
-+"""
-+
-+# TODO: Should libunwind use `-fvisibility-inlines-hidden` and
-+# `-fvisibility=hidden`, similar to libc++?
-+libunwind_copts = [
-+    "-D_LIBUNWIND_IS_NATIVE_ONLY",
-+    "-O3",
-+    "-fPIC",
-+    "-fno-exceptions",
-+    "-fno-rtti",
-+    "-funwind-tables",
-+    "-nostdinc++",
-+]

bazel/llvm_project/0005_Introduce_basic_sources_exporting_for_libcxx_and_libcxxabi.patch

@@ -1,236 +0,0 @@
-Commit ID: 1fd710ed69a0f47f454c386d39302ddb756a88b3
-Change ID: mstnwoqruyypnoouksnyqssllrsozpos
-Bookmarks: bz-libcxx* bz-libcxx@git
-Author   : Chandler Carruth <chandlerc@gmail.com> (2025-09-25 22:55:26)
-Committer: Chandler Carruth <chandlerc@gmail.com> (2026-03-08 07:41:54)
-
-    Introduce basic sources exporting for libcxx and libcxxabi
-
-    This exports the source files directly so that they can be used to build
-    a libcxx runtime library on demand. It also differentiates between
-    normal sources and textual sources.
-
-diff --git a/utils/bazel/llvm-project-overlay/libcxx/BUILD.bazel b/utils/bazel/llvm-project-overlay/libcxx/BUILD.bazel
-new file mode 100644
-index 0000000000..c8b517ab56
---- /dev/null
-+++ b/utils/bazel/llvm-project-overlay/libcxx/BUILD.bazel
-@@ -0,0 +1,139 @@
-+# This file is licensed under the Apache License v2.0 with LLVM Exceptions.
-+# See https://llvm.org/LICENSE.txt for license information.
-+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-+
-+licenses(["notice"])
-+
-+package(
-+    default_visibility = ["//visibility:public"],
-+)
-+
-+exports_files([
-+    "include/__config_site.in",
-+    "include/module.modulemap.in",
-+    "vendor/llvm/default_assertion_handler.in",
-+])
-+
-+filegroup(
-+    name = "libcxx_hdrs",
-+    srcs = glob(
-+        [
-+            # Top level includes and those in `experimental` and `ext` sometimes
-+            # have no extension.
-+            "include/*",
-+            "include/experimental/*",
-+            "include/ext/*",
-+
-+            # Implementation detail headers all use `.h` extensions
-+            "include/**/*.h",
-+        ],
-+        exclude = [
-+            # Omit CMake and CMake-configured files that get caught by the
-+            # extension-less patterns.
-+            "**/*.in",
-+            "**/CMakeLists.txt",
-+
-+            # Omit C++03 compatibility headers as current users don't need them.
-+            "include/__cxx03/**",
-+        ],
-+    ),
-+)
-+
-+LIBCXX_SRCS_PSTL_LIBDISPATCH = [
-+    "src/pstl/libdispatch.cpp",
-+]
-+
-+filegroup(
-+    name = "libcxx_srcs_pstl_libdispatch",
-+    srcs = LIBCXX_SRCS_PSTL_LIBDISPATCH,
-+)
-+
-+LIBCXX_SRCS_SUPPORT_IBM_PATTERNS = [
-+    "src/support/ibm/**/*.cpp",
-+]
-+
-+filegroup(
-+    name = "libcxx_srcs_support_ibm",
-+    srcs = glob(LIBCXX_SRCS_SUPPORT_IBM_PATTERNS),
-+)
-+
-+LIBCXX_SRCS_SUPPORT_WIN32_PATTERNS = [
-+    "src/support/win32/**/*.cpp",
-+]
-+
-+filegroup(
-+    name = "libcxx_srcs_support_win32",
-+    srcs = glob(LIBCXX_SRCS_SUPPORT_WIN32_PATTERNS),
-+)
-+
-+LIBCXX_SRCS_TZDB = [
-+    "src/experimental/chrono_exception.cpp",
-+    "src/experimental/time_zone.cpp",
-+    "src/experimental/tzdb.cpp",
-+    "src/experimental/tzdb_list.cpp",
-+]
-+
-+filegroup(
-+    name = "libcxx_srcs_tzdb",
-+    srcs = LIBCXX_SRCS_TZDB,
-+)
-+
-+# Exclude platform-dependent patterns that are provided by per-target filegroups
-+# above.
-+LIBCXX_SRCS_TARGET_EXCLUDES = (
-+    LIBCXX_SRCS_PSTL_LIBDISPATCH +
-+    LIBCXX_SRCS_SUPPORT_IBM_PATTERNS +
-+    LIBCXX_SRCS_SUPPORT_WIN32_PATTERNS +
-+    LIBCXX_SRCS_TZDB
-+)
-+
-+filegroup(
-+    name = "libcxx_srcs_generic",
-+    srcs = glob(
-+        [
-+            "src/**/*.cpp",
-+            "src/**/*.h",
-+            "src/**/*.ipp",
-+        ],
-+        exclude = [
-+            # Build is for use with libc++abi and so don't need 'new.cpp'.
-+            "src/new.cpp",
-+
-+            # Build is for compiler-rt platforms so we have its int128 support.
-+            "src/filesystem/int128_builtins.cpp",
-+        ] + LIBCXX_SRCS_TARGET_EXCLUDES,
-+    ),
-+)
-+
-+filegroup(
-+    name = "libcxx_linux_srcs",
-+    srcs = [
-+        ":libcxx_srcs_generic",
-+        ":libcxx_srcs_tzdb",
-+    ],
-+)
-+
-+filegroup(
-+    name = "libcxx_macos_srcs",
-+    srcs = [
-+        ":libcxx_srcs_generic",
-+        # TODO: Include libdispatch sources here to enable that pstl backend.
-+    ],
-+)
-+
-+filegroup(
-+    name = "libcxx_win32_srcs",
-+    srcs = [
-+        ":libcxx_srcs_generic",
-+        ":libcxx_srcs_support_win32",
-+    ],
-+)
-+
-+filegroup(
-+    name = "libcxx_all_srcs",
-+    srcs = [
-+        ":libcxx_linux_srcs",
-+        ":libcxx_macos_srcs",
-+        ":libcxx_win32_srcs",
-+    ],
-+)
-diff --git a/utils/bazel/llvm-project-overlay/libcxx/libcxx_library.bzl b/utils/bazel/llvm-project-overlay/libcxx/libcxx_library.bzl
-new file mode 100644
-index 0000000000..66f64f3610
---- /dev/null
-+++ b/utils/bazel/llvm-project-overlay/libcxx/libcxx_library.bzl
-@@ -0,0 +1,37 @@
-+# This file is licensed under the Apache License v2.0 with LLVM Exceptions.
-+# See https://llvm.org/LICENSE.txt for license information.
-+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-+
-+"""Starlark variables and macros for building libc++ and libc++abi.
-+
-+Variables provide base line information for how to build libc++ and libc++abi
-+source files. These can be used to generate non-Bazel builds of the library.
-+
-+TODO: Add macros that provide a convenient way to construct Bazel `cc_library`
-+rules for libc++ and libc++abi.
-+
-+TODO: Add either sufficient usage in the macros, or add a how-to example here in
-+the documentation so the use of these variables is more clear.
-+"""
-+
-+_libcxx_base_copts = [
-+    "-std=c++26",
-+    "-O3",
-+    "-fPIC",
-+    "-fvisibility-inlines-hidden",
-+    "-fvisibility=hidden",
-+    "-nostdinc++",
-+]
-+
-+_libcxx_defines = [
-+    "-D_LIBCPP_BUILDING_LIBRARY",
-+    "-D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES",
-+]
-+
-+_libcxxabi_defines = [
-+    "-DLIBCXX_BUILDING_LIBCXXABI",
-+]
-+
-+libcxx_copts = _libcxx_base_copts + _libcxx_defines
-+libcxxabi_copts = _libcxx_base_copts + _libcxxabi_defines
-+libcxx_and_abi_copts = _libcxx_base_copts + _libcxx_defines + _libcxxabi_defines
-diff --git a/utils/bazel/llvm-project-overlay/libcxxabi/BUILD.bazel b/utils/bazel/llvm-project-overlay/libcxxabi/BUILD.bazel
-new file mode 100644
-index 0000000000..2db70b54e2
---- /dev/null
-+++ b/utils/bazel/llvm-project-overlay/libcxxabi/BUILD.bazel
-@@ -0,0 +1,30 @@
-+# This file is licensed under the Apache License v2.0 with LLVM Exceptions.
-+# See https://llvm.org/LICENSE.txt for license information.
-+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-+
-+licenses(["notice"])
-+
-+package(
-+    default_visibility = ["//visibility:public"],
-+)
-+
-+filegroup(
-+    name = "libcxxabi_hdrs",
-+    srcs = glob(["include/*.h"]),
-+)
-+
-+filegroup(
-+    name = "libcxxabi_srcs",
-+    srcs = glob([
-+        "src/**/*.cpp",
-+        "src/**/*.h",
-+    ]),
-+)
-+
-+filegroup(
-+    name = "libcxxabi_textual_srcs",
-+    srcs = glob([
-+        "src/**/*.def",
-+        "src/**/*.inc",
-+    ]),
-+)

bazel/llvm_project/0006_Add_more_libc_math_excludes.patch

@@ -1,37 +0,0 @@
-Removes additional libc-backed arithmetic builtins added
-by https://github.com/llvm/llvm-project/pull/207092
-and https://github.com/llvm/llvm-project/pull/209984
----
---- a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-+++ b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
-@@ -303,6 +303,30 @@
-     "lib/builtins/extendsfdf2.cpp",
-     "lib/builtins/extendsftf2.cpp",
-     "lib/builtins/extendxftf2.cpp",
-+    "lib/builtins/fixdfdi.cpp",
-+    "lib/builtins/fixdfsi.cpp",
-+    "lib/builtins/fixdfti.cpp",
-+    "lib/builtins/fixsfdi.cpp",
-+    "lib/builtins/fixsfsi.cpp",
-+    "lib/builtins/fixsfti.cpp",
-+    "lib/builtins/fixunsdfdi.cpp",
-+    "lib/builtins/fixunsdfsi.cpp",
-+    "lib/builtins/fixunsdfti.cpp",
-+    "lib/builtins/fixunssfdi.cpp",
-+    "lib/builtins/fixunssfsi.cpp",
-+    "lib/builtins/fixunssfti.cpp",
-+    "lib/builtins/floatdidf.cpp",
-+    "lib/builtins/floatdisf.cpp",
-+    "lib/builtins/floatsidf.cpp",
-+    "lib/builtins/floatsisf.cpp",
-+    "lib/builtins/floattidf.cpp",
-+    "lib/builtins/floattisf.cpp",
-+    "lib/builtins/floatundidf.cpp",
-+    "lib/builtins/floatundisf.cpp",
-+    "lib/builtins/floatunsidf.cpp",
-+    "lib/builtins/floatunsisf.cpp",
-+    "lib/builtins/floatuntidf.cpp",
-+    "lib/builtins/floatuntisf.cpp",
-     "lib/builtins/muldf3.cpp",
-     "lib/builtins/mulsf3.cpp",
-     "lib/builtins/multf3.cpp",

bazel/llvm_project/0013_Exclude_additional_libc_math_builtins.patch

@@ -0,0 +1,33 @@
+diff --git a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
+index 2f3eeb68dd..16d3c92349 100644
+--- a/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
++++ b/utils/bazel/llvm-project-overlay/compiler-rt/BUILD.bazel
+@@ -317,12 +317,15 @@ BUILTINS_LIBC_MATH_EXCLUDES = [
+ BUILTINS_LIBC_MATH_EXCLUDES = [
+     # Sources requiring COMPILER_RT_USE_LIBC_MATH (e.g. LIBC_NAMESPACE::shared::*).
+     "lib/builtins/adddf3.cpp",
++    "lib/builtins/addsf3.cpp",
+     "lib/builtins/addtf3.cpp",
+-    "lib/builtins/addsf3.cpp",
+     "lib/builtins/divdf3.cpp",
+     "lib/builtins/divsf3.cpp",
+     "lib/builtins/divtf3.cpp",
+     "lib/builtins/extenddftf2.cpp",
++    "lib/builtins/extendhfdf2.cpp",
++    "lib/builtins/extendhfsf2.cpp",
++    "lib/builtins/extendhftf2.cpp",
+     "lib/builtins/extendsfdf2.cpp",
+     "lib/builtins/extendsftf2.cpp",
+     "lib/builtins/extendxftf2.cpp",
+@@ -358,8 +361,11 @@ BUILTINS_LIBC_MATH_EXCLUDES = [
+     "lib/builtins/subdf3.cpp",
+     "lib/builtins/subsf3.cpp",
+     "lib/builtins/subtf3.cpp",
++    "lib/builtins/truncdfhf2.cpp",
+     "lib/builtins/truncdfsf2.cpp",
++    "lib/builtins/truncsfhf2.cpp",
+     "lib/builtins/trunctfdf2.cpp",
++    "lib/builtins/trunctfhf2.cpp",
+     "lib/builtins/trunctfsf2.cpp",
+     "lib/builtins/trunctfxf2.cpp",
+ ]

common/BUILD

@@ -113,6 +113,12 @@ cc_test(
     ],
 )
 
+cc_library(
+    name = "checked_save_and_restore",
+    hdrs = ["checked_save_and_restore.h"],
+    deps = [":check"],
+)
+
 cc_library(
     name = "concepts",
     hdrs = ["concepts.h"],
@@ -404,6 +410,12 @@ cc_test(
     ],
 )
 
+cc_library(
+    name = "increment_scope",
+    hdrs = ["increment_scope.h"],
+    deps = [":check"],
+)
+
 cc_library(
     name = "init_llvm",
     srcs = ["init_llvm.cpp"],
@@ -534,6 +546,25 @@ cc_test(
     ],
 )
 
+cc_library(
+    name = "preserve_none",
+    hdrs = ["preserve_none.h"],
+    deps = [
+        "@llvm-project//llvm:Support",
+    ],
+)
+
+cc_test(
+    name = "preserve_none_test",
+    size = "small",
+    srcs = ["preserve_none_test.cpp"],
+    deps = [
+        ":preserve_none",
+        "//testing/base:gtest_main",
+        "@googletest//:gtest",
+    ],
+)
+
 cc_library(
     name = "pretty_stack_trace_function",
     hdrs = ["pretty_stack_trace_function.h"],

common/checked_save_and_restore.h

@@ -0,0 +1,51 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_CHECKED_SAVE_AND_RESTORE_H_
+#define CARBON_COMMON_CHECKED_SAVE_AND_RESTORE_H_
+
+#include <utility>
+
+#include "common/check.h"
+
+namespace Carbon {
+
+// Sets a variable to a new value, and restores the original value when
+// the CheckedSaveAndRestore is destroyed. CHECKs that the variable is still
+// holding the new value when restoring the original value, to catch any data
+// loss.
+//
+// The value must be copyable in order to compare with the new value at the time
+// the CheckedSaveAndRestore is destroyed.
+template <typename T>
+class [[nodiscard]] CheckedSaveAndRestore {
+ public:
+  CheckedSaveAndRestore(T& var, const T& new_value)
+      : var_(&var),
+        new_value_(new_value),
+        orig_value_(std::exchange(var, new_value_)) {}
+
+  ~CheckedSaveAndRestore() {
+    CARBON_CHECK(*var_ == new_value_);
+    *var_ = std::move(orig_value_);
+  }
+
+  CheckedSaveAndRestore(const CheckedSaveAndRestore&) = delete;
+  auto operator=(const CheckedSaveAndRestore&)
+      -> CheckedSaveAndRestore& = delete;
+
+ private:
+  T* var_;
+  T new_value_;
+  T orig_value_;
+};
+
+template <typename T>
+CheckedSaveAndRestore(T&, const T&) -> CheckedSaveAndRestore<T>;
+template <typename T>
+CheckedSaveAndRestore(T&, T&&) -> CheckedSaveAndRestore<T>;
+
+}  // namespace Carbon
+
+#endif  // CARBON_COMMON_CHECKED_SAVE_AND_RESTORE_H_

common/hashing.h

@@ -487,7 +487,8 @@ inline auto CarbonHashValue(llvm::ArrayRef<T> values) -> HashCode {
   return static_cast<HashCode>(hasher);
 }
 
-inline auto CarbonHashValue(llvm::ArrayRef<std::byte> bytes) -> HashCode {
+[[clang::always_inline]] inline auto CarbonHashValue(
+    llvm::ArrayRef<std::byte> bytes) -> HashCode {
   Hasher hasher;
   hasher.HashSizedBytes(bytes);
   return static_cast<HashCode>(hasher);
@@ -495,16 +496,19 @@ inline auto CarbonHashValue(llvm::ArrayRef<std::byte> bytes) -> HashCode {
 
 // Hashing implementation for `llvm::StringRef`. We forward all the other
 // string-like types that support heterogeneous lookup to this one.
-inline auto CarbonHashValue(llvm::StringRef value) -> HashCode {
+[[clang::always_inline]] inline auto CarbonHashValue(llvm::StringRef value)
+    -> HashCode {
   return CarbonHashValue(llvm::ArrayRef(
       reinterpret_cast<const std::byte*>(value.data()), value.size()));
 }
 
-inline auto CarbonHashValue(std::string_view value) -> HashCode {
+[[clang::always_inline]] inline auto CarbonHashValue(std::string_view value)
+    -> HashCode {
   return CarbonHashValue(llvm::StringRef(value.data(), value.size()));
 }
 
-inline auto CarbonHashValue(const std::string& value) -> HashCode {
+[[clang::always_inline]] inline auto CarbonHashValue(const std::string& value)
+    -> HashCode {
   return CarbonHashValue(llvm::StringRef(value.data(), value.size()));
 }
 
@@ -607,7 +611,7 @@ concept CanHashAsRawDataType = std::same_as<T, std::nullptr_t> ||
 // overloads, either here, or via ADL. Note that similar to
 // `HasCarbonHashValue`, this must not be moved above any of those overloads.
 template <typename T>
-inline auto DispatchImpl(const T& value) -> HashCode {
+[[clang::always_inline]] inline auto DispatchImpl(const T& value) -> HashCode {
   if constexpr (HasCarbonHashValue<T>) {
     // If we have an explicit overload for `CarbonHashValue`, call it. This may
     // be provided above or via ADL, and is preferred as it represents an
@@ -638,7 +642,7 @@ inline auto DispatchImpl(const T& value) -> HashCode {
 }  // namespace InternalHashDispatch
 
 template <typename T>
-inline auto HashValue(const T& value) -> HashCode {
+[[clang::always_inline]] inline auto HashValue(const T& value) -> HashCode {
   return InternalHashDispatch::DispatchImpl(value);
 }
 
@@ -931,7 +935,8 @@ inline auto Hasher::HashRaw(const T& value) -> void {
   HashSizedBytesLarge(llvm::ArrayRef<std::byte>(data_ptr, sizeof(T)));
 }
 
-inline auto Hasher::HashSizedBytes(llvm::ArrayRef<std::byte> bytes) -> void {
+[[clang::always_inline]] inline auto Hasher::HashSizedBytes(
+    llvm::ArrayRef<std::byte> bytes) -> void {
   const std::byte* data_ptr = bytes.data();
   const ssize_t size = bytes.size();
 

common/increment_scope.h

@@ -0,0 +1,36 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_INCREMENT_SCOPE_H_
+#define CARBON_COMMON_INCREMENT_SCOPE_H_
+
+#include <cstdint>
+
+#include "common/check.h"
+
+namespace Carbon {
+
+// An RAII object that increments a value while it remains in scope,
+// decrementing it when it is destroyed.
+class [[nodiscard]] IncrementScope {
+ public:
+  explicit IncrementScope(int32_t& value [[clang::lifetimebound]])
+      : value_(&value), before_(value) {
+    (*value_)++;
+  }
+  ~IncrementScope() {
+    (*value_)--;
+    CARBON_CHECK(*value_ == before_,
+                 "IncrementScope value not restored - incorrect bracketing of "
+                 "increments and decrements?");
+  }
+
+ private:
+  int32_t* value_;
+  int32_t before_;
+};
+
+}  // namespace Carbon
+
+#endif  // CARBON_COMMON_INCREMENT_SCOPE_H_

common/preserve_none.h

@@ -0,0 +1,25 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef CARBON_COMMON_PRESERVE_NONE_H_
+#define CARBON_COMMON_PRESERVE_NONE_H_
+
+#include "llvm/Support/Compiler.h"
+
+// Applies `[[clang::preserve_none]]` when supported and safe.
+//
+// Disabled when building with AddressSanitizer on AArch64 to work around an
+// LLVM backend bug: when sanitizer stack instrumentation combines dynamic stack
+// allocation with stack realignment, AArch64 reserves `x19` as the stack base
+// pointer (`hasBasePointer`) and fails to save and restore `x19` across calls
+// to `[[clang::preserve_none]]` callees that clobber `x19`.
+#if __has_cpp_attribute(clang::preserve_none) &&     \
+    !((defined(__aarch64__) || defined(_M_ARM64)) && \
+      (LLVM_ADDRESS_SANITIZER_BUILD || LLVM_HWADDRESS_SANITIZER_BUILD))
+#define CARBON_PRESERVE_NONE [[clang::preserve_none]]
+#else
+#define CARBON_PRESERVE_NONE
+#endif
+
+#endif  // CARBON_COMMON_PRESERVE_NONE_H_

common/preserve_none_test.cpp

@@ -0,0 +1,48 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "common/preserve_none.h"
+
+#include <gtest/gtest.h>
+
+#include <array>
+
+namespace Carbon {
+namespace {
+
+using DispatchFn = auto(int value, int remaining) CARBON_PRESERVE_NONE->int;
+
+CARBON_PRESERVE_NONE static auto DispatchDone(int value, int /*remaining*/)
+    -> int {
+  return value;
+}
+
+CARBON_PRESERVE_NONE static auto DispatchStep(int value, int remaining) -> int;
+
+constexpr std::array<DispatchFn*, 2> DispatchTable = {
+    &DispatchDone,
+    &DispatchStep,
+};
+
+CARBON_PRESERVE_NONE static auto DispatchStep(int value, int remaining) -> int {
+  // Touch enough temporaries that register allocation exercises caller-saved
+  // registers under `preserve_none`.
+  int next = (value * 3 + 1) ^ (remaining * 7);
+  int next_remaining = remaining - 1;
+  [[clang::musttail]] return DispatchTable[next_remaining != 0 ? 1 : 0](
+      next, next_remaining);
+}
+
+TEST(PreserveNoneTest, TailCallDispatchWithAlignedCallerStack) {
+  // Overalign a local buffer so sanitizer stack instrumentation exercises stack
+  // realignment alongside `CARBON_PRESERVE_NONE` calls.
+  alignas(64) std::array<int, 8> scratch = {1, 2, 3, 4, 5, 6, 7, 8};
+  DispatchFn* entry = &DispatchStep;
+  int result = entry(scratch[0], 16);
+  EXPECT_EQ(scratch[7], 8);
+  EXPECT_NE(result, 0);
+}
+
+}  // namespace
+}  // namespace Carbon

common/set.h

@@ -366,7 +366,9 @@ auto SetBase<InputKeyT, InputKeyContextT>::Insert(LookupKeyT lookup_key,
   return Insert(
       lookup_key,
       [&key_cb](LookupKeyT /*lookup_key*/, void* key_storage) {
-        new (key_storage) KeyT(key_cb());
+        // Despite being on the cold path, inlining is essential to keep
+        // fast-path register allocation efficient.
+        [[clang::always_inline]] new (key_storage) KeyT(key_cb());
       },
       key_context);
 }
@@ -386,7 +388,10 @@ auto SetBase<InputKeyT, InputKeyContextT>::Insert(LookupKeyT lookup_key,
     return InsertResult(false, entry->key());
   }
 
-  insert_cb(lookup_key, static_cast<void*>(&entry->key_storage));
+  // Despite being on the cold path, inlining is essential to keep fast-path
+  // register allocation efficient.
+  [[clang::always_inline]] insert_cb(lookup_key,
+                                     static_cast<void*>(&entry->key_storage));
   return InsertResult(true, entry->key());
 }
 

common/terminal/buffer.cpp

@@ -7,14 +7,13 @@
 #include <algorithm>
 #include <array>
 #include <cstdint>
+#include <string_view>
 #include <utility>
 
 #include "common/check.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/Sequence.h"
 #include "llvm/ADT/SmallString.h"
-#include "llvm/Support/ConvertUTF.h"
-#include "llvm/Support/Unicode.h"
 
 namespace Carbon::Terminal {
 
@@ -23,59 +22,77 @@ namespace Carbon::Terminal {
 // let a single column of output carry unbounded bytes.
 static constexpr size_t MaxCombiningBytes = 32;
 
-// Glyphs for every combination of line directions, indexed by the direction
-// bits.
-static constexpr std::array<char32_t, 16> Utf8LineGlyphs = {
-    U'·',  // (none): a line between one center and itself, which is a point
-    U'╴',  // left
-    U'╶',  // right
-    U'─',  // left, right
-    U'╵',  // up
-    U'╯',  // left, up
-    U'╰',  // right, up
-    U'┴',  // left, right, up
-    U'╷',  // down
-    U'╮',  // left, down
-    U'╭',  // right, down
-    U'┬',  // left, right, down
-    U'│',  // up, down
-    U'┤',  // left, up, down
-    U'├',  // right, up, down
-    U'┼',  // left, right, up, down
-};
-
-// The ASCII stand-ins. Each keeps the axis its line runs through, which leaves
-// `+` meaning a crossing and nothing else:
+// Glyphs for every combination of line directions and weights, indexed by the
+// heavy bits and then the direction bits.
 //
-// - Running through horizontally is `-`, vertically `|`, and both ways `+`.
-// - A tee keeps its through-stroke and leaves the branch to what is drawn
-//   beside it: the dashes either side of a `|` are what `├` and `┤` reach, and
-//   the line under a `-` is what a `┬` reaches. Drawing a tee as `+` reads as
-//   the crossing it is not.
+// The first row is the all-light set, whose corners are the rounded forms; a
+// corner with any heavy line is square, since Unicode has no rounded heavy
+// corner. The other rows match the Unicode character names, which list the
+// weight of the line in each direction. `DrawLine` never sets a heavy bit
+// without its direction bit, and those entries repeat the glyph with the
+// direction present.
+static constexpr std::array<std::u32string_view, 16> Utf8LineGlyphs = {
+    U"·╴╶─╵╯╰┴╷╮╭┬│┤├┼",  // All light.
+    U"╸╸╾╾┙┙┵┵┑┑┭┭┥┥┽┽",  // Heavy left.
+    U"╺╼╺╼┕┶┕┶┍┮┍┮┝┾┝┾",  // Heavy right.
+    U"━━━━┷┷┷┷┯┯┯┯┿┿┿┿",  // Heavy left and right.
+    U"╹┚┖┸╹┚┖┸╿┦┞╀╿┦┞╀",  // Heavy up.
+    U"┛┛┹┹┛┛┹┹┩┩╃╃┩┩╃╃",  // Heavy left and up.
+    U"┗┺┗┺┗┺┗┺┡╄┡╄┡╄┡╄",  // Heavy right and up.
+    U"┻┻┻┻┻┻┻┻╇╇╇╇╇╇╇╇",  // Heavy left, right, and up.
+    U"╻┒┎┰╽┧┟╁╻┒┎┰╽┧┟╁",  // Heavy down.
+    U"┓┓┱┱┪┪╅╅┓┓┱┱┪┪╅╅",  // Heavy left and down.
+    U"┏┲┏┲┢╆┢╆┏┲┏┲┢╆┢╆",  // Heavy right and down.
+    U"┳┳┳┳╈╈╈╈┳┳┳┳╈╈╈╈",  // Heavy left, right, and down.
+    U"┃┨┠╂┃┨┠╂┃┨┠╂┃┨┠╂",  // Heavy up and down.
+    U"┫┫╉╉┫┫╉╉┫┫╉╉┫┫╉╉",  // Heavy left, up, and down.
+    U"┣╊┣╊┣╊┣╊┣╊┣╊┣╊┣╊",  // Heavy right, up, and down.
+    U"╋╋╋╋╋╋╋╋╋╋╋╋╋╋╋╋",  // All heavy.
+};
+// Every row has a glyph for each set of direction bits.
+static_assert(std::all_of(Utf8LineGlyphs.begin(), Utf8LineGlyphs.end(),
+                          [](std::u32string_view glyphs) {
+                            return glyphs.size() == 16;
+                          }));
+
+// The ASCII glyphs, which use `+` only for a crossing:
+//
+// - A line running through horizontally is `-`, vertically `|`, and both ways
+//   `+`.
 // - A corner is `.` where its line leaves downward and `'` where it arrives
 //   from above, which is where those characters sit in their cells.
+// - A tee where a line leaves a vertical line sideways, `├` or `┤`, is `|`,
+//   and the dash drawn beside it shows the line leaving. A tee where a line
+//   leaves a horizontal line upward or downward, `┴` or `┬`, is the corner
+//   glyph for that direction: the dashes either side show the horizontal line,
+//   and this cell must show the line leaving it, because the rest of that line
+//   can be rows away with only gaps between. `+` would look like a crossing.
 // - A point, a line between one center and itself, is `.`.
 //
-// What a diagnostic draws is then still told apart: the rule closing a snippet
-// from the one separating two, and the anchor opening a diagnostic from the one
-// carrying it on.
+// What a diagnostic draws is then still told apart: the line closing a snippet
+// from the one separating two, and the line opening a diagnostic from the one
+// continuing it.
+//
+// A cell with a heavy line leaving it is `^` instead, under the rule
+// `LineWeight` describes. That covers the top cell of a heavy vertical line, so
+// the line is `^` above `|`.
 static constexpr std::array<char32_t, 16> AsciiLineGlyphs = {
-    U'.',   // (none): a point
-    U'-',   // left
-    U'-',   // right
-    U'-',   // left, right
-    U'|',   // up
-    U'\'',  // left, up
-    U'\'',  // right, up
-    U'-',   // left, right, up
-    U'|',   // down
-    U'.',   // left, down
-    U'.',   // right, down
-    U'-',   // left, right, down
-    U'|',   // up, down
-    U'|',   // left, up, down
-    U'|',   // right, up, down
-    U'+',   // left, right, up, down
+    U'.',   // None: a point.
+    U'-',   // Left.
+    U'-',   // Right.
+    U'-',   // Left and right.
+    U'|',   // Up.
+    U'\'',  // Left and up.
+    U'\'',  // Right and up.
+    U'\'',  // Left, right, and up.
+    U'|',   // Down.
+    U'.',   // Left and down.
+    U'.',   // Right and down.
+    U'.',   // Left, right, and down.
+    U'|',   // Up and down.
+    U'|',   // Left, up, and down.
+    U'|',   // Right, up, and down.
+    U'+',   // Left, right, up, and down.
 };
 
 // Returns the next tab stop after `x` on a line whose stops are `tab_width`
@@ -241,13 +258,8 @@ auto Buffer::PlaceCodePoint(int x, int y, char32_t code_point,
   return x + width;
 }
 
-// Returns the glyphs a cell's directions are read from.
-static auto LineGlyphs(Charset charset) -> const std::array<char32_t, 16>& {
-  return charset == Charset::Utf8 ? Utf8LineGlyphs : AsciiLineGlyphs;
-}
-
-auto Buffer::DrawLine(int x, int y, uint8_t directions, const Style& style)
-    -> void {
+auto Buffer::DrawLine(int x, int y, uint8_t directions, LineWeight weight,
+                      const Style& style) -> void {
   CARBON_DCHECK(directions <= LineDirections,
                 "Direction bits {0} name no glyph.", directions);
   EnsureColumn(x);
@@ -262,7 +274,22 @@ auto Buffer::DrawLine(int x, int y, uint8_t directions, const Style& style)
 
   Cell& cell = CellAt(x, y);
   cell.lines = existing | directions | LineCell;
-  cell.code_point = LineGlyphs(metrics_.charset())[cell.lines & LineDirections];
+  cell.heavy_lines &= ~directions;
+  if (weight == LineWeight::Heavy) {
+    cell.heavy_lines |= directions;
+  }
+  uint8_t cell_directions = cell.lines & LineDirections;
+  bool line_enters_from_above = (cell_directions & LineUp) != 0;
+  bool light_line_leaves_downward =
+      (cell_directions & LineDown) != 0 && (cell.heavy_lines & LineDown) == 0;
+  if (metrics_.charset() == Charset::Utf8) {
+    cell.code_point = Utf8LineGlyphs[cell.heavy_lines][cell_directions];
+  } else if (cell.heavy_lines != 0 && !line_enters_from_above &&
+             !light_line_leaves_downward) {
+    cell.code_point = U'^';
+  } else {
+    cell.code_point = AsciiLineGlyphs[cell_directions];
+  }
   cell.style = style;
 }
 
@@ -279,7 +306,8 @@ static auto CheckLineFits(int position, int length, int limit) -> void {
 }
 
 auto Buffer::DrawHorizontalLine(int x, int y, int length, const Style& style,
-                                LineEnd start, LineEnd end) -> DrawEnd {
+                                LineEnd start, LineEnd end, LineWeight weight)
+    -> DrawEnd {
   CheckOrigin(x, y);
   CheckLineFits(x, length, columns_);
   for (int i : llvm::seq(length)) {
@@ -289,20 +317,76 @@ auto Buffer::DrawHorizontalLine(int x, int y, int length, const Style& style,
     uint8_t directions =
         (i > 0 || start == LineEnd::Edge ? LineLeft : 0) |
         (i + 1 < length || end == LineEnd::Edge ? LineRight : 0);
-    DrawLine(x + i, y, directions, style);
+    DrawLine(x + i, y, directions, weight, style);
   }
   return {.x = x + length, .y = y};
 }
 
 auto Buffer::DrawVerticalLine(int x, int y, int length, const Style& style,
-                              LineEnd start, LineEnd end) -> DrawEnd {
+                              LineEnd start, LineEnd end, LineWeight weight)
+    -> DrawEnd {
   CheckOrigin(x, y);
   CheckLineFits(y, length, MaxRows);
   for (int i : llvm::seq(length)) {
     uint8_t directions =
         (i > 0 || start == LineEnd::Edge ? LineUp : 0) |
         (i + 1 < length || end == LineEnd::Edge ? LineDown : 0);
-    DrawLine(x, y + i, directions, style);
+    DrawLine(x, y + i, directions, weight, style);
+  }
+  return {.x = x, .y = y + length};
+}
+
+auto Buffer::HoldsText(int x, int y) const -> bool {
+  if (x < 0 || x >= width_ || y >= height()) {
+    return false;
+  }
+  const Cell& cell = CellAt(x, y);
+  if ((cell.lines & LineCell) != 0) {
+    return false;
+  }
+  if (cell.code_point != U' ' || cell.is_continuation) {
+    return true;
+  }
+  return !combining_marks_.empty() &&
+         combining_marks_.contains(CellIndex(x, y));
+}
+
+auto Buffer::DrawCrossingVerticalLine(int x, int y, int length,
+                                      const Style& style, LineEnd start)
+    -> DrawEnd {
+  CheckOrigin(x, y);
+  CheckLineFits(y, length, MaxRows);
+
+  for (int i : llvm::seq(length)) {
+    int cell_y = y + i;
+    bool is_center_start = i == 0 && start == LineEnd::Center;
+    uint8_t directions = is_center_start ? LineDown : (LineUp | LineDown);
+    if (cell_y >= height()) {
+      // Rows below the buffer hold nothing drawn earlier.
+      DrawLine(x, cell_y, directions, LineWeight::Light, style);
+      continue;
+    }
+    const Cell& cell = CellAt(x, cell_y);
+    if (is_center_start) {
+      // A line already leaving this cell downward is left as it is, keeping
+      // its weight. Otherwise this line joins the cell.
+      if ((cell.lines & LineDown) == 0) {
+        DrawLine(x, cell_y, LineDown, LineWeight::Light, style);
+      }
+      continue;
+    }
+    if ((cell.lines & LineCell) != 0) {
+      // An earlier line keeps its cell: a crossed line stays unbroken, and a
+      // line already running down this column keeps its weight.
+      continue;
+    }
+    // Skip text, and blank cells beside text, where a line could be mistaken
+    // for part of a word.
+    if (HoldsText(x, cell_y) || HoldsText(x - 1, cell_y) ||
+        HoldsText(x + 1, cell_y)) {
+      continue;
+    }
+    DrawLine(x, cell_y, directions, LineWeight::Light, style);
   }
   return {.x = x, .y = y + length};
 }
@@ -377,18 +461,49 @@ auto Buffer::MeasureText(int x, int y, int margin, llvm::StringRef text) const
 
 // Returns whether wrapped text can be broken at `c`.
 //
-// This is the one definition of where wrapping may introduce a break, so that
-// measuring what text wraps into and drawing it wrapped agree about it.
 // Carriage returns count so that a CRLF ending is whitespace rather than part
 // of the word before it; what becomes of the `\r` is then up to the drawing.
 static constexpr auto IsWrapBreak(char c) -> bool {
   return c == ' ' || c == '\t' || c == '\r';
 }
 
+// Removes the word at the front of `text` and returns it: everything up to the
+// next break or newline, except that with `backticks_are_quotes`, a pair of
+// backticks on one line keeps everything between them in the word. When
+// `text` starts with a break or a newline, returns an empty word and leaves
+// `text` unchanged, so the caller has to step past that character itself.
+//
+// With `IsWrapBreak` this is the one definition of where wrapping may
+// introduce a break, so that measuring what text wraps into and drawing it
+// wrapped agree about it.
+static auto TakeWord(llvm::StringRef& text, bool backticks_are_quotes)
+    -> llvm::StringRef {
+  size_t end = 0;
+  while (end < text.size()) {
+    char c = text[end];
+    if (c == '\n' || IsWrapBreak(c)) {
+      break;
+    }
+    if (backticks_are_quotes && c == '`') {
+      size_t close = text.find('`', end + 1);
+      if (close != llvm::StringRef::npos &&
+          text.slice(end + 1, close).find_first_of("\t\n\r") ==
+              llvm::StringRef::npos) {
+        end = close + 1;
+        continue;
+      }
+    }
+    ++end;
+  }
+  llvm::StringRef word = text.take_front(end);
+  text = text.drop_front(end);
+  return word;
+}
+
 template <typename PlaceFn>
 auto Buffer::WalkWrappedText(int x, int y, int margin, int max_width,
-                             llvm::StringRef text, PlaceFn place) const
-    -> DrawEnd {
+                             llvm::StringRef text, bool backticks_are_quotes,
+                             PlaceFn place) const -> DrawEnd {
   CheckTextSize(text);
   // The block runs from the margin to `margin + max_width`, lies within the
   // buffer, and holds the column the text starts in, which is every bound on
@@ -452,9 +567,7 @@ auto Buffer::WalkWrappedText(int x, int y, int margin, int max_width,
       continue;
     }
 
-    llvm::StringRef word =
-        text.take_until([](char c) { return c == '\n' || IsWrapBreak(c); });
-    text = text.drop_front(word.size());
+    llvm::StringRef word = TakeWord(text, backticks_are_quotes);
 
     // Move a word that doesn't fit down to the next row, which minimizes the
     // overhang when it doesn't fit there either. The word is drawn into the row
@@ -474,9 +587,9 @@ auto Buffer::WalkWrappedText(int x, int y, int margin, int max_width,
 }
 
 auto Buffer::DrawWrappedText(int x, int y, int margin, int max_width,
-                             llvm::StringRef text, const Style& style)
-    -> DrawEnd {
-  return WalkWrappedText(x, y, margin, max_width, text,
+                             llvm::StringRef text, const Style& style,
+                             bool backticks_are_quotes) -> DrawEnd {
+  return WalkWrappedText(x, y, margin, max_width, text, backticks_are_quotes,
                          [&](int cur_x, int cur_y, char32_t code_point) {
                            return PlaceCodePoint(cur_x, cur_y, code_point,
                                                  style);
@@ -484,20 +597,23 @@ auto Buffer::DrawWrappedText(int x, int y, int margin, int max_width,
 }
 
 auto Buffer::MeasureWrappedText(int x, int y, int margin, int max_width,
-                                llvm::StringRef text) const -> DrawEnd {
-  return WalkWrappedText(x, y, margin, max_width, text,
+                                llvm::StringRef text,
+                                bool backticks_are_quotes) const -> DrawEnd {
+  return WalkWrappedText(x, y, margin, max_width, text, backticks_are_quotes,
                          [&](int cur_x, int /*cur_y*/, char32_t code_point) {
                            return cur_x + metrics_.CodePointWidth(code_point);
                          });
 }
 
-auto Buffer::MeasureWrapWidth(llvm::StringRef text) const -> int {
+auto Buffer::MeasureWrapWidth(llvm::StringRef text,
+                              bool backticks_are_quotes) const -> int {
   int width = 0;
   while (!text.empty()) {
-    llvm::StringRef word =
-        text.take_until([](char c) { return c == '\n' || IsWrapBreak(c); });
+    llvm::StringRef word = TakeWord(text, backticks_are_quotes);
     width = std::max(width, metrics_.Width(word));
-    text = text.drop_front(std::max<size_t>(word.size(), 1));
+    if (word.empty()) {
+      text = text.drop_front();
+    }
   }
   return width;
 }

common/terminal/buffer.h

@@ -47,6 +47,20 @@ enum class LineEnd : int8_t {
   Edge,
 };
 
+// The stroke weight a line is drawn with.
+//
+// Weight is recorded for each direction a line leaves a cell in, and Unicode
+// has a glyph for every mix of light and heavy lines meeting in a cell, so
+// where a heavy line meets a light one each keeps its weight. ASCII has no
+// weights: a cell with a heavy line leaving it is `^`, the underline `rustc`
+// uses for a primary range, unless a line enters it from above or a light line
+// leaves it downward. Those cells keep their light glyph, so the junction or
+// corner stays visible.
+enum class LineWeight : int8_t {
+  Light,
+  Heavy,
+};
+
 // A grid of styled cells staged for rendering to a terminal.
 //
 // Coordinates are 0-based with (0, 0) at the top left, `x` counting terminal
@@ -247,14 +261,17 @@ class Buffer {
   // The block and the origin are checked as drawing checks them, so measuring
   // answers only for arguments drawing would accept.
   auto MeasureWrappedText(int x, int y, int margin, int max_width,
-                          llvm::StringRef text) const -> DrawEnd;
+                          llvm::StringRef text,
+                          bool backticks_are_quotes = false) const -> DrawEnd;
 
   // Returns the fewest columns `text` wraps into without overhanging them,
   // which is the width of its widest word since wrapping never breaks one.
+  // `backticks_are_quotes` is as for `DrawWrappedText`.
   //
   // Wrapping into fewer columns still draws everything; the excess overhangs.
   // So this is a layout preference rather than a minimum.
-  auto MeasureWrapWidth(llvm::StringRef text) const -> int;
+  auto MeasureWrapWidth(llvm::StringRef text,
+                        bool backticks_are_quotes = false) const -> int;
 
   // Draws `code_point` at (x, y), which must be a non-negative column and a row
   // inside `MaxRows`, widening the buffer and adding rows as needed to reach
@@ -286,16 +303,42 @@ class Buffer {
   // containing `-` or `+` is never redrawn as line art.
   //
   // A cell's style is whatever was drawn there last, so crossing lines of
-  // different styles do depend on order.
+  // different styles depend on order. Weight is kept per direction instead:
+  // each direction a line leaves a cell in has the weight of the line drawn
+  // there last, so crossing lines keep their own weights in either order, and
+  // a heavy line meeting a light one forms a mixed junction.
   auto DrawHorizontalLine(int x, int y, int length, const Style& style,
                           LineEnd start = LineEnd::Center,
-                          LineEnd end = LineEnd::Center) -> DrawEnd;
+                          LineEnd end = LineEnd::Center,
+                          LineWeight weight = LineWeight::Light) -> DrawEnd;
 
   // Draws a vertical line down `length` rows starting at (x, y), with the same
   // meaning for its ends. Returns the row after it, in the column it ran down.
   auto DrawVerticalLine(int x, int y, int length, const Style& style,
                         LineEnd start = LineEnd::Center,
-                        LineEnd end = LineEnd::Center) -> DrawEnd;
+                        LineEnd end = LineEnd::Center,
+                        LineWeight weight = LineWeight::Light) -> DrawEnd;
+
+  // Draws a light vertical line down `length` rows starting at (x, y), behind
+  // everything drawn before it. The line runs out through the bottom of its
+  // last cell.
+  //
+  // The line is drawn only into blank cells with no text directly left or
+  // right of them. Everything drawn earlier stays as it was, so where this
+  // line crosses another, the other line is unbroken and this one has a gap,
+  // and the two don't join. A line beside text could be mistaken for part of
+  // the text, which is why those cells are skipped too.
+  //
+  // With `start` at `LineEnd::Center`, the first cell is the exception: the
+  // line leaves it downward and joins what it holds, as `DrawVerticalLine`
+  // would. If a line already leaves that cell downward, the cell is left
+  // unchanged, so that line keeps its weight. With `start` at `LineEnd::Edge`,
+  // the line enters its first cell from above like the rest, which continues
+  // a line broken off above it.
+  //
+  // Returns the row after the line, in the column it ran down.
+  auto DrawCrossingVerticalLine(int x, int y, int length, const Style& style,
+                                LineEnd start = LineEnd::Center) -> DrawEnd;
 
   // Draws the outline of a box with its top-left corner at (x, y).
   //
@@ -359,6 +402,12 @@ class Buffer {
   // word, and one too long for a row of its own is moved down to one and then
   // overhangs it rather than being broken.
   //
+  // With `backticks_are_quotes`, backticks pair up from the left on each line
+  // and no break falls between a pair, so a quoted type with spaces in it
+  // wraps whole or overhangs, as a URL does. A backtick with no partner later
+  // on its line, or with a tab or carriage return before its partner, is an
+  // ordinary character.
+  //
   // Whitespace stops at the block's edge rather than running past it, so the
   // spaces between two words stay on the row the first of them ended and the
   // row the second wraps onto begins at the margin. Spaces the text opens with,
@@ -392,7 +441,8 @@ class Buffer {
   // or only those a previous wrapping introduced -- is a question about where
   // that text came from.
   auto DrawWrappedText(int x, int y, int margin, int max_width,
-                       llvm::StringRef text, const Style& style) -> DrawEnd;
+                       llvm::StringRef text, const Style& style,
+                       bool backticks_are_quotes = false) -> DrawEnd;
 
   // Renders the grid, appending the bytes that draw it to `out`.
   //
@@ -434,7 +484,7 @@ class Buffer {
 
   struct Cell {
     // The code point rendered here. For a cell with `lines` set, this is
-    // derived from those bits and the charset.
+    // derived from `lines`, `heavy_lines`, and the charset.
     char32_t code_point = ' ';
 
     Style style;
@@ -443,6 +493,10 @@ class Buffer {
     // or zero for a cell holding text.
     uint8_t lines = 0;
 
+    // The directions in `lines` that a heavy line leaves in. Always a subset
+    // of the directions set there.
+    uint8_t heavy_lines = 0;
+
     // Whether this cell is the right half of a double-width character, and so
     // renders nothing of its own.
     bool is_continuation = false;
@@ -501,7 +555,8 @@ class Buffer {
                 PlaceFn place) const -> DrawEnd;
   template <typename PlaceFn>
   auto WalkWrappedText(int x, int y, int margin, int max_width,
-                       llvm::StringRef text, PlaceFn place) const -> DrawEnd;
+                       llvm::StringRef text, bool backticks_are_quotes,
+                       PlaceFn place) const -> DrawEnd;
 
   // Adds rows until row `y` exists.
   auto EnsureRow(int y) -> void;
@@ -518,8 +573,14 @@ class Buffer {
   // Appends `code_point` to the marks rendered with the cell before column `x`.
   auto AttachCombiningMark(int x, int y, char32_t code_point) -> void;
 
-  // Adds `directions` to the lines through (x, y) and updates its glyph.
-  auto DrawLine(int x, int y, uint8_t directions, const Style& style) -> void;
+  // Adds `directions` to the lines through (x, y) at `weight`, replacing the
+  // weight those directions had, and updates its glyph.
+  auto DrawLine(int x, int y, uint8_t directions, LineWeight weight,
+                const Style& style) -> void;
+
+  // Returns whether the cell at (x, y) holds text. Cells outside what has been
+  // drawn hold nothing.
+  auto HoldsText(int x, int y) const -> bool;
 
   // Returns the last column in row `y` that renders anything under `mode`, or
   // -1 when the row renders nothing.

common/terminal/buffer_test.cpp

@@ -6,7 +6,6 @@
 
 #include <gtest/gtest.h>
 
-#include <limits>
 #include <optional>
 #include <string>
 
@@ -171,18 +170,222 @@ TEST(BufferTest, Tees) {
             "│ │ │\n"
             "╰─┴─╯\n");
 
-  // The same table in ASCII, where every tee keeps its through-stroke and only
-  // the crossing in the middle is a `+`.
+  // The same table in ASCII: only the crossing in the middle is a `+`, a tee
+  // where a line leaves sideways is `|`, and a tee where a line leaves upward
+  // or downward is the corner glyph for that direction.
   Buffer ascii(5, Charset::Ascii);
   ascii.DrawBox(0, 0, 5, 5, Style());
   ascii.DrawHorizontalLine(0, 2, 5, Style());
   ascii.DrawVerticalLine(2, 0, 5, Style());
   EXPECT_EQ(Render(ascii),
-            ".---.\n"
+            ".-.-.\n"
             "| | |\n"
             "|-+-|\n"
             "| | |\n"
-            "'---'\n");
+            "'-'-'\n");
+}
+
+TEST(BufferTest, HeavyLines) {
+  // A heavy line keeps the same geometry as a light one, drawn in the heavy
+  // glyphs, ends included.
+  Buffer heavy(4, Charset::Utf8);
+  heavy.DrawHorizontalLine(0, 0, 4, Style(), LineEnd::Center, LineEnd::Center,
+                           LineWeight::Heavy);
+  heavy.DrawVerticalLine(0, 1, 3, Style(), LineEnd::Center, LineEnd::Center,
+                         LineWeight::Heavy);
+  EXPECT_EQ(Render(heavy),
+            "╺━━╸\n"
+            "╻\n"
+            "┃\n"
+            "╹\n");
+
+  // Heavy lines join each other the way light ones do, into heavy junctions.
+  Buffer joined(5, Charset::Utf8);
+  joined.DrawHorizontalLine(0, 0, 5, Style(), LineEnd::Edge, LineEnd::Edge,
+                            LineWeight::Heavy);
+  joined.DrawVerticalLine(2, 0, 3, Style(), LineEnd::Center, LineEnd::Center,
+                          LineWeight::Heavy);
+  joined.DrawHorizontalLine(2, 2, 3, Style(), LineEnd::Center, LineEnd::Edge,
+                            LineWeight::Heavy);
+  EXPECT_EQ(Render(joined),
+            "━━┳━━\n"
+            "  ┃\n"
+            "  ┗━━\n");
+}
+
+TEST(BufferTest, HeavyAndLightLinesMix) {
+  // Weight is recorded per direction, so a heavy line crossing a light one
+  // draws a junction that is heavy left and right and light up and down.
+  Buffer cross(3, Charset::Utf8);
+  cross.DrawHorizontalLine(0, 1, 3, Style(), LineEnd::Edge, LineEnd::Edge,
+                           LineWeight::Heavy);
+  cross.DrawVerticalLine(1, 0, 3, Style(), LineEnd::Edge, LineEnd::Edge);
+  EXPECT_EQ(Render(cross),
+            " │\n"
+            "━┿━\n"
+            " │\n");
+
+  // A light line leaving a heavy one downward: each line keeps its own weight
+  // in the tee.
+  Buffer tee(4, Charset::Utf8);
+  tee.DrawHorizontalLine(0, 0, 4, Style(), LineEnd::Edge, LineEnd::Edge,
+                         LineWeight::Heavy);
+  tee.DrawVerticalLine(1, 0, 2, Style(), LineEnd::Center, LineEnd::Center);
+  EXPECT_EQ(Render(tee),
+            "━┯━━\n"
+            " ╵\n");
+
+  // A line's weight in a cell is that of whatever drew it last, as a cell's
+  // style is: a heavy line drawn over a light one leaves it heavy, and a light
+  // one drawn over part of a heavy one lightens the stretch it covers and
+  // nothing else.
+  Buffer overdrawn(3, Charset::Utf8);
+  overdrawn.DrawHorizontalLine(0, 0, 3, Style());
+  overdrawn.DrawHorizontalLine(0, 0, 3, Style(), LineEnd::Center,
+                               LineEnd::Center, LineWeight::Heavy);
+  EXPECT_EQ(Render(overdrawn), "╺━╸\n");
+
+  Buffer lightened(4, Charset::Utf8);
+  lightened.DrawHorizontalLine(0, 0, 4, Style(), LineEnd::Edge, LineEnd::Edge,
+                               LineWeight::Heavy);
+  lightened.DrawHorizontalLine(2, 0, 2, Style(), LineEnd::Edge, LineEnd::Edge);
+  EXPECT_EQ(Render(lightened), "━━──\n");
+}
+
+TEST(BufferTest, HeavyLinesInAscii) {
+  // ASCII has no weights, so a heavy horizontal line becomes `^`, including
+  // the tee where a heavy vertical line leaves it.
+  Buffer underline(8, Charset::Ascii);
+  underline.DrawHorizontalLine(0, 0, 7, Style(), LineEnd::Edge, LineEnd::Edge,
+                               LineWeight::Heavy);
+  underline.DrawVerticalLine(3, 0, 2, Style(), LineEnd::Center, LineEnd::Center,
+                             LineWeight::Heavy);
+  EXPECT_EQ(Render(underline),
+            "^^^^^^^\n"
+            "   |\n");
+
+  // So does the top cell of a heavy vertical line, which no line enters from
+  // above, so the line is `^` above `|`.
+  Buffer vertical(3, Charset::Ascii);
+  vertical.DrawVerticalLine(1, 0, 3, Style(), LineEnd::Center, LineEnd::Center,
+                            LineWeight::Heavy);
+  EXPECT_EQ(Render(vertical),
+            " ^\n"
+            " |\n"
+            " |\n");
+
+  // A light line leaving the heavy underline downward keeps the light tee's
+  // `.`, which `^` would hide, while a heavy line leaving it downward keeps the
+  // `^`.
+  Buffer tees(7, Charset::Ascii);
+  tees.DrawHorizontalLine(0, 0, 7, Style(), LineEnd::Edge, LineEnd::Edge,
+                          LineWeight::Heavy);
+  tees.DrawVerticalLine(2, 0, 2, Style(), LineEnd::Center, LineEnd::Center);
+  tees.DrawVerticalLine(5, 0, 2, Style(), LineEnd::Center, LineEnd::Center,
+                        LineWeight::Heavy);
+  EXPECT_EQ(Render(tees),
+            "^^.^^^^\n"
+            "  |  |\n");
+
+  // A cell a line enters from above keeps its light glyph, so the crossing is
+  // `+`, and the heavy vertical line is drawn as a light one.
+  Buffer crossed(3, Charset::Ascii);
+  crossed.DrawHorizontalLine(0, 1, 3, Style(), LineEnd::Edge, LineEnd::Edge,
+                             LineWeight::Heavy);
+  crossed.DrawVerticalLine(1, 0, 3, Style(), LineEnd::Edge, LineEnd::Edge,
+                           LineWeight::Heavy);
+  EXPECT_EQ(Render(crossed),
+            " |\n"
+            "^+^\n"
+            " |\n");
+}
+
+TEST(BufferTest, CrossingLinePassesBehindEarlierLines) {
+  // A crossing line joins the line its first cell holds. Below that, it has a
+  // gap where it crosses an earlier line, which stays unbroken, and the two
+  // don't join.
+  Buffer buffer(5, Charset::Utf8);
+  buffer.DrawHorizontalLine(0, 0, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  buffer.DrawHorizontalLine(0, 2, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  buffer.DrawCrossingVerticalLine(2, 0, 4, Style());
+  EXPECT_EQ(Render(buffer),
+            "──┬──\n"
+            "  │\n"
+            "─────\n"
+            "  │\n");
+
+  // The same crossing in ASCII, where the tee is the corner glyph for the line
+  // leaving downward and the crossed line is unbroken.
+  Buffer ascii(5, Charset::Ascii);
+  ascii.DrawHorizontalLine(0, 0, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  ascii.DrawHorizontalLine(0, 2, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  ascii.DrawCrossingVerticalLine(2, 0, 4, Style());
+  EXPECT_EQ(Render(ascii),
+            "--.--\n"
+            "  |\n"
+            "-----\n"
+            "  |\n");
+
+  // Weight plays no part in it: a heavy line drawn first stays whole the
+  // same way.
+  Buffer heavy(5, Charset::Utf8);
+  heavy.DrawHorizontalLine(0, 0, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  heavy.DrawHorizontalLine(0, 1, 5, Style(), LineEnd::Edge, LineEnd::Edge,
+                           LineWeight::Heavy);
+  heavy.DrawCrossingVerticalLine(2, 0, 4, Style());
+  EXPECT_EQ(Render(heavy),
+            "──┬──\n"
+            "━━━━━\n"
+            "  │\n"
+            "  │\n");
+}
+
+TEST(BufferTest, CrossingLineResumesFromAnEdge) {
+  // A crossing line starting at an edge enters its first cell from above, so
+  // it continues a line broken off above a stretch it skipped, and it still
+  // has a gap where it crosses an earlier line.
+  Buffer buffer(5, Charset::Utf8);
+  buffer.DrawHorizontalLine(0, 0, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  buffer.DrawText(0, 1, "words", Style());
+  buffer.DrawHorizontalLine(0, 3, 5, Style(), LineEnd::Edge, LineEnd::Edge);
+  buffer.DrawCrossingVerticalLine(2, 0, 1, Style());
+  buffer.DrawCrossingVerticalLine(2, 2, 3, Style(), LineEnd::Edge);
+  EXPECT_EQ(Render(buffer),
+            "──┬──\n"
+            "words\n"
+            "  │\n"
+            "─────\n"
+            "  │\n");
+}
+
+TEST(BufferTest, CrossingLineSharesAColumnWithoutRedrawing) {
+  // A crossing line leaves the cells of an earlier line in its column as they
+  // are, so that line keeps its weight.
+  Buffer buffer(3, Charset::Utf8);
+  buffer.DrawHorizontalLine(0, 0, 3, Style(), LineEnd::Edge, LineEnd::Edge);
+  buffer.DrawVerticalLine(1, 0, 3, Style(), LineEnd::Center, LineEnd::Center,
+                          LineWeight::Heavy);
+  buffer.DrawCrossingVerticalLine(1, 0, 3, Style());
+  EXPECT_EQ(Render(buffer),
+            "─┰─\n"
+            " ┃\n"
+            " ╹\n");
+}
+
+TEST(BufferTest, CrossingLineStaysClearOfText) {
+  // The line skips text, and blank cells directly beside text, where a line
+  // could be mistaken for part of a word.
+  Buffer buffer(6, Charset::Utf8);
+  buffer.DrawHorizontalLine(0, 0, 6, Style(), LineEnd::Edge, LineEnd::Edge);
+  buffer.DrawText(0, 1, "ab", Style());
+  buffer.DrawText(0, 3, "abc", Style());
+  buffer.DrawCrossingVerticalLine(2, 0, 5, Style());
+  EXPECT_EQ(Render(buffer),
+            "──┬───\n"
+            "ab\n"
+            "  │\n"
+            "abc\n"
+            "  │\n");
 }
 
 TEST(BufferTest, ALineBetweenOneCenterAndItselfIsAPoint) {
@@ -673,6 +876,69 @@ TEST(BufferTest, TabWidthComesFromCapabilities) {
   EXPECT_EQ(Render(clamped), "a b\n");
 }
 
+TEST(BufferTest, WrappedTextKeepsAQuotedRunWhole) {
+  // With backticks as quotes, the spaces between a pair of backticks are not
+  // breaks, so the quoted run wraps to the next row whole.
+  Buffer buffer(20, Charset::Utf8);
+  buffer.DrawWrappedText(0, 0, 0, 20, "type `{.a: i32}` is not", Style(),
+                         /*backticks_are_quotes=*/true);
+  EXPECT_EQ(Render(buffer),
+            "type `{.a: i32}` is\n"
+            "not\n");
+
+  Buffer split(14, Charset::Utf8);
+  split.DrawWrappedText(0, 0, 0, 14, "type `{.a: i32}` is not", Style(),
+                        /*backticks_are_quotes=*/true);
+  EXPECT_EQ(Render(split),
+            "type\n"
+            "`{.a: i32}` is\n"
+            "not\n");
+
+  // Otherwise, the same text breaks inside the quotes like anywhere else.
+  Buffer unquoted(14, Charset::Utf8);
+  unquoted.DrawWrappedText(0, 0, 0, 14, "type `{.a: i32}` is not", Style());
+  EXPECT_EQ(Render(unquoted),
+            "type `{.a:\n"
+            "i32}` is not\n");
+
+  // A backtick with no partner on its line is an ordinary character, so the
+  // text after it still breaks at its spaces.
+  Buffer stray(14, Charset::Utf8);
+  stray.DrawWrappedText(0, 0, 0, 14, "a `stray quote and more words", Style(),
+                        /*backticks_are_quotes=*/true);
+  EXPECT_EQ(Render(stray),
+            "a `stray quote\n"
+            "and more words\n");
+
+  // Nor does a run reach across a newline the caller wrote.
+  Buffer lines(14, Charset::Utf8);
+  lines.DrawWrappedText(0, 0, 0, 14, "a `b\nc d` e f g h", Style(),
+                        /*backticks_are_quotes=*/true);
+  EXPECT_EQ(Render(lines),
+            "a `b\n"
+            "c d` e f g h\n");
+
+  // A word can't hold a tab or carriage return, so a quoted run doesn't reach
+  // across one either.
+  Buffer tabbed(14, Charset::Utf8);
+  tabbed.DrawWrappedText(0, 0, 0, 14, "a `b\tc` d", Style(),
+                         /*backticks_are_quotes=*/true);
+  EXPECT_EQ(Render(tabbed), "a `b    c` d\n");
+  EXPECT_EQ(buffer.MeasureWrapWidth("`b\tc`", /*backticks_are_quotes=*/true),
+            2);
+  EXPECT_EQ(buffer.MeasureWrapWidth("`b\rc`", /*backticks_are_quotes=*/true),
+            2);
+
+  // Measuring agrees: the quoted run is the widest word.
+  EXPECT_EQ(buffer.MeasureWrapWidth("type `{.a: i32}` is not",
+                                    /*backticks_are_quotes=*/true),
+            11);
+  EXPECT_EQ(buffer.MeasureWrapWidth("type `{.a: i32}` is not"), 5);
+  EXPECT_EQ(buffer.MeasureWrappedText(0, 0, 0, 14, "type `{.a: i32}` is not",
+                                      /*backticks_are_quotes=*/true),
+            DrawEnd(3, 2));
+}
+
 TEST(BufferTest, WrappedTextKeepsTheBreaksItIsGiven) {
   // Wrapping only adds breaks. Text that arrives wrapped to some other width
   // keeps that wrapping rather than being reflowed into this one, even where

core/prelude/destroy.carbon

@@ -3,33 +3,37 @@
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 package Core library "prelude/destroy";
 
-// TODO: uncomment when `require impls` adds a witness table entry.
-// interface SubobjectDestroy {
-//   final fn Op(ref self) = "subobject.destroy";
-// }
+import library "prelude/operators/as";
+import library "prelude/operators/bitwise";
 
-interface Destroy {
+// TODO: uncomment when `require impls` adds a witness table entry.
+interface SubobjectDestroy {
+  fn Op(ref self);
+}
+
+// TODO: rename to `Destroy` when `require impls` adds a witness table entry.
+interface DestroyImpl {
   // TODO: uncomment when `require impls` adds a witness table entry.
   // require impls SubobjectDestroy;
 
   // TODO: change `Self` to `partial Self`.
   fn Op(ref self);
+}
 
-  // TODO: remove when `require impls` adds a witness table entry.
-  final fn SubobjectDestroy(unused ref self) {
-    // This function body is ignored by the toolchain. We can't use
-    // a built-in because the compiler treats it as an error. We
-    // should address this at some point, but it's not a high
-    // priority right now.
-  }
+impl forall [T: SubobjectDestroy] T as DestroyImpl {
+  fn Op(ref self) = "no_op";
+}
 
-  final fn SelfDestruct(ref self) {
-    self.Op();
-    self.SubobjectDestroy();
-  }
+constraint Destroy {
+  extend require impls SubobjectDestroy & DestroyImpl;
 }
 
 // TODO: uncomment when `require impls` adds a witness table entry.
 // impl forall [T: SubobjectDestroy] partial T as Destroy {
 //   alias Op = T.Op;
 // }
+
+fn SelfDestruct[T: Destroy](ref t: T) {
+    t.(DestroyImpl.Op)();
+    t.(SubobjectDestroy.Op)();
+}

core/prelude/operators/index.carbon

@@ -4,7 +4,25 @@
 
 package Core library "prelude/operators/index";
 
+export import library "prelude/copy";
+
 interface IndexWith(SubscriptType: type) {
   let ElementType: type;
   fn At(self, subscript: SubscriptType) -> ElementType;
 }
+
+interface IndirectIndexWith(SubscriptType: type) {
+  let ElementType: type;
+  fn Ref(self, subscript: SubscriptType) -> ref ElementType;
+}
+
+final impl forall [
+        SubscriptType: type,
+        T: IndirectIndexWith(SubscriptType)
+        where .ElementType impls Copy
+    ] T as IndexWith(SubscriptType)
+        where .ElementType = T.ElementType {
+  fn At(self, subscript: SubscriptType) -> ElementType {
+    return self.Ref(subscript);
+  }
+}

docs/design/classes.md

@@ -121,8 +121,8 @@ constant value.
 The use cases for classes include both cases motivated by C++ interop, and cases
 that we expect to be included in idiomatic Carbon-only code.
 
-**This design currently only attempts to address the "data classes" and
-"encapsulated types" use cases.** Addressing the "interface as base class",
+**This design currently only attempts to address the "data classes",
+"encapsulated types", and "interface as base class" use cases.** Addressing the
 "interop with C++ multiple inheritance" and "mixin" use cases is future work.
 
 ### Data classes

docs/design/expressions/member_access.md

@@ -1176,8 +1176,8 @@ Assert((r as __Binding_C_F).(Call(()).Op)() == 9);
 
 How does this arise?
 
-1.  First the simple member access evaluates `typeof(a).F`, which is `C.F` (`__C_F`
-    with type `__TypeOf_C_F`).
+1.  First the simple member access evaluates `typeof(a).F`, which is `C.F`
+    (`__C_F` with type `__TypeOf_C_F`).
 2.  It then looks at the expression to the left of the `.`:
     -   If it is a reference expression, the "member binding to reference"
         (`BindToRef`) operator is applied.

docs/design/generics/details.md

@@ -3337,15 +3337,14 @@ the expression `G.E.N.E.N` is one equality away from `G.N.E.N` and so it is
 allowed. This is true even though `G.N.E.N` isn't the type expression
 immediately prior to `G.E.N.E.N`.
 
-An `observe` declaration inside an `interface` definition may only refer to
-names dependent on:
+An `observe` declaration inside an `interface` definition refers to related
+values, which are type expressions dependent on at least one of:
 
--   `.Self`,
--   generic parameters, and
--   associated constants that are part of the enclosing `interface`.
+-   a generic parameter, including `Self`, or
+-   an associated constant that is part of the enclosing `interface`.
 
-Equivalence chains may include at most one unrelated value.
-`observe .. == .. impls` declarations may include unrelated values that
+Inside an `interface`, equivalence chains may include at most one unrelated
+value. `observe .. == .. impls` declarations may include unrelated values that
 immediately satisfy the `impls` constraint.
 
 The `observe` declaration in the example below is allowed because `X.T` and

docs/design/pattern_matching.md

@@ -414,8 +414,9 @@ matches if all of these sub-matches succeed.
 
 A struct can be matched with a struct pattern.
 
--   _struct-pattern_ ::= `{` [_field-pattern_ [`,` _field-pattern_ ]\* ] `}`
--   _struct-pattern_ ::= `{` [_field-pattern_ `,`]+ `_` `}`
+-   _struct-pattern_ ::= `{` [_field-pattern_ [`,` _field-pattern_ ]\* `,`? ]
+    `}`
+-   _struct-pattern_ ::= `{` [_field-pattern_ `,`]+ `_` `,`? `}`
 -   _field-pattern_ ::= _designator_ `=` _pattern_
 -   _field-pattern_ ::= _binding-pattern_
 -   _pattern_ ::= _struct-pattern_

docs/design/variadics.md

@@ -23,6 +23,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 -   [Typechecking](#typechecking)
     -   [Tuples, packs, segments, and shapes](#tuples-packs-segments-and-shapes)
     -   [Iterative typechecking of pack expansions](#iterative-typechecking-of-pack-expansions)
+        -   [Type versus execution semantics](#type-versus-execution-semantics)
     -   [Typechecking patterns](#typechecking-patterns)
     -   [Typechecking pattern matches](#typechecking-pattern-matches)
 -   [Appendix: Type system formalism](#appendix-type-system-formalism)
@@ -491,6 +492,41 @@ expansion itself is relatively straightforward:
 
 > **TODO:** Discuss typechecking `...expand`.
 
+#### Type versus execution semantics
+
+Within a pack expansion, there's a subtle discrepancy between the type of an
+expression and its execution semantics. As discussed in the previous section, an
+expression within a pack expansion has a pack type rather than a scalar type, as
+if it were evaluated once and produced a pack value. However, as discussed
+[earlier](#expressions-and-statements), the execution of a pack expansion is
+modeled as a procedural loop that successively evaluates the expansion body for
+each element of the input pack, and expressions have scalar values within each
+iteration. In effect, the type system treats pack expansions as if they were
+SIMD code, with expressions operating on "vectors" of data in parallel, even
+though they actually execute iteratively on a series of scalar values.
+
+As a result, it can be misleading to use the type system as a guide to the
+execution semantics, because the two models disagree in some corner cases.
+For example, consider:
+
+```carbon
+fn F(i: i32) -> i64;
+
+fn G(... (each c: bool, each x: i32)) {
+  ...if each c then F(each x) else 0 as i64;
+}
+```
+
+In that code, `F(each x)` has type `«i64; ‖each x‖»`, in other words, a pack
+with the same arity as `each x`, whose elements are all `i64`. That may give the
+impression that the expression `F(each x)` evaluates to a pack of `i64` values
+with arity `‖each x‖`, and so `F` is called `‖each x‖` times, but that's not the
+case. In fact, `F(each x)` is evaluated once for each `true` value in `each c`,
+and each evaluation produces a scalar `i64` value. The results of those
+evaluations are never actually assembled into a pack, and in fact they
+_couldn't_ be, because the arity of such a pack isn't known until run time, and
+typically wouldn't match the arity of the enclosing pack expansion.
+
 ### Typechecking patterns
 
 A pack expansion pattern has _fixed arity_ if it contains at least one usage of

github_tools/requirements.txt

@@ -4,9 +4,9 @@
 #
 #    bazel run //:requirements.update
 #
-certifi==2024.7.4 \
-    --hash=sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b \
-    --hash=sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90
+certifi==2026.10.19 \
+    --hash=sha256:035c1e85572b141bf1c3a3eff0e33a8eb74917522f2559075ff8d2fea780a1ae \
+    --hash=sha256:42d35340cc5b9ba78e818ddbff0c5b1c2b779de448a340fd1b989f5fa8670e1b
     # via requests
 charset-normalizer==3.3.2 \
     --hash=sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027 \

proposals/p000285-if-else.md

@@ -59,11 +59,11 @@ A few syntaxes that are likely to influence `if`/`else` are:
 
     ```python
     if x:
-      print("x is true");
+        print("x is true")
     elif y:
-      print("y is true");
+        print("y is true")
     else:
-      print("Neither was true");
+        print("Neither was true")
     ```
 
 -   Swift

proposals/p000340-while-loops.md

@@ -53,17 +53,17 @@ similar in many languages, even if details may change.
 
     ```python
     while x:
-      DoSomething()
+        DoSomething()
 
     while True:
-      DoSomethingElse()
-      if not y:
-        break
+        DoSomethingElse()
+        if not y:
+            break
 
     while z:
-      print("z is true")
+        print("z is true")
     else:
-      print("z was never true")
+        print("z was never true")
     ```
 
 -   Swift: Swift uses `repeat` instead of `do`.

proposals/p000438-functions.md

@@ -146,10 +146,11 @@ For exhaustive function examples:
 
     ```python
     def sum(a, b):
-      return a + b
+        return a + b
+
 
     def sum(a: int, b: int) -> int:
-      return a + b
+        return a + b
     ```
 
 -   R

proposals/p001885-for-statement-and-user-types.md

@@ -116,14 +116,15 @@ as the next value to the `for` loop. Iteration ends when the function returns.
 
 ```python
 def generate_fibonacci():
-  yield 0;
-  n1, n2 = 0, 1;
-  while True:
-    n1, n2 = n2, n1+n2;
-    yield n1;
+    yield 0
+    n1, n2 = 0, 1
+    while True:
+        n1, n2 = n2, n1 + n2
+        yield n1
+
 
 for n in generate():
-  print(n);
+    print(n)
 ```
 
 #### Rust

proposals/p002665-semicolons-terminate-statements.md

@@ -83,6 +83,8 @@ In Python, a line is a
 parentheses are an idiomatic way to create multi-line statements. Semicolons may
 be used to explicitly separate statements. For example:
 
+<!-- fmt:off -->
+
 ```python
 value = (
   "text"
@@ -90,6 +92,8 @@ value = (
 a = 1; b = 2; c = 3
 ```
 
+<!-- fmt:on -->
+
 Swift allows some statements to wrap lines, although multiple statements on the
 same line (`x = 1 x = 1`) require a semicolon. The detailed rules aren't
 documented so it's difficult to assess other than that Swift developers are
@@ -101,6 +105,8 @@ documents that line-breaks are treated as whitespace. However, there are
 observable ways the behavior can lead to small mistakes; these may may often be
 caught by the compiler, but will sometimes be missed. For example:
 
+<!-- fmt:off -->
+
 ```swift
 // One statement in Swift, but two in Python and Kotlin.
 var x = 1
@@ -116,12 +122,16 @@ Make()
 ()
 ```
 
+<!-- fmt:on -->
+
 Kotlin permits a newline to be used to terminate statements instead of a
 semicolon. Kotlin's grammar
 [explicitly enumerates](https://kotlinlang.org/spec/syntax-and-grammar.html) all
 the places where newlines can appear (see mentions of `NL` in the grammar), and
 doesn't allow newlines in places where they would introduce ambiguity.
 
+<!-- fmt:off -->
+
 ```kotlin
 // This is unambiguously parsed as two statements, because
 // a newline is not permitted before a `+` operator.
@@ -129,6 +139,8 @@ var x = 1
 + 1
 ```
 
+<!-- fmt:on -->
+
 In JavaScript and TypeScript, semicolons are part of the formal syntax, and
 ECMAScript provides
 [Automatic Semicolon Insertion (ASI)](https://tc39.es/ecma262/#sec-automatic-semicolon-insertion).

scripts/lldbinit.py

@@ -64,6 +64,11 @@ Args:
        can come with a `0x` prefix, allowing easier copy-paste from raw printed
        hex values (such as via the `p/x` lldb command).
 
+Aliases:
+  dc [<EXPR>|-- <EXPR>|<TYPE><ID>|<TYPE> <ID>]
+    - Shorthand for `dump context` which is the common case where there is a
+      `context` var or method in scope.
+
 Example usage:
   # Dumps the `inst_id` local variable, with a `context` local variable.
   dump context inst_id
@@ -92,7 +97,7 @@ Example usage:
         "function": "SemIR::MakeFunctionId",
         "generated_function": "SemIR::MakeGeneratedFunctionId",
         "generic": "SemIR::MakeGenericId",
-        "identified_facet_type": "SemIR::MakeIdentifiedFacetTypeId",
+        "identified_facet_type": "Check::MakeIdentifiedFacetTypeId",
         "impl": "SemIR::MakeImplId",
         "inst": "SemIR::MakeInstId",
         "inst_block": "SemIR::MakeInstBlockId",
@@ -204,6 +209,7 @@ def format_carbon_id(
 
 def __lldb_init_module(debugger: Any, internal_dict: Any) -> None:
     RunCommand("command script add -f lldbinit.cmd_dump dump")
+    RunCommand("command alias dc dump context")
     RunCommand(
         "type summary add --python-function lldbinit.format_carbon_id"
         + " --recognizer-function lldbinit.is_carbon_id"

testing/base/BUILD

@@ -37,6 +37,7 @@ cc_library(
     testonly = 1,
     srcs = ["benchmark_main.cpp"],
     deps = [
+        ":benchmark_helpers",
         ":global_exe_path",
         "//common:init_llvm",
         "@abseil-cpp//absl/flags:parse",
@@ -48,7 +49,9 @@ cc_library(
 cc_library(
     name = "benchmark_helpers",
     testonly = 1,
+    srcs = ["benchmark_helpers.cpp"],
     hdrs = ["benchmark_helpers.h"],
+    deps = ["//common:check"],
 )
 
 cc_library(

testing/base/benchmark_helpers.cpp

@@ -0,0 +1,27 @@
+// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+// Exceptions. See /LICENSE for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#include "testing/base/benchmark_helpers.h"
+
+#include <optional>
+
+#include "common/check.h"
+
+static constinit std::optional<bool> benchmark_dry_run = {};
+
+namespace Carbon::Testing {
+
+auto IsBenchmarkDryRun() -> bool {
+  CARBON_CHECK(benchmark_dry_run,
+               "queried `IsBenchmarkDryRun` before it was set");
+  return *benchmark_dry_run;
+}
+
+auto SetBenchmarkDryRun(bool is_dry_run) -> void {
+  CARBON_CHECK(!benchmark_dry_run,
+               "called `SetBenchmarkDryRun` more than once");
+  benchmark_dry_run = is_dry_run;
+}
+
+}  // namespace Carbon::Testing

testing/base/benchmark_helpers.h

@@ -65,6 +65,20 @@ template <typename T>
   DoNotOptimize(container.data());
 }
 
+// Returns true if the benchmark binary was invoked in dry-run mode
+// (`--benchmark_dry_run` or `BENCHMARK_DRY_RUN`).
+//
+// Google Benchmark's `--benchmark_dry_run` caps `state.max_iterations` to `1`,
+// so benchmarks that use `state.KeepRunningBatch(n)` or multi-input setup still
+// execute a full batch of `n` unless they also clamp `n` on dry runs.
+auto IsBenchmarkDryRun() -> bool;
+
+// Sets whether the benchmark binary is running in dry-run mode.
+//
+// This function must only be called once for an execution, and before any
+// callers to `IsBenchmarkDryRun`. Typically, it is called from within `main`.
+auto SetBenchmarkDryRun(bool is_dry_run) -> void;
+
 }  // namespace Carbon::Testing
 
 #endif  // CARBON_TESTING_BASE_BENCHMARK_HELPERS_H_

testing/base/benchmark_main.cpp

@@ -4,15 +4,55 @@
 
 #include <benchmark/benchmark.h>
 
+#include <cstdlib>
+
 #include "absl/flags/parse.h"
 #include "common/init_llvm.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringRef.h"
+#include "testing/base/benchmark_helpers.h"
 #include "testing/base/global_exe_path.h"
 
+// Returns true if `value` is truthy under Google Benchmark's boolean flag
+// rules: anything that isn't clearly _false_ is treated as truthy.
+static auto IsTruthyBenchmarkFlagValue(llvm::StringRef value) -> bool {
+  if (value.empty()) {
+    return true;
+  }
+  if (value.size() == 1) {
+    char c = value[0];
+    return llvm::isAlnum(c) && !llvm::StringRef("0fFnN").contains(c);
+  }
+  return !value.equals_insensitive("false") &&
+         !value.equals_insensitive("no") && !value.equals_insensitive("off");
+}
+
+// Detects `--benchmark_dry_run` (and `BENCHMARK_DRY_RUN` in the environment)
+// using the same rules as `benchmark::Initialize`, which consumes the flag
+// without exposing its value through Google Benchmark's public API.
+static auto DetectBenchmarkDryRun(llvm::ArrayRef<char*> args) -> bool {
+  bool dry_run = false;
+  if (const char* env = std::getenv("BENCHMARK_DRY_RUN")) {
+    dry_run = IsTruthyBenchmarkFlagValue(env);
+  }
+  for (llvm::StringRef arg : args) {
+    if (arg == "--benchmark_dry_run") {
+      dry_run = true;
+    } else if (arg.consume_front("--benchmark_dry_run=")) {
+      dry_run = IsTruthyBenchmarkFlagValue(arg);
+    }
+  }
+  return dry_run;
+}
+
 auto main(int orig_argc, char** orig_argv) -> int {
   // Do LLVM's initialization first, this will also transform UTF-16 to UTF-8.
   Carbon::InitLLVM init_llvm(orig_argc, orig_argv);
 
   Carbon::Testing::SetExePath(orig_argv[0]);
+  Carbon::Testing::SetBenchmarkDryRun(
+      DetectBenchmarkDryRun(llvm::ArrayRef(orig_argv, orig_argc).drop_front()));
 
   // Inject a flag to override the defaults for benchmarks. This can still be
   // disabled by user arguments.

toolchain/base/BUILD

@@ -72,7 +72,9 @@ cc_library(
         "@llvm-project//clang:basic",
         "@llvm-project//clang:driver",
         "@llvm-project//clang:frontend",
+        "@llvm-project//llvm:MC",
         "@llvm-project//llvm:Support",
+        "@llvm-project//llvm:TargetParser",
     ],
 )
 

toolchain/base/canonical_value_store.h

@@ -17,7 +17,9 @@ namespace Carbon {
 // A wrapper for accumulating immutable values with deduplication, providing IDs
 // to later retrieve the value.
 //
-// `ValueT` represents the type being stored.
+// `ValueT` represents the type being stored. Be careful storing non-owned
+// values (e.g. `llvm::StringRef`), as the store will not prevent the
+// underlying data from being freed.
 //
 // `KeyT` can optionally be different from `ValueT`, and if so is used for the
 // argument to `Lookup`. In this case, `ValueT` must provide a `GetAsKey` member

toolchain/base/clang_invocation.cpp

@@ -7,14 +7,21 @@
 #include <filesystem>
 #include <string>
 
+#include "clang/Basic/DiagnosticDriver.h"
+#include "clang/Basic/TargetInfo.h"
 #include "clang/Driver/CreateInvocationFromArgs.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/Utils.h"
 #include "common/string_helpers.h"
 #include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/MC/MCSubtargetInfo.h"
+#include "llvm/MC/TargetRegistry.h"
 #include "llvm/Support/FormatVariadic.h"
+#include "llvm/TargetParser/SubtargetFeature.h"
+#include "llvm/TargetParser/Triple.h"
 
 namespace Carbon {
 
@@ -50,17 +57,51 @@ auto ClangDriverDiagnosticConsumer::HandleDiagnostic(
   }
 }
 
+// Validates `target_opts` (such as `-march`, `-mcpu`, `-mtune`, and
+// `-target-feature`) and canonicalizes `target_opts.Features` via
+// `clang::TargetInfo::CreateTargetInfo`. Returns `false` if any target option
+// or feature is invalid.
+static auto ValidateTargetOptions(clang::DiagnosticsEngine& driver_diags,
+                                  clang::TargetOptions& target_opts) -> bool {
+  llvm::IntrusiveRefCntPtr<clang::TargetInfo> target_info(
+      clang::TargetInfo::CreateTargetInfo(driver_diags, target_opts));
+
+  // `TargetInfo::CreateTargetInfo` expands Clang-level feature aliases into
+  // `TargetOptions::Features`, but passes unknown `+feature`/`-feature`
+  // entries through to the backend where `MCSubtargetInfo` only prints a raw
+  // stderr warning. Validate that every canonicalized backend feature is
+  // recognized by the target's `MCSubtargetInfo` when the target is registered.
+  std::string target_error;
+  llvm::Triple triple(target_opts.Triple);
+  if (const llvm::Target* target =
+          llvm::TargetRegistry::lookupTarget(triple, target_error)) {
+    std::unique_ptr<llvm::MCSubtargetInfo> subtarget_info(
+        target->createMCSubtargetInfo(triple, /*CPU=*/"", /*Features=*/""));
+    if (subtarget_info) {
+      llvm::ArrayRef<llvm::SubtargetFeatureKV> valid_features =
+          subtarget_info->getAllProcessorFeatures();
+      const auto& features =
+          target_info ? target_opts.Features : target_opts.FeaturesAsWritten;
+      for (llvm::StringRef feature : features) {
+        llvm::StringRef name = llvm::SubtargetFeatures::StripFlag(feature);
+        const auto* it = llvm::lower_bound(valid_features, name);
+        if (it == valid_features.end() || llvm::StringRef(it->key()) != name) {
+          driver_diags.Report(clang::diag::err_drv_unsupported_option_argument)
+              << "-target-feature" << feature;
+        }
+      }
+    }
+  }
+
+  return target_info && !driver_diags.hasErrorOccurred();
+}
+
 auto BuildClangInvocation(Diagnostics::Consumer& consumer,
                           llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> fs,
                           const InstallPaths& install_paths,
                           llvm::StringRef target_str,
                           llvm::ArrayRef<llvm::StringRef> extra_args)
     -> std::unique_ptr<clang::CompilerInvocation> {
-  Diagnostics::ErrorTrackingConsumer error_tracker(consumer);
-  Diagnostics::NoLocEmitter emitter(&error_tracker);
-
-  ClangDriverDiagnosticConsumer diagnostics_consumer(&emitter);
-
   llvm::SmallVector<std::string> args;
   args.push_back("--start-no-unused-arguments");
   AppendDefaultClangArgs(install_paths, target_str, args);
@@ -83,6 +124,8 @@ auto BuildClangInvocation(Diagnostics::Consumer& consumer,
 
   // Build a diagnostics engine. Note that we don't have any diagnostic options
   // yet; they're produced by running the driver.
+  Diagnostics::NoLocEmitter emitter(&consumer);
+  ClangDriverDiagnosticConsumer diagnostics_consumer(&emitter);
   clang::DiagnosticOptions driver_diag_opts;
   llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine> driver_diags(
       clang::CompilerInstance::createDiagnostics(*fs, driver_diag_opts,
@@ -91,23 +134,22 @@ auto BuildClangInvocation(Diagnostics::Consumer& consumer,
 
   // Ask the driver to process the arguments and build a corresponding clang
   // frontend invocation.
-  auto invocation =
+  std::unique_ptr<clang::CompilerInvocation> invocation =
       clang::createInvocation(cstr_args, {.Diags = driver_diags, .VFS = fs});
 
   // If Clang produced an error, throw away its invocation.
-  if (error_tracker.seen_error()) {
+  if (!invocation || driver_diags->hasErrorOccurred() ||
+      !ValidateTargetOptions(*driver_diags, invocation->getTargetOpts())) {
     return nullptr;
   }
 
-  if (invocation) {
-    // Track submodule visibility in the preprocessor and Sema.
-    invocation->getLangOpts().Modules = true;
-    invocation->getLangOpts().ModulesLocalVisibility = true;
+  // Track submodule visibility in the preprocessor and Sema.
+  invocation->getLangOpts().Modules = true;
+  invocation->getLangOpts().ModulesLocalVisibility = true;
 
-    // Do not emit Clang's name and version as the creator of the output file.
-    invocation->getCodeGenOpts().EmitVersionIdentMetadata = false;
-    invocation->getCodeGenOpts().DiscardValueNames = false;
-  }
+  // Do not emit Clang's name and version as the creator of the output file.
+  invocation->getCodeGenOpts().EmitVersionIdentMetadata = false;
+  invocation->getCodeGenOpts().DiscardValueNames = false;
 
   return invocation;
 }

toolchain/base/clang_invocation.h

@@ -5,6 +5,7 @@
 #ifndef CARBON_TOOLCHAIN_BASE_CLANG_INVOCATION_H_
 #define CARBON_TOOLCHAIN_BASE_CLANG_INVOCATION_H_
 
+#include <memory>
 #include <string>
 
 #include "clang/Basic/Diagnostic.h"
@@ -37,8 +38,10 @@ class ClangDriverDiagnosticConsumer : public clang::DiagnosticConsumer {
 };
 
 // Builds and returns a clang `CompilerInvocation` to use when building code for
-// interop, from a list of clang driver arguments. Emits diagnostics to
-// `consumer` if the arguments are invalid.
+// interop, from a list of clang driver arguments. Validates the target options
+// and canonicalizes `TargetOptions::Features` via
+// `clang::TargetInfo::CreateTargetInfo`. Emits diagnostics to `consumer` if the
+// arguments or target options are invalid.
 auto BuildClangInvocation(Diagnostics::Consumer& consumer,
                           llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> fs,
                           const InstallPaths& install_paths,

toolchain/base/id_tag.h

@@ -68,28 +68,40 @@ struct IdTag {
 
   auto Apply(int32_t index) const -> IdT {
     CARBON_DCHECK(index >= 0, "{0}", index);
-    if (index < initial_reserved_ids_) {
+    if constexpr (IdTagIsUntagged<IdTag>) {
+      // Untagged ids are the index itself, so return directly with no tag
+      // computation.
       return IdT(index);
+    } else {
+      if (index < initial_reserved_ids_) {
+        return IdT(index);
+      }
+      // TODO: Assert that tag_ doesn't have the second highest bit set.
+      auto tagged_index = index ^ tag_;
+      CARBON_DCHECK(tagged_index >= 0, "{0}", tagged_index);
+      return IdT(tagged_index);
     }
-    // TODO: Assert that tag_ doesn't have the second highest bit set.
-    auto tagged_index = index ^ tag_;
-    CARBON_DCHECK(tagged_index >= 0, "{0}", tagged_index);
-    return IdT(tagged_index);
   }
 
   auto Remove(IdT id) const -> int32_t {
     CARBON_DCHECK(id.index >= 0, "{0}", id);
-    if (!HasTag(id.index)) {
-      CARBON_DCHECK(id.index < initial_reserved_ids_,
-                    "This untagged index is outside the initial reserved ids "
-                    "and should have been tagged.");
+    if constexpr (IdTagIsUntagged<IdTag>) {
+      // Untagged ids are the index itself, so return directly with no tag
+      // computation.
       return id.index;
+    } else {
+      if (!HasTag(id.index)) {
+        CARBON_DCHECK(id.index < initial_reserved_ids_,
+                      "This untagged index is outside the initial reserved ids "
+                      "and should have been tagged.");
+        return id.index;
+      }
+      auto untagged_index = id.index ^ tag_;
+      CARBON_DCHECK(untagged_index >= initial_reserved_ids_,
+                    "When removing tagging bits, found an index that "
+                    "shouldn't've been tagged in the first place.");
+      return untagged_index;
     }
-    auto untagged_index = id.index ^ tag_;
-    CARBON_DCHECK(untagged_index >= initial_reserved_ids_,
-                  "When removing tagging bits, found an index that "
-                  "shouldn't've been tagged in the first place.");
-    return untagged_index;
   }
 
   // Gets the value unique to this IdTag instance that is added to indices in

toolchain/base/mem_usage.h

@@ -33,9 +33,9 @@ namespace Carbon {
 //                   data_.reserved_bytes());
 //     // Common library types like `Map` and `llvm::SmallVector` have
 //     // type-specific support.
-//     mem_usage.Add(MemUsage::Concat(label, "array_"), array_);
+//     mem_usage.Add(MemUsage::ConcatLabel(label, "array_"), array_);
 //     // Implementing `CollectMemUsage` allows use with the same interface.
-//     mem_usage.Collect(MemUsage::Concat(label, "obj_"), obj_);
+//     mem_usage.Collect(MemUsage::ConcatLabel(label, "obj_"), obj_);
 //   }
 class MemUsage {
  public:
@@ -104,6 +104,11 @@ class MemUsage {
     Add(std::move(label), array.size_in_bytes(), array.capacity_in_bytes());
   }
 
+  // Adds memory usage of a string's data.
+  auto Collect(std::string label, const std::string& string) -> void {
+    Add(std::move(label), string.size(), string.capacity());
+  }
+
   // Adds memory usage for an object that provides `CollectMemUsage`.
   //
   // The expected signature of `CollectMemUsage` is above, in MemUsage class

toolchain/base/shared_value_stores.h

@@ -25,6 +25,9 @@ class SharedValueStores : public Yaml::Printable<SharedValueStores> {
   using IntStore = IntStore;
   using RealStore = ValueStore<RealId, Real>;
   using FloatStore = CanonicalValueStore<FloatId, llvm::APFloat>;
+  // Note that this stores string references. The underlying string data must
+  // be stored somewhere more durable (e.g. a string literal or
+  // `clang::ASTContext::Idents`).
   using IdentifierStore = CanonicalValueStore<IdentifierId, llvm::StringRef>;
   using StringLiteralStore =
       CanonicalValueStore<StringLiteralValueId, llvm::StringRef>;

toolchain/base/value_store.h

@@ -15,6 +15,7 @@
 
 #include "common/check.h"
 #include "common/ostream.h"
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/Sequence.h"
 #include "llvm/ADT/SmallVector.h"
@@ -263,8 +264,10 @@ class ValueStore
 
     CARBON_DCHECK(static_cast<size_t>(chunk_index) <= chunks_.size(),
                   "{0} <= {1}", chunk_index, chunks_.size());
-    if (static_cast<size_t>(chunk_index) == chunks_.size()) {
-      chunks_.emplace_back();
+    if (LLVM_UNLIKELY(static_cast<size_t>(chunk_index) == chunks_.size())) {
+      // Sink all the growth and allocation logic into an out-of-line helper as
+      // this is inherently a cold-path.
+      AllocateNextChunk();
     }
 
     CARBON_DCHECK(pos == chunks_[chunk_index].size());
@@ -349,6 +352,13 @@ class ValueStore
     return EnumerateRange(*this);
   }
 
+  // Returns the chunks of values in order. Each chunk is a contiguous range of
+  // values, so bulk operations over all values can iterate over contiguous
+  // memory rather than looking up each value by its ID.
+  auto chunks() const [[clang::lifetimebound]] -> auto {
+    return llvm::ArrayRef<Chunk>(chunks_);
+  }
+
   auto GetIdTag() const -> IdTagType { return tag_; }
   auto GetRawIndex(IdT id) const -> int32_t {
     CARBON_DCHECK(id.index >= 0, "{0}", id.index);
@@ -481,7 +491,10 @@ class ValueStore
         -> void
       requires(std::is_copy_constructible_v<ValueT>);
 
+    auto data() const -> const ValueType* { return buf_; }
     auto size() const -> int32_t { return num_; }
+    auto begin() const -> const ValueType* { return buf_; }
+    auto end() const -> const ValueType* { return buf_ + num_; }
 
    private:
     // Verify using an `int32_t` for `num_` is sound.
@@ -520,6 +533,10 @@ class ValueStore
     return RawIndexToChunkIndices(GetRawIndex(id));
   }
 
+  [[clang::noinline, clang::preserve_most]] auto AllocateNextChunk() -> void {
+    chunks_.emplace_back();
+  }
+
   // Number of elements added to the store. The number should never exceed what
   // fits in an `int32_t`, which is checked in non-optimized builds in Add().
   int32_t size_ = 0;

toolchain/benchmarking/BUILD

@@ -33,6 +33,7 @@ cc_test(
     size = "small",
     srcs = ["source_gen_test.cpp"],
     data = ["//toolchain/install:install_data"],
+    shard_count = 8,
     deps = [
         ":source_gen_lib",
         "//common:all_llvm_targets",
@@ -93,10 +94,10 @@ sh_test(
     srcs = [":compile_benchmark"],
     args = [
         "--benchmark_dry_run",
-        # Dry-run the smallest (256-line) case of every benchmark. Also run just
-        # the lex benchmark that executes a subprocess to reduce the overhead
-        # and reduce timeouts. The `$$` is repeated for Bazel escaping of `$`.
-        "--benchmark_filter=^(BM_CompileApi.*/256|BM_CompileExec.*Phase::Lex.*/256/real_time)$$",
+        # Dry-run the smallest case of every benchmark. Also run just the lex
+        # benchmark that executes a subprocess to reduce the overhead and reduce
+        # timeouts. The `$$` is repeated for Bazel escaping of `$`.
+        "--benchmark_filter=^(BM_CompileApiFile(DenseDecls|RealisticHeader).*/256|BM_CompileImplFile.*/1024|BM_CompileExec.*Phase::Lex.*/256/real_time)$$",
     ],
     env = cc_env(),
 )

toolchain/benchmarking/compile_benchmark.cpp

@@ -255,6 +255,59 @@ using CompilerFor =
     std::conditional_t<M == Mode::InProcess, InProcessCompiler<L>,
                        SubprocessCompiler<L>>;
 
+// Parameters for the dense declarations pattern, which models an API file:
+// classes of mostly function declarations, with a few small inline definitions.
+static auto DenseDeclParams() -> SourceGen::DenseDeclParams {
+  SourceGen::DenseDeclParams params;
+  params.class_params.inline_function_defs = 1;
+  params.class_params.inline_getters = 1;
+  params.class_params.inline_forwarders = 1;
+  params.free_function_decls_per_class = 2;
+  return params;
+}
+
+// Parameters for the implementation file pattern: classes whose functions and
+// methods are all defined out-of-line, with the default definition sizes. Two
+// thirds of the functions are methods, and they average two parameters (`0..2`
+// from `max_params = 2`, plus the extra parameter added to out-of-line
+// definitions), as in the Carbon toolchain's, LLVM's, and Clang's
+// implementation files.
+static auto ImplFileParams() -> SourceGen::DenseDeclParams {
+  SourceGen::DenseDeclParams params;
+  params.class_params = {.public_function_decls = 2,
+                         .public_function_decl_params = {.max_params = 2},
+                         .public_method_decls = 6,
+                         .public_method_decl_params = {.max_params = 2},
+                         .private_function_decls = 2,
+                         .private_function_decl_params = {.max_params = 2},
+                         .private_method_decls = 2,
+                         .private_method_decl_params = {.max_params = 2},
+                         .private_field_decls = 4};
+  params.define_decls_out_of_line = true;
+  params.free_function_decls_per_class = 2;
+  return params;
+}
+
+// Parameters for the realistic header pattern, which models a typical API file.
+// Its classes define about two thirds of their member functions inline, and
+// the mix of inline definitions approximates the inline definitions in the
+// Carbon toolchain's, LLVM's, and Clang's headers.
+static auto RealisticHeaderParams() -> SourceGen::DenseDeclParams {
+  SourceGen::DenseDeclParams params;
+  params.class_params = {.public_function_decls = 0,
+                         .public_method_decls = 2,
+                         .private_function_decls = 0,
+                         .private_method_decls = 0,
+                         .private_field_decls = 3,
+                         .inline_function_defs = 2,
+                         .inline_function_decl_params = {.max_params = 2},
+                         .inline_getters = 2,
+                         .inline_predicates = 1,
+                         .inline_forwarders = 2};
+  params.free_function_decls_per_class = 1;
+  return params;
+}
+
 // Benchmark on multiple files of the same size but with different source code
 // in order to avoid branch prediction perfectly learning a particular file's
 // structure and shape, and to get closer to a cache-cold benchmark number which
@@ -265,78 +318,75 @@ using CompilerFor =
 // For simplicity, we compute a number of files from the target line count as a
 // heuristic, clamped to the `[min_files, max_files]` range provided by the
 // compilation harness.
-static auto ComputeFileCount(int target_lines, int min_files,
+static auto ComputeFileCount(Lang lang, int target_lines, int min_files,
                              [[maybe_unused]] int max_files) -> int {
+  if (IsBenchmarkDryRun()) {
+    // Dry runs only smoke-test that the benchmark runs cleanly, so generate and
+    // compile just two files. We do more than one to exercise our stability
+    // checks.
+    return 2;
+  }
   int file_count = (1024 * 1024) / target_lines;
+  if (lang == Lang::Cpp) {
+    // Reduce the number of files with C++ to balance the longer compile times.
+    file_count /= 2;
+  }
 #ifndef NDEBUG
   // Use a smaller number of files in debug builds where compiles are slower,
   // capping at the release-mode minimum.
-  return std::max(1, std::min(min_files, file_count));
+  file_count = std::min(min_files, file_count);
 #else
-  return std::max(min_files, std::min(max_files, file_count));
+  file_count = std::min(max_files, file_count);
 #endif
+  return std::max(min_files, file_count);
 }
 
 template <Lang L, Phase P, Mode M = Mode::InProcess>
-static auto BM_CompileApiFileDenseDecls(benchmark::State& state) -> void {
+static auto RunApiFileBenchmark(benchmark::State& state,
+                                const SourceGen::DenseDeclParams& params)
+    -> void {
   using Compiler = CompilerFor<L, M>;
   Compiler bench;
-  CompileHelper carbon_compile_helper;
 
   int target_lines = state.range(0);
   int num_files =
-      ComputeFileCount(target_lines, Compiler::MinFiles, Compiler::MaxFiles);
-  if constexpr (L == Lang::Cpp) {
-    // Reduce the number of files with C++ to balance the longer compile times.
-    num_files /= 2;
-  }
+      ComputeFileCount(L, target_lines, Compiler::MinFiles, Compiler::MaxFiles);
 
   llvm::SmallVector<std::string> sources;
   sources.reserve(num_files);
-
-  double total_bytes = 0.0;
-  double total_lines = 0.0;
-  double total_tokens = 0.0;
-
-  ssize_t first_bytes = 0;
-  ssize_t first_lines = 0;
-
   for (auto _ : llvm::seq(num_files)) {
-    sources.push_back(bench.gen().GenApiFileDenseDecls(
-        target_lines, SourceGen::DenseDeclParams{}));
-    const auto& source = sources.back();
-    ssize_t bytes = source.size();
-    ssize_t lines = llvm::count(source, '\n');
-
-    total_bytes += bytes;
-    total_lines += lines;
-    if constexpr (L == Lang::Carbon) {
-      total_tokens += carbon_compile_helper.GetTokenizedBuffer(source).size();
-    }
-
-    // The generator varies each file's content but not its size, which keeps
-    // results comparable across files and runs.
-    if (sources.size() == 1) {
-      first_bytes = bytes;
-      first_lines = lines;
-      continue;
-    }
-
-    CARBON_CHECK(bytes == first_bytes,
-                 "Generated file {0} has {1} bytes but file 0 has {2}.",
-                 sources.size() - 1, bytes, first_bytes);
-    CARBON_CHECK(lines == first_lines,
-                 "Generated file {0} has {1} lines but file 0 has {2}.",
-                 sources.size() - 1, lines, first_lines);
+    sources.push_back(bench.gen().GenApiFileDenseDecls(target_lines, params));
   }
 
+  ssize_t bytes = sources[0].size();
+  ssize_t lines = llvm::count(sources[0], '\n');
+
+  for (llvm::StringRef source : llvm::ArrayRef(sources).drop_front()) {
+    // The generator varies each file's content but not its size, which keeps
+    // results comparable across files and runs.
+    ssize_t file_bytes = source.size();
+    ssize_t file_lines = llvm::count(source, '\n');
+
+    CARBON_CHECK(file_bytes == bytes,
+                 "Generated file has {0} bytes but expected {1}.", file_bytes,
+                 bytes);
+    CARBON_CHECK(file_lines == lines,
+                 "Generated file has {0} lines but expected {1}.", file_lines,
+                 lines);
+  }
   state.counters["Bytes"] =
-      benchmark::Counter(total_bytes / sources.size(),
-                         benchmark::Counter::kIsIterationInvariantRate);
+      benchmark::Counter(bytes, benchmark::Counter::kIsIterationInvariantRate);
   state.counters["Lines"] =
-      benchmark::Counter(total_lines / sources.size(),
-                         benchmark::Counter::kIsIterationInvariantRate);
+      benchmark::Counter(lines, benchmark::Counter::kIsIterationInvariantRate);
+
+  // TODO: The token count should be stable as well, so we should be able to
+  // switch it to the pattern above.
   if constexpr (L == Lang::Carbon) {
+    double total_tokens = 0.0;
+    CompileHelper carbon_compile_helper;
+    for (llvm::StringRef source : sources) {
+      total_tokens += carbon_compile_helper.GetTokenizedBuffer(source).size();
+    }
     state.counters["Tokens"] =
         benchmark::Counter(total_tokens / sources.size(),
                            benchmark::Counter::kIsIterationInvariantRate);
@@ -367,6 +417,21 @@ static auto BM_CompileApiFileDenseDecls(benchmark::State& state) -> void {
   }
 }
 
+template <Lang L, Phase P, Mode M = Mode::InProcess>
+static auto BM_CompileApiFileDenseDecls(benchmark::State& state) -> void {
+  RunApiFileBenchmark<L, P, M>(state, DenseDeclParams());
+}
+
+template <Lang L, Phase P, Mode M = Mode::InProcess>
+static auto BM_CompileImplFile(benchmark::State& state) -> void {
+  RunApiFileBenchmark<L, P, M>(state, ImplFileParams());
+}
+
+template <Lang L, Phase P, Mode M = Mode::InProcess>
+static auto BM_CompileApiFileRealisticHeader(benchmark::State& state) -> void {
+  RunApiFileBenchmark<L, P, M>(state, RealisticHeaderParams());
+}
+
 // A thin wrapper for the subprocess benchmarks: they reuse the shared
 // implementation above, but register under their own (terser) name rather than
 // spelling out `Mode::Subprocess` at each registration.
@@ -375,44 +440,77 @@ static auto BM_CompileExecApiFileDenseDecls(benchmark::State& state) -> void {
   BM_CompileApiFileDenseDecls<L, P, Mode::Subprocess>(state);
 }
 
-// Applies the shared range configuration used by every compile benchmark:
-// 256-line test cases through 256k-line test cases.
-static auto ConfigureCompileBenchmark(benchmark::Benchmark* b) -> void {
+// Applies the range configuration of the API file benchmarks (dense
+// declarations and realistic header): 256-line test cases through 256k-line
+// test cases.
+static auto ConfigureApiFileBenchmark(benchmark::Benchmark* b) -> void {
   b->RangeMultiplier(4)->Range(256, static_cast<int64_t>(256 * 1024));
 }
 
+// The implementation file benchmarks start at 1024 lines instead, because a
+// class with all of its functions defined doesn't fit in 256 lines.
+static auto ConfigureImplFileBenchmark(benchmark::Benchmark* b) -> void {
+  b->RangeMultiplier(4)->Range(1024, static_cast<int64_t>(256 * 1024));
+}
+
 // In-process benchmarks measure the compiler as a library across each phase of
 // compilation. The mode defaults to `Mode::InProcess`.
 BENCHMARK(BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Lex>)
-    ->Apply(ConfigureCompileBenchmark);
+    ->Apply(ConfigureApiFileBenchmark);
 BENCHMARK(BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Parse>)
-    ->Apply(ConfigureCompileBenchmark);
+    ->Apply(ConfigureApiFileBenchmark);
 BENCHMARK(BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Check>)
-    ->Apply(ConfigureCompileBenchmark);
+    ->Apply(ConfigureApiFileBenchmark);
 BENCHMARK(BM_CompileApiFileDenseDecls<Lang::Cpp, Phase::Lex>)
-    ->Apply(ConfigureCompileBenchmark);
+    ->Apply(ConfigureApiFileBenchmark);
 BENCHMARK(BM_CompileApiFileDenseDecls<Lang::Cpp, Phase::Check>)
-    ->Apply(ConfigureCompileBenchmark);
+    ->Apply(ConfigureApiFileBenchmark);
 
 // Subprocess benchmarks measure the same work end-to-end by executing the
 // installed compiler binary, including process startup cost. They use real
 // (wall-clock) time because the compilation happens in a child process whose
 // CPU time isn't attributed to the benchmark process.
 BENCHMARK(BM_CompileExecApiFileDenseDecls<Lang::Carbon, Phase::Lex>)
-    ->Apply(ConfigureCompileBenchmark)
+    ->Apply(ConfigureApiFileBenchmark)
     ->UseRealTime();
 BENCHMARK(BM_CompileExecApiFileDenseDecls<Lang::Carbon, Phase::Parse>)
-    ->Apply(ConfigureCompileBenchmark)
+    ->Apply(ConfigureApiFileBenchmark)
     ->UseRealTime();
 BENCHMARK(BM_CompileExecApiFileDenseDecls<Lang::Carbon, Phase::Check>)
-    ->Apply(ConfigureCompileBenchmark)
+    ->Apply(ConfigureApiFileBenchmark)
     ->UseRealTime();
 BENCHMARK(BM_CompileExecApiFileDenseDecls<Lang::Cpp, Phase::Lex>)
-    ->Apply(ConfigureCompileBenchmark)
+    ->Apply(ConfigureApiFileBenchmark)
     ->UseRealTime();
 BENCHMARK(BM_CompileExecApiFileDenseDecls<Lang::Cpp, Phase::Check>)
-    ->Apply(ConfigureCompileBenchmark)
+    ->Apply(ConfigureApiFileBenchmark)
     ->UseRealTime();
 
+// In-process benchmarks of the implementation file pattern.
+BENCHMARK(BM_CompileImplFile<Lang::Carbon, Phase::Lex>)
+    ->Apply(ConfigureImplFileBenchmark);
+BENCHMARK(BM_CompileImplFile<Lang::Carbon, Phase::Parse>)
+    ->Apply(ConfigureImplFileBenchmark);
+BENCHMARK(BM_CompileImplFile<Lang::Carbon, Phase::Check>)
+    ->Apply(ConfigureImplFileBenchmark);
+
+BENCHMARK(BM_CompileImplFile<Lang::Cpp, Phase::Lex>)
+    ->Apply(ConfigureImplFileBenchmark);
+BENCHMARK(BM_CompileImplFile<Lang::Cpp, Phase::Check>)
+    ->Apply(ConfigureImplFileBenchmark);
+
+// In-process benchmarks of the realistic header pattern.
+BENCHMARK(BM_CompileApiFileRealisticHeader<Lang::Carbon, Phase::Lex>)
+    ->Apply(ConfigureApiFileBenchmark);
+BENCHMARK(BM_CompileApiFileRealisticHeader<Lang::Carbon, Phase::Parse>)
+    ->Apply(ConfigureApiFileBenchmark);
+BENCHMARK(BM_CompileApiFileRealisticHeader<Lang::Carbon, Phase::Check>)
+    ->Apply(ConfigureApiFileBenchmark);
+
+BENCHMARK(BM_CompileApiFileRealisticHeader<Lang::Cpp, Phase::Lex>)
+    ->Apply(ConfigureApiFileBenchmark);
+BENCHMARK(BM_CompileApiFileRealisticHeader<Lang::Cpp, Phase::Check>)
+    ->Apply(ConfigureApiFileBenchmark);
+
 }  // namespace
 }  // namespace Carbon::Testing

toolchain/benchmarking/source_gen.h

@@ -69,6 +69,15 @@ class SourceGen {
     int max_params = 4;
   };
 
+  // A definition size in a histogram of definition sizes, with its weight.
+  // `statements` counts the statements in a plan-based definition beyond the
+  // three that every such definition has: an accumulator, a statement reading
+  // the parameters, and the final `return`.
+  struct DefSizeWeight {
+    int statements;
+    int weight;
+  };
+
   // Parameters used to generate a class in a generated file.
   //
   // Currently, this uses a fixed number of each kind of declaration, with
@@ -76,9 +85,6 @@ class SourceGen {
   // classes with lots of nested declarations.
   // TODO: Switch these to distributions based on data.
   //
-  // TODO: Add support for generating definitions and parameters to control
-  // them.
-  //
   // TODO: Add heuristic for how many functions have return types.
   struct ClassParams {
     int public_function_decls = 4;
@@ -94,6 +100,46 @@ class SourceGen {
     MethodDeclParams private_method_decl_params = {.max_params = 6};
 
     int private_field_decls = 6;
+
+    // The number of general public class functions to define inline. Unlike the
+    // single-expression member idioms below (`inline_getters`,
+    // `inline_predicates`, and `inline_forwarders`), which have fixed
+    // signatures coupled to paired fields or private declarations on the class,
+    // these take arbitrary parameter lists and have multi-statement definitions
+    // generated from `DefPlan`s (sharing the plan builder with out-of-line
+    // definitions).
+    int inline_function_defs = 0;
+    FunctionDeclParams inline_function_decl_params = {.max_params = 4};
+
+    // The histogram of plan sizes for `inline_function_defs`. A file's inline
+    // function definitions follow it as closely as their number allows.
+    //
+    // The default weights are percentages (rounded to sum to 100) of the
+    // multi-statement function definitions defined in classes in the headers
+    // included by a sample of the Carbon toolchain's, LLVM's, and Clang's
+    // implementation files (single-statement inline definitions are modeled by
+    // `inline_getters`, `inline_predicates`, and `inline_forwarders` below).
+    // Each size is a bucket's median, less the three statements that every
+    // plan-based definition has. The 68% of multi-statement inline definitions
+    // that have two or three statements get `.statements = 0`.
+    llvm::SmallVector<DefSizeWeight> inline_function_def_sizes = {
+        {.statements = 0, .weight = 68}, {.statements = 1, .weight = 23},
+        {.statements = 4, .weight = 5},  {.statements = 8, .weight = 2},
+        {.statements = 13, .weight = 1}, {.statements = 23, .weight = 1},
+    };
+
+    // The number of public methods to define inline that return a field. Each
+    // one returns a field of its own, in addition to `private_field_decls`.
+    int inline_getters = 0;
+
+    // The number of public methods to define inline that test a field with the
+    // field type's predicate template. Each one tests a field of its own.
+    int inline_predicates = 0;
+
+    // The number of public methods to define inline that pass their parameters
+    // on to a private method declaration with the same signature.
+    int inline_forwarders = 0;
+    MethodDeclParams inline_forwarder_params = {.max_params = 4};
   };
 
   // Parameters used to select type _uses_, as opposed to definitions.
@@ -121,33 +167,115 @@ class SourceGen {
   struct TypeUseParams {
     // The weights in the histogram start with a sequence fixed types described
     // with a Carbon and C++ string, and their associated weight.
+    //
+    // Each type also has a value expression, which a definition uses to produce
+    // a value of the type, such as a return value. Types without one, like
+    // pointers, are only used where no value is produced.
+    //
+    // Each type also has consumer templates, which a definition uses to read a
+    // parameter of the type so that no parameter is unused. A template is an
+    // `i32` expression (`int` in C++) with a `{0}` placeholder for the
+    // parameter name. Every type needs at least one.
+    //
+    // A type can also have a predicate template, a `bool` expression with a
+    // `{0}` placeholder for a value of the type. Only types with a predicate
+    // template and a value expression are used for the fields that predicates
+    // test.
+    //
+    // Each type can also have an argument expression, which definitions pass
+    // for parameters of the type when calling free functions. It defaults to
+    // the value expression, so only types without a value expression need to
+    // specify one. It can use the definition's `acc` (an `i32`) and `acc64`
+    // (an `i64`).
     struct FixedTypeWeight {
       llvm::StringRef carbon_spelling;
       llvm::StringRef cpp_spelling;
       int weight;
+      llvm::StringRef carbon_value;
+      llvm::StringRef cpp_value;
+      llvm::SmallVector<llvm::StringRef> carbon_consumers;
+      llvm::SmallVector<llvm::StringRef> cpp_consumers;
+      llvm::StringRef carbon_predicate;
+      llvm::StringRef cpp_predicate;
+      llvm::StringRef carbon_arg;
+      llvm::StringRef cpp_arg;
     };
 
-    llvm::SmallVector<FixedTypeWeight> fixed_type_weights = {
+    // `FixedTypeWeight` exceeds `SmallVector`'s 256-byte limit for default
+    // inline storage.
+    llvm::SmallVector<FixedTypeWeight, 0> fixed_type_weights = {
         // Combined weight of 65 for a core set of builtin types.
-        {.carbon_spelling = "bool", .cpp_spelling = "bool", .weight = 25},
-        {.carbon_spelling = "i32", .cpp_spelling = "int", .weight = 20},
+        {.carbon_spelling = "bool",
+         .cpp_spelling = "bool",
+         .weight = 25,
+         .carbon_value = "true",
+         .cpp_value = "true",
+         .carbon_consumers = {"(if {0} then 1 else 0)",
+                              "(if {0} then 2 else 3)"},
+         .cpp_consumers = {"({0} ? 1 : 0)", "({0} ? 2 : 3)"},
+         .carbon_predicate = "not {0}",
+         .cpp_predicate = "!{0}"},
+        {.carbon_spelling = "i32",
+         .cpp_spelling = "int",
+         .weight = 20,
+         .carbon_value = "0",
+         .cpp_value = "0",
+         .carbon_consumers = {"{0} + 1", "{0} * 2", "({0} % 7) + 1"},
+         .cpp_consumers = {"{0} + 1", "{0} * 2", "({0} % 7) + 1"},
+         .carbon_predicate = "{0} == 0",
+         .cpp_predicate = "{0} == 0"},
         {.carbon_spelling = "i64",
          .cpp_spelling = "std::int64_t",
-         .weight = 10},
-        {.carbon_spelling = "i32*", .cpp_spelling = "int*", .weight = 5},
+         .weight = 10,
+         .carbon_value = "0",
+         .cpp_value = "0",
+         .carbon_consumers = {"({0} as i32) - 2", "(({0} + 1) as i32)"},
+         .cpp_consumers = {"static_cast<int>({0}) - 2",
+                           "static_cast<int>({0} + 1)"},
+         .carbon_predicate = "{0} > 0",
+         .cpp_predicate = "{0} > 0"},
+        {.carbon_spelling = "i32*",
+         .cpp_spelling = "int*",
+         .weight = 5,
+         .carbon_consumers = {"*{0} + 1", "*{0} * 2"},
+         .cpp_consumers = {"*{0} + 1", "*{0} * 2"},
+         .carbon_arg = "&acc",
+         .cpp_arg = "&acc"},
         {.carbon_spelling = "i64*",
          .cpp_spelling = "std::int64_t*",
-         .weight = 5},
+         .weight = 5,
+         .carbon_consumers = {"(*{0} as i32) - 2", "((*{0} * 2) as i32)"},
+         .cpp_consumers = {"static_cast<int>(*{0}) - 2",
+                           "static_cast<int>(*{0} * 2)"},
+         .carbon_arg = "&acc64",
+         .cpp_arg = "&acc64"},
 
         // A weight of 5 distributed across tuple structures
         {.carbon_spelling = "(bool, i64)",
          .cpp_spelling = "std::pair<bool, std::int64_t>",
-         .weight = 2},
+         .weight = 2,
+         .carbon_value = "(true, 0)",
+         .cpp_value = "{true, 0}",
+         .carbon_consumers = {"(if {0}.0 then 1 else 0)", "({0}.1 as i32) + 1"},
+         .cpp_consumers = {"({0}.first ? 1 : 0)",
+                           "static_cast<int>({0}.second) + 1"},
+         .carbon_predicate = "{0}.1 > 0",
+         .cpp_predicate = "{0}.second > 0"},
         {.carbon_spelling = "(i32, i64*)",
          .cpp_spelling = "std::pair<int, std::int64_t*>",
-         .weight = 3},
+         .weight = 3,
+         .carbon_consumers = {"{0}.0 + 1", "{0}.0 * 2", "(*{0}.1 as i32) - 1"},
+         .cpp_consumers = {"{0}.first + 1", "{0}.first * 2",
+                           "static_cast<int>(*{0}.second) - 1"},
+         .carbon_arg = "(0, &acc64)",
+         .cpp_arg = "{0, &acc64}"},
     };
 
+    // Consumer templates for class types, in both languages. These call the
+    // generated `Checksum` method, which is the same in every class.
+    llvm::SmallVector<llvm::StringRef> class_consumers = {
+        "{0}.Checksum()", "{0}.Checksum() + 1", "{0}.Checksum() * 2"};
+
     // The weight for using types declared in the file. These will be randomly
     // shuffled references, and when there are more type references than
     // declared, include repeated references.
@@ -164,6 +292,34 @@ class SourceGen {
 
     // Parameters used to guide the selection of types for use in declarations.
     TypeUseParams type_use_params = {};
+
+    // Whether to define each declared function and method out-of-line, after
+    // every class.
+    bool define_decls_out_of_line = false;
+
+    // The histogram of sizes of out-of-line definitions. A file's definitions
+    // follow it as closely as their number allows.
+    //
+    // The default weights are percentages (rounded to sum to 100, with the
+    // 0.4% tail bucket rounded up to 1%) of the function definitions in a
+    // sample of the Carbon toolchain's, LLVM's, and Clang's implementation
+    // files, in buckets of their statement counts. Each size is a bucket's
+    // median, less the three statements that every generated definition has.
+    // The 43% of real definitions that have at most three statements get no
+    // more statements.
+    llvm::SmallVector<DefSizeWeight> out_of_line_def_sizes = {
+        {.statements = 0, .weight = 43}, {.statements = 2, .weight = 18},
+        {.statements = 5, .weight = 11}, {.statements = 8, .weight = 9},
+        {.statements = 13, .weight = 7}, {.statements = 20, .weight = 5},
+        {.statements = 29, .weight = 3}, {.statements = 43, .weight = 2},
+        {.statements = 67, .weight = 1}, {.statements = 125, .weight = 1},
+    };
+
+    // The number of free function declarations at the start of the file, per
+    // class. Definitions call these. They precede the classes, so their
+    // signatures only use fixed types.
+    int free_function_decls_per_class = 0;
+    FunctionDeclParams free_function_decl_params = {.max_params = 4};
   };
 
   // Access a global instance of this type to generate Carbon code for
@@ -276,14 +432,70 @@ class SourceGen {
 
   auto GetShuffledInts(int number, int min, int max) -> llvm::SmallVector<int>;
 
-  auto GenerateFunctionDecl(
-      llvm::StringRef name, bool is_private, bool is_method, int param_count,
-      llvm::StringRef indent,
-      llvm::SmallVectorImpl<llvm::StringRef>& param_names,
-      llvm::function_ref<auto()->llvm::StringRef> get_type_name,
-      llvm::raw_ostream& os) -> void;
-  auto GenerateClassDef(const ClassParams& params, ClassGenState& state,
+  // A parameter or a field: its name, and the spelling of its type.
+  struct TypedName {
+    llvm::StringRef name;
+    llvm::StringRef type;
+  };
+
+  // A function's signature, and the index of its definition plan.
+  struct FunctionSig {
+    llvm::StringRef name;
+    bool is_method;
+    llvm::SmallVector<TypedName> params;
+    llvm::SmallVector<llvm::StringRef> param_consumers;
+    llvm::StringRef return_type;
+    int def_plan;
+  };
+
+  // A declaration to define out-of-line, after every class: the name of its
+  // class, and its signature.
+  struct OutOfLineDef {
+    llvm::StringRef class_name;
+    FunctionSig sig;
+  };
+
+  auto EmitParams(bool is_method, llvm::ArrayRef<TypedName> params,
+                  llvm::StringRef indent, llvm::raw_ostream& os) -> void;
+  auto EmitFunctionDecl(llvm::StringRef name, bool is_private, bool is_method,
+                        llvm::ArrayRef<TypedName> params,
+                        llvm::StringRef return_type, llvm::StringRef indent,
                         llvm::raw_ostream& os) -> void;
+  auto GenerateFunctionDecl(ClassGenState& state, llvm::StringRef name,
+                            bool is_private, bool is_method, int param_count,
+                            llvm::StringRef indent, llvm::raw_ostream& os,
+                            FunctionSig* captured = nullptr) -> void;
+  auto GenerateFreeFunctionDecls(ClassGenState& state, llvm::raw_ostream& os)
+      -> void;
+  auto EmitArrangedDef(ClassGenState& state, int def_plan,
+                       llvm::ArrayRef<llvm::StringRef> locals,
+                       llvm::StringRef reads, llvm::StringRef return_value,
+                       llvm::StringRef indent, llvm::raw_ostream& os) -> void;
+  auto GenerateFunctionDef(ClassGenState& state, const FunctionSig& sig,
+                           llvm::StringRef indent, llvm::raw_ostream& os)
+      -> void;
+  auto GenerateOutOfLineDef(ClassGenState& state, llvm::StringRef class_name,
+                            const FunctionSig& sig, llvm::raw_ostream& os)
+      -> void;
+  auto GenerateInlineFunctionDef(ClassGenState& state, llvm::StringRef name,
+                                 int param_count, llvm::StringRef indent,
+                                 llvm::raw_ostream& os) -> void;
+  auto GenerateGetter(llvm::StringRef name, TypedName field,
+                      llvm::raw_ostream& os) -> void;
+  auto GeneratePredicate(llvm::StringRef name, TypedName field,
+                         llvm::StringRef predicate, llvm::raw_ostream& os)
+      -> void;
+  auto GenerateForwarder(llvm::StringRef name, llvm::ArrayRef<TypedName> params,
+                         llvm::StringRef return_type, llvm::raw_ostream& os)
+      -> void;
+  auto GenerateMakeFunction(ClassGenState& state, llvm::StringRef class_name,
+                            llvm::ArrayRef<TypedName> fields,
+                            llvm::raw_ostream& os) -> void;
+  auto GenerateChecksumFunction(llvm::raw_ostream& os) -> void;
+  auto GenerateClassDef(const ClassParams& params, ClassGenState& state,
+                        llvm::raw_ostream& os,
+                        llvm::SmallVectorImpl<OutOfLineDef>& out_of_line_defs)
+      -> void;
 
   absl::BitGen rng_;
   llvm::BumpPtrAllocator storage_;

toolchain/benchmarking/source_gen_test.cpp

@@ -28,12 +28,15 @@ namespace {
 using ::testing::AllOf;
 using ::testing::ContainerEq;
 using ::testing::Contains;
+using ::testing::ContainsRegex;
 using ::testing::Each;
 using ::testing::Eq;
 using ::testing::Ge;
 using ::testing::Gt;
+using ::testing::HasSubstr;
 using ::testing::Le;
 using ::testing::MatchesRegex;
+using ::testing::Not;
 using ::testing::SizeIs;
 using ::testing::StrEq;
 using ::testing::TestParamInfo;
@@ -84,7 +87,7 @@ TEST(SourceGenTest, Identifiers) {
   // Check that repeated calls are different in interesting ways, but have the
   // exact same total bytes.
   ssize_t idents_size_sum = SumSizes(idents);
-  for ([[maybe_unused]] auto _ : llvm::seq(10)) {
+  for (auto _ : llvm::seq(10)) {
     auto idents2 = gen.GetShuffledIdentifiers(1000);
     EXPECT_THAT(idents2, SizeIs(1000));
     // Should be (at least) a different shuffle of identifiers.
@@ -449,7 +452,8 @@ TEST_P(GenApiFileDenseDeclsParamsTest, ManyFieldsAndOneMethod) {
       GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
 }
 
-// No functions or methods, so every type use is a fixed type.
+// Classes with only fields (no functions or methods). Because fields have their
+// own type pool, fields can still reference earlier classes.
 TEST_P(GenApiFileDenseDeclsParamsTest, OnlyFields) {
   SourceGen::DenseDeclParams params = {
       .class_params = {.public_function_decls = 0,
@@ -462,5 +466,318 @@ TEST_P(GenApiFileDenseDeclsParamsTest, OnlyFields) {
       GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
 }
 
+// Definitions read class-typed parameters through `Checksum`, and return class
+// values through `Make`.
+static auto InlineDefsParams() -> SourceGen::DenseDeclParams {
+  return {.class_params = {.inline_function_defs = 3,
+                           .inline_getters = 2,
+                           .inline_predicates = 2,
+                           .inline_forwarders = 2}};
+}
+
+TEST_P(GenApiFileDenseDeclsParamsTest, InlineDefs) {
+  SourceGen::DenseDeclParams params = InlineDefsParams();
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(2000, params, 16));
+  EXPECT_TRUE(TestCompile(
+      GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(2000, params)));
+}
+
+TEST(SourceGenTest, GenApiFileDenseDeclsInlineDefs) {
+  SourceGen gen;
+  std::string source = gen.GenApiFileDenseDecls(2000, InlineDefsParams());
+  EXPECT_THAT(source, HasSubstr(".Checksum()"));
+  EXPECT_THAT(source, HasSubstr(".Make()"));
+  EXPECT_THAT(source, HasSubstr("Impl("));
+}
+
+// C++ initializes a class with a default constructor in place of `Make`, which
+// leaves the fields private.
+TEST(SourceGenTest, GenApiFileDenseDeclsInlineDefsCpp) {
+  SourceGen gen(SourceGen::Language::Cpp);
+  std::string source = gen.GenApiFileDenseDecls(2000, InlineDefsParams());
+  EXPECT_THAT(source, HasSubstr("()\n      : tag(0)"));
+  EXPECT_THAT(source, HasSubstr("\n private:\n  int tag;\n"));
+}
+
+// Many large inline definitions and few other declarations. Locals and
+// parameters draw from the same identifiers of each length, and C++ rejects a
+// local that has the same name as a parameter, so compile several seeds.
+TEST_P(GenApiFileDenseDeclsParamsTest, LargeInlineDefs) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {
+          .public_function_decls = 1,
+          .public_method_decls = 1,
+          .private_function_decls = 0,
+          .private_method_decls = 0,
+          .private_field_decls = 4,
+          .inline_function_defs = 8,
+          .inline_function_def_sizes = {{.statements = 20, .weight = 1},
+                                        {.statements = 60, .weight = 1}}}};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 24));
+  for (int _ : llvm::seq(4)) {
+    EXPECT_TRUE(TestCompile(
+        GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
+  }
+}
+
+// Many fields, which `Make` initializes, often with an earlier class's
+// `Make`.
+TEST_P(GenApiFileDenseDeclsParamsTest, ManyFieldsToMake) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.public_function_decls = 1,
+                       .public_method_decls = 1,
+                       .private_function_decls = 0,
+                       .private_method_decls = 0,
+                       .private_field_decls = 24,
+                       .inline_function_defs = 2}};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 24));
+  EXPECT_TRUE(TestCompile(
+      GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
+}
+
+// Many getters and predicates, and no other fields.
+TEST_P(GenApiFileDenseDeclsParamsTest, ManyGettersAndPredicates) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.private_field_decls = 0,
+                       .inline_getters = 8,
+                       .inline_predicates = 8}};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 24));
+  EXPECT_TRUE(TestCompile(
+      GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
+}
+
+// Many forwarders with many parameters, which can have class types.
+TEST_P(GenApiFileDenseDeclsParamsTest, ManyForwarders) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.inline_forwarders = 8,
+                       .inline_forwarder_params = {.max_params = 8}}};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 24));
+  EXPECT_TRUE(TestCompile(
+      GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
+}
+
+// Every declared function and method has an out-of-line definition, using the
+// default size distribution (from 0-statement minimal definitions through
+// deeply nested definitions).
+TEST_P(GenApiFileDenseDeclsParamsTest, OutOfLineDefs) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.inline_function_defs = 2},
+      .define_decls_out_of_line = true};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 16));
+  EXPECT_TRUE(TestCompile(
+      GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
+}
+
+// Large out-of-line definitions, which nest deeply. A definition's block
+// arrangement and which locals its statements read depend on the seed, so
+// compile several seeds.
+TEST_P(GenApiFileDenseDeclsParamsTest, LargeOutOfLineDefs) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.public_function_decls = 1,
+                       .public_method_decls = 2,
+                       .private_function_decls = 0,
+                       .private_method_decls = 1,
+                       .private_field_decls = 2},
+      .define_decls_out_of_line = true,
+      .out_of_line_def_sizes = {{.statements = 40, .weight = 1},
+                                {.statements = 150, .weight = 1}}};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 24));
+  for (int _ : llvm::seq(4)) {
+    EXPECT_TRUE(TestCompile(
+        GetParam(), SourceGen(GetParam()).GenApiFileDenseDecls(3000, params)));
+  }
+}
+
+// Plans declare class-typed locals (using both `Self` and other classes in both
+// inline and out-of-line definitions) and destructure tuples.
+TEST(SourceGenTest, GenApiFileDenseDeclsPlanDeclarations) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.public_function_decls = 1,
+                       .public_method_decls = 2,
+                       .private_function_decls = 0,
+                       .private_method_decls = 1,
+                       .private_field_decls = 2,
+                       .inline_function_defs = 2,
+                       .inline_function_def_sizes = {{.statements = 40,
+                                                      .weight = 1}}},
+      .define_decls_out_of_line = true,
+      .out_of_line_def_sizes = {{.statements = 60, .weight = 1}}};
+
+  SourceGen carbon_gen;
+  std::string source = carbon_gen.GenApiFileDenseDecls(3000, params);
+  auto [carbon_inline, carbon_out_of_line] =
+      llvm::StringRef(source).split("\nfn ");
+  for (llvm::StringRef section : {carbon_inline, carbon_out_of_line}) {
+    EXPECT_THAT(section, HasSubstr(": Self = Make();"));
+    EXPECT_THAT(section, ContainsRegex("var [A-Za-z0-9_]+: [A-Za-z0-9_]{5,} = "
+                                       "[A-Za-z0-9_]+\\.Make\\(\\);"));
+  }
+  EXPECT_THAT(source, ContainsRegex("(let|var) \\([A-Za-z0-9_]+: "));
+
+  SourceGen cpp_gen(SourceGen::Language::Cpp);
+  source = cpp_gen.GenApiFileDenseDecls(3000, params);
+  auto [cpp_inline, cpp_out_of_line] = llvm::StringRef(source).split("\nauto ");
+  for (llvm::StringRef section : {cpp_inline, cpp_out_of_line}) {
+    EXPECT_THAT(section, HasSubstr(" = Self();"));
+    EXPECT_THAT(section, ContainsRegex("[A-Za-z0-9_]{5,} [A-Za-z0-9_]+ = "
+                                       "[A-Za-z0-9_]+\\(\\);"));
+  }
+  EXPECT_THAT(source, HasSubstr("auto ["));
+}
+
+// Definitions call free functions, passing `&acc` and `&acc64` for pointer
+// parameters and destructuring tuple-returning calls.
+TEST_P(GenApiFileDenseDeclsParamsTest, FreeFunctionCalls) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.inline_function_defs = 2},
+      .define_decls_out_of_line = true,
+      .free_function_decls_per_class = 2};
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 16));
+  std::string source = SourceGen(GetParam()).GenApiFileDenseDecls(3000, params);
+  EXPECT_TRUE(TestCompile(GetParam(), source));
+  EXPECT_THAT(source, HasSubstr("&acc"));
+  EXPECT_THAT(source, HasSubstr("&acc64"));
+}
+
+// When no free function returns a tuple in `CallDestructureForms`,
+// `DestructureCall` steps fall back to literal destructuring while keeping file
+// sizes seed-independent.
+TEST_P(GenApiFileDenseDeclsParamsTest, FreeFunctionCallsWithoutTupleReturns) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.public_function_decls = 1,
+                       .public_method_decls = 1,
+                       .private_function_decls = 0,
+                       .private_method_decls = 0,
+                       .private_field_decls = 2},
+      .define_decls_out_of_line = true,
+      .out_of_line_def_sizes = {{.statements = 60, .weight = 1}},
+      .free_function_decls_per_class = 2};
+  // Drop the tuple entries at the end of `fixed_type_weights` so no free
+  // function returns a destructurable tuple.
+  llvm::erase_if(params.type_use_params.fixed_type_weights,
+                 [](const SourceGen::TypeUseParams::FixedTypeWeight& fw) {
+                   return fw.carbon_spelling.starts_with("(");
+                 });
+  EXPECT_THAT(GetParam(), GeneratesSeedIndependentSize(3000, params, 16));
+  std::string source = SourceGen(GetParam()).GenApiFileDenseDecls(3000, params);
+  EXPECT_TRUE(TestCompile(GetParam(), source));
+  EXPECT_THAT(source, HasSubstr("&acc64"));
+  if (GetParam() == SourceGen::Language::Carbon) {
+    EXPECT_THAT(source,
+                ContainsRegex("(let|var) \\([A-Za-z0-9_]+: [^)]+\\) = \\("));
+    EXPECT_THAT(source, Not(ContainsRegex("let \\([^)]+\\) = [A-Za-z]")));
+  } else {
+    EXPECT_THAT(source, HasSubstr("] = std::pair{"));
+    EXPECT_THAT(
+        source,
+        Not(ContainsRegex("auto \\[[A-Za-z0-9_, ]+\\] = [A-Za-z0-9_]+\\(")));
+  }
+}
+
+TEST(SourceGenTest, GenApiFileDenseDeclsFreeFunctionCalls) {
+  SourceGen::DenseDeclParams params = {
+      .class_params = {.public_function_decls = 1,
+                       .public_method_decls = 1,
+                       .private_function_decls = 0,
+                       .private_method_decls = 0,
+                       .private_field_decls = 2},
+      .define_decls_out_of_line = true,
+      .out_of_line_def_sizes = {{.statements = 60, .weight = 1}},
+      .free_function_decls_per_class = 4};
+
+  auto carbon_destructure_call_regex = [](llvm::StringRef first_type,
+                                          llvm::StringRef second_type) {
+    return llvm::formatv(
+               "let \\([A-Za-z0-9_]+: {0}, [A-Za-z0-9_]+: {1}\\) = "
+               "[A-Za-z0-9_]{{5,}\\(",
+               first_type, second_type)
+        .str();
+  };
+
+  SourceGen carbon_gen;
+  std::string source = carbon_gen.GenApiFileDenseDecls(3000, params);
+  auto [carbon_free_decls, carbon_classes_and_defs] =
+      llvm::StringRef(source).split("\nclass ");
+  EXPECT_THAT(carbon_free_decls, ContainsRegex("\nfn [A-Za-z0-9_]{5,}\\("));
+  EXPECT_THAT(carbon_classes_and_defs,
+              ContainsRegex(carbon_destructure_call_regex("bool", "i64")));
+  EXPECT_THAT(carbon_classes_and_defs,
+              ContainsRegex(carbon_destructure_call_regex("i32", "i64\\*")));
+
+  SourceGen cpp_gen(SourceGen::Language::Cpp);
+  source = cpp_gen.GenApiFileDenseDecls(3000, params);
+  auto [cpp_free_decls, cpp_classes_and_defs] =
+      llvm::StringRef(source).split("\nclass ");
+  EXPECT_THAT(cpp_free_decls, ContainsRegex("\nauto [A-Za-z0-9_]{5,}\\("));
+  EXPECT_THAT(cpp_classes_and_defs,
+              ContainsRegex("const auto \\[[A-Za-z0-9_]+, [A-Za-z0-9_]+\\] = "
+                            "[A-Za-z0-9_]{5,}\\("));
+}
+
+// The line estimates have to track the emitted lines closely, or files miss
+// their target size. The target is large so that rounding to a whole number of
+// classes is small next to the tolerance. C++ gets a larger tolerance for its
+// access specifier lines, which the estimates don't count.
+TEST(SourceGenTest, GenApiFileDenseDeclsLineTargetAccuracy) {
+  struct NamedParams {
+    llvm::StringLiteral name;
+    SourceGen::DenseDeclParams params;
+  };
+  llvm::SmallVector<NamedParams, 0> param_set;
+  param_set.push_back({.name = "dense_decls",
+                       .params = {.class_params = {.inline_function_defs = 1,
+                                                   .inline_getters = 1,
+                                                   .inline_predicates = 1,
+                                                   .inline_forwarders = 1},
+                                  .free_function_decls_per_class = 2}});
+  // The implementation file shape.
+  param_set.push_back(
+      {.name = "impl_file",
+       .params = {
+           .class_params = {.public_function_decls = 2,
+                            .public_function_decl_params = {.max_params = 2},
+                            .public_method_decls = 6,
+                            .public_method_decl_params = {.max_params = 2},
+                            .private_function_decls = 2,
+                            .private_function_decl_params = {.max_params = 2},
+                            .private_method_decls = 2,
+                            .private_method_decl_params = {.max_params = 2},
+                            .private_field_decls = 4},
+           .define_decls_out_of_line = true,
+           .free_function_decls_per_class = 2}});
+  // The realistic header shape.
+  param_set.push_back(
+      {.name = "realistic_header",
+       .params = {
+           .class_params = {.public_function_decls = 0,
+                            .public_method_decls = 2,
+                            .private_function_decls = 0,
+                            .private_method_decls = 0,
+                            .private_field_decls = 3,
+                            .inline_function_defs = 2,
+                            .inline_function_decl_params = {.max_params = 2},
+                            .inline_getters = 2,
+                            .inline_predicates = 1,
+                            .inline_forwarders = 2},
+           .free_function_decls_per_class = 1}});
+
+  constexpr int TargetLines = 20000;
+  for (const auto& [name, params] : param_set) {
+    for (SourceGen::Language language :
+         {SourceGen::Language::Carbon, SourceGen::Language::Cpp}) {
+      SourceGen gen(language);
+      std::string source = gen.GenApiFileDenseDecls(TargetLines, params);
+      ssize_t lines = CountLines(source);
+      if (language == SourceGen::Language::Carbon) {
+        // Within 2% of the requested line count.
+        EXPECT_THAT(lines, AllOf(Ge(19600), Le(20400))) << name;
+      } else {
+        // Within 10% of the requested line count.
+        EXPECT_THAT(lines, AllOf(Ge(18000), Le(22000))) << name;
+      }
+    }
+  }
+}
+
 }  // namespace
 }  // namespace Carbon::Testing

toolchain/check/BUILD

@@ -48,6 +48,7 @@ cc_library(
         "function.cpp",
         "generic.cpp",
         "global_init.cpp",
+        "identified_facet_type.cpp",
         "impl.cpp",
         "impl_lookup.cpp",
         "impl_validation.cpp",
@@ -72,6 +73,7 @@ cc_library(
         "require_impls_stack.cpp",
         "return.cpp",
         "scope_stack.cpp",
+        "struct.cpp",
         "subst.cpp",
         "thunk.cpp",
         "type.cpp",
@@ -116,6 +118,7 @@ cc_library(
         "function.h",
         "generic.h",
         "global_init.h",
+        "identified_facet_type.h",
         "impl.h",
         "impl_lookup.h",
         "impl_validation.h",
@@ -145,6 +148,7 @@ cc_library(
         "return.h",
         "scope_index.h",
         "scope_stack.h",
+        "struct.h",
         "subst.h",
         "thunk.h",
         "type.h",
@@ -157,11 +161,13 @@ cc_library(
         ":node_stack",
         "//common:array_stack",
         "//common:check",
+        "//common:checked_save_and_restore",
         "//common:concepts",
         "//common:emplace_by_calling",
         "//common:enum_mask_base",
         "//common:find",
         "//common:growing_range",
+        "//common:increment_scope",
         "//common:map",
         "//common:move_only",
         "//common:ostream",
@@ -186,6 +192,7 @@ cc_library(
         "//toolchain/sem_ir:absolute_node_ref",
         "//toolchain/sem_ir:clang_decl",
         "//toolchain/sem_ir:cpp_file",
+        "//toolchain/sem_ir:diagnostic_loc_converter",
         "//toolchain/sem_ir:expr_info",
         "//toolchain/sem_ir:file",
         "//toolchain/sem_ir:formatter",
@@ -259,6 +266,7 @@ cc_library(
         "//toolchain/base:timings",
         "//toolchain/diagnostics:emitter",
         "//toolchain/diagnostics:format_providers",
+        "//toolchain/diagnostics:renderer",
         "//toolchain/lex:token_index",
         "//toolchain/lex:token_kind",
         "//toolchain/lex:tokenized_buffer",

toolchain/check/action.cpp

@@ -482,6 +482,12 @@ auto Internal::BeginPerformDelayedAction(Context& context) -> void {
   // this changes, we should push a region too.
   context.inst_block_stack().Push();
   context.pattern_block_stack().Push();
+
+  // The action is being performed within a specific, so the instructions it
+  // creates belong to that specific, not to whatever generic we might be in the
+  // middle of checking. Track them in a separate generic region, which
+  // `EndPerformDelayedAction` discards, leaving them unattached.
+  StartGenericDecl(context);
 }
 
 auto Internal::EndPerformDelayedAction(Context& context,
@@ -495,27 +501,32 @@ auto Internal::EndPerformDelayedAction(Context& context,
       (pattern_contents == llvm::ArrayRef(result_id) && contents.empty())) {
     context.inst_block_stack().PopAndDiscard();
     context.pattern_block_stack().PopAndDiscard();
-    return result_id;
+  } else {
+    // Otherwise, create a splice_block to represent the sequence of
+    // instructions created by the action.
+    auto block_id = SemIR::InstBlockId::None;
+    if (pattern_contents.empty()) {
+      block_id = context.inst_block_stack().Pop();
+      context.pattern_block_stack().PopAndDiscard();
+    } else {
+      // TODO: pattern insts can depend on non-pattern insts, so we'll probably
+      // eventually need to support actions that produce both.
+      CARBON_CHECK(contents.empty());
+      block_id = context.pattern_block_stack().Pop();
+      context.inst_block_stack().PopAndDiscard();
+    }
+    auto result = context.insts().GetWithLocId(result_id);
+    result_id =
+        AddInstInNoBlock(context, result.loc_id,
+                         SemIR::SpliceBlock{.type_id = result.inst.type_id(),
+                                            .block_id = block_id,
+                                            .result_id = result_id});
   }
 
-  // Otherwise, create a splice_block to represent the sequence of instructions
-  // created by the action.
-  auto block_id = SemIR::InstBlockId::None;
-  if (pattern_contents.empty()) {
-    block_id = context.inst_block_stack().Pop();
-    context.pattern_block_stack().PopAndDiscard();
-  } else {
-    // TODO: pattern insts can depend on non-pattern insts, so we'll probably
-    // eventually need to support actions that produce both.
-    CARBON_CHECK(contents.empty());
-    block_id = context.pattern_block_stack().Pop();
-    context.inst_block_stack().PopAndDiscard();
-  }
-  auto result = context.insts().GetWithLocId(result_id);
-  return AddInstInNoBlock(context, result.loc_id,
-                          SemIR::SpliceBlock{.type_id = result.inst.type_id(),
-                                             .block_id = block_id,
-                                             .result_id = result_id});
+  // Discard the generic region pushed by `BeginPerformDelayedAction`. This is
+  // done last so that the `splice_block` above is also left unattached.
+  DiscardGenericDecl(context);
+  return result_id;
 }
 
 }  // namespace Carbon::Check

toolchain/check/call.cpp

@@ -83,16 +83,17 @@ static auto ResolveCalleeInCall(Context& context, SemIR::LocId loc_id,
                       " expecting {2} argument{2:s}",
                       Diagnostics::IntAsSelect, Diagnostics::IntAsSelect,
                       Diagnostics::IntAsSelect);
-    CARBON_DIAGNOSTIC(InCallToEntity, Note,
-                      "calling {0:=0:function|=1:generic class|=2:generic "
-                      "interface|=3:generic constraint}"
-                      " declared here",
-                      Diagnostics::IntAsSelect);
+    CARBON_DIAGNOSTIC_LABEL(
+        InCallToEntity, Info,
+        "calling {0:=0:function|=1:generic class|=2:generic "
+        "interface|=3:generic constraint}"
+        " declared here",
+        Diagnostics::IntAsSelect);
     context.emitter()
         .Build(loc_id, CallArgCountMismatch, arg_ids.size(),
                static_cast<int>(entity_kind_for_diagnostic), expected_args_size)
-        .Note(entity.latest_decl_id(), InCallToEntity,
-              static_cast<int>(entity_kind_for_diagnostic))
+        .Attach(entity.latest_decl_id(), InCallToEntity,
+                static_cast<int>(entity_kind_for_diagnostic))
         .Emit();
     return std::nullopt;
   }
@@ -194,23 +195,21 @@ static auto BuildCalleeSpecificFunction(
     // This is an associated function in an interface; the callee is the
     // specific function in the impl that corresponds to the specific function
     // we deduced.
-    callee_id =
-        GetOrAddInst(context, SemIR::LocId(generic_callee_id),
-                     SemIR::SpecificImplFunction{
-                         .type_id = GetSingletonType(
-                             context, SemIR::SpecificFunctionType::TypeInstId),
-                         .callee_id = generic_callee_id,
-                         .specific_id = callee_specific_id});
+    callee_id = GetOrAddInst<SemIR::SpecificImplFunction>(
+        context, SemIR::LocId(generic_callee_id),
+        {.type_id =
+             GetSingletonType(context, SemIR::SpecificFunctionType::TypeInstId),
+         .callee_id = generic_callee_id,
+         .specific_id = callee_specific_id});
   } else {
     // This is a regular generic function. The callee is the specific function
     // we deduced.
-    callee_id =
-        GetOrAddInst(context, SemIR::LocId(generic_callee_id),
-                     SemIR::SpecificFunction{
-                         .type_id = GetSingletonType(
-                             context, SemIR::SpecificFunctionType::TypeInstId),
-                         .callee_id = generic_callee_id,
-                         .specific_id = callee_specific_id});
+    callee_id = GetOrAddInst<SemIR::SpecificFunction>(
+        context, SemIR::LocId(generic_callee_id),
+        {.type_id =
+             GetSingletonType(context, SemIR::SpecificFunctionType::TypeInstId),
+         .callee_id = generic_callee_id,
+         .specific_id = callee_specific_id});
   }
 
   // Add the `self` argument back if there was one.
@@ -249,6 +248,30 @@ auto PerformCallToFunction(Context& context, SemIR::LocId loc_id,
   }
 
   auto& callee = context.functions().Get(callee_function.function_id);
+
+  // We can't call a function before its return type is known. This happens for
+  // a function with a deduced return type that is called from within its own
+  // body, or from the body of a class member function that precedes it. Within
+  // a generic function's body, after its return type is deduced, it can still
+  // only be called with its own generic parameters as arguments. And when a
+  // generic function is instantiated, the instantiation can't call the same
+  // specific before reaching the point where its return type is deduced.
+  if (!ResolveReturnTypeInSpecific(context, loc_id, callee,
+                                   *callee_specific_id)) {
+    CARBON_DIAGNOSTIC(CallBeforeReturnTypeDeduced, Error,
+                      "call to `{0}` before its return type is known",
+                      SemIR::NameId);
+    CARBON_DIAGNOSTIC_LABEL(
+        ReturnTypeDeducedFromDefinition, Info,
+        "return type of `{0}` is deduced from its definition", SemIR::NameId);
+    context.emitter()
+        .Build(loc_id, CallBeforeReturnTypeDeduced, callee.name_id)
+        .Attach(callee.latest_decl_id(), ReturnTypeDeducedFromDefinition,
+                callee.name_id)
+        .Emit();
+    return SemIR::ErrorInst::InstId;
+  }
+
   auto return_type_id =
       callee.GetDeclaredReturnType(context.sem_ir(), *callee_specific_id);
   if (!return_type_id.has_value()) {
@@ -259,9 +282,9 @@ auto PerformCallToFunction(Context& context, SemIR::LocId loc_id,
   if (callee.return_pattern_id.has_value()) {
     Diagnostics::AnnotationScope annotate_diagnostics(
         &context.emitter(), [&](auto& builder) {
-          CARBON_DIAGNOSTIC(IncompleteReturnTypeHere, Note,
-                            "return type declared here");
-          builder.Note(callee.return_pattern_id, IncompleteReturnTypeHere);
+          CARBON_DIAGNOSTIC_LABEL(IncompleteReturnTypeHere, Info,
+                                  "return type declared here");
+          builder.Attach(callee.return_pattern_id, IncompleteReturnTypeHere);
         });
     auto arg_type_id = CheckFunctionReturnPatternType(
         context, loc_id, callee.return_pattern_id, *callee_specific_id);
@@ -280,8 +303,8 @@ auto PerformCallToFunction(Context& context, SemIR::LocId loc_id,
   }
   // Convert the arguments to match the parameters.
   auto converted_args_id =
-      ConvertCallArgs(context, callee_function.self_id, arg_ids, return_arg_id,
-                      callee, *callee_specific_id, is_desugared);
+      ConvertCallArgs(context, loc_id, callee_function.self_id, arg_ids,
+                      return_arg_id, callee, *callee_specific_id, is_desugared);
   switch (callee.special_function_kind) {
     case SemIR::Function::SpecialFunctionKind::Thunk: {
       // If we're about to form a direct call to a thunk, inline it.
@@ -464,7 +487,7 @@ static auto PerformCallHelper(Context& context, SemIR::LocId loc_id,
     }
 
     case CARBON_KIND(SemIR::CalleeCppOverloadSet overload): {
-      return PerformCallToCppFunction(context, loc_id,
+      return PerformCallToCppFunction(context, loc_id, callee_id,
                                       overload.cpp_overload_set_id,
                                       overload.self_id, arg_ids, is_desugared);
     }

toolchain/check/check.cpp

@@ -123,9 +123,9 @@ static auto TrackImport(Map<ImportKey, UnitAndImports*>& api_map,
         !insert_result.is_inserted()) {
       CARBON_DIAGNOSTIC(RepeatedImport, Error,
                         "library imported more than once");
-      CARBON_DIAGNOSTIC(FirstImported, Note, "first import here");
+      CARBON_DIAGNOSTIC_LABEL(FirstImported, Info, "first import here");
       unit_info.emitter.Build(import.node_id, RepeatedImport)
-          .Note(insert_result.value(), FirstImported)
+          .Attach(insert_result.value(), FirstImported)
           .Emit();
       return;
     }
@@ -311,20 +311,17 @@ static auto BuildApiMapAndDiagnosePackaging(
       bool is_api_with_impl_ext = !is_impl && filename.ends_with(ImplExt);
       auto want_ext = is_impl ? ImplExt : ApiExt;
       if (is_api_with_impl_ext || !filename.ends_with(want_ext)) {
+        // The diagnostic is about the file rather than anything in it, so
+        // there is no range to mark and the hint goes in the message: a label
+        // would have no location of its own to be drawn at.
         CARBON_DIAGNOSTIC(
             IncorrectExtension, Error,
-            "file extension of `{0:.impl|}.carbon` required for {0:`impl`|api}",
-            Diagnostics::BoolAsSelect);
-        auto diag = unit_info.emitter.Build(
+            "file extension of `{0:.impl|}.carbon` required for {0:`impl`|api}"
+            "{1:; `.impl.carbon` names an `impl` file|}",
+            Diagnostics::BoolAsSelect, Diagnostics::BoolAsSelect);
+        unit_info.emitter.Emit(
             packaging ? packaging->names.node_id : Parse::NodeId::None,
-            IncorrectExtension, is_impl);
-        if (is_api_with_impl_ext) {
-          CARBON_DIAGNOSTIC(
-              IncorrectExtensionImplNote, Note,
-              "file extension of `.impl.carbon` only allowed for `impl`");
-          diag.Note(Parse::NodeId::None, IncorrectExtensionImplNote);
-        }
-        diag.Emit();
+            IncorrectExtension, is_impl, is_api_with_impl_ext);
       }
     }
   }

toolchain/check/check_fuzzer.cpp

@@ -40,14 +40,25 @@ extern "C" int LLVMFuzzerTestOneInput(const unsigned char* data, size_t size) {
 
   llvm::raw_null_ostream null_ostream;
   Driver driver(fs, install_paths, /*input_stream=*/nullptr, &null_ostream,
-                &null_ostream, /*fuzzing=*/true);
+                &null_ostream, /*error_file=*/{}, /*fuzzing=*/true);
+
+  // Force Unicode and color output. The output stream isn't a terminal, so
+  // diagnostics would otherwise render as plain ASCII without color, and the
+  // fuzzer would never reach the renderer's handling of UTF-8 and double-width
+  // characters.
+  static constexpr llvm::StringLiteral RenderArgs[] = {
+      "--terminal-unicode=always", "--color=always"};
 
   // TODO: Get checking to a point where it can handle invalid parse trees
   // without crashing.
-  if (!driver.RunCommand({"compile", "--phase=parse", TestFileName}).success) {
+  if (!driver
+           .RunCommand({RenderArgs[0], RenderArgs[1], "compile",
+                        "--phase=parse", TestFileName})
+           .success) {
     return 0;
   }
-  driver.RunCommand({"compile", "--phase=check", TestFileName});
+  driver.RunCommand(
+      {RenderArgs[0], RenderArgs[1], "compile", "--phase=check", TestFileName});
   return 0;
 }
 

toolchain/check/check_unit.cpp

@@ -20,6 +20,7 @@
 #include "toolchain/base/kind_switch.h"
 #include "toolchain/check/cpp/generate_ast.h"
 #include "toolchain/check/cpp/import.h"
+#include "toolchain/check/cpp/location.h"
 #include "toolchain/check/diagnostic_helpers.h"
 #include "toolchain/check/generic.h"
 #include "toolchain/check/handle.h"
@@ -34,6 +35,7 @@
 #include "toolchain/check/type_structure.h"
 #include "toolchain/check/unused.h"
 #include "toolchain/diagnostics/diagnostic.h"
+#include "toolchain/diagnostics/renderer.h"
 #include "toolchain/sem_ir/function.h"
 #include "toolchain/sem_ir/ids.h"
 #include "toolchain/sem_ir/import_ir.h"
@@ -93,6 +95,10 @@ auto CheckUnit::Run() -> void {
 
   if (!ProcessNodeIds()) {
     context_.sem_ir().set_has_errors(true);
+
+    // TODO: Are there more steps from `FinishRun` we should be doing here?
+    FinalizeCppAstFragment(context_);
+    DisconnectCppAst(context_);
     return;
   }
 
@@ -380,10 +386,10 @@ auto CheckUnit::ProcessNodeIds() -> bool {
   PrettyStackTraceFunction node_dumper([&](llvm::raw_ostream& output) {
     const auto& tree = tree_and_subtrees_getter_();
     auto converted = tree.NodeToDiagnosticLoc(node_id, /*token_only=*/false);
-    converted.loc.FormatLocation(output);
+    output << Diagnostics::FormatLocation(converted.loc) << ": ";
     output << "Checking " << context_.parse_tree().node_kind(node_id) << "\n";
     // Crash output has a tab indent; try to indent slightly past that.
-    converted.loc.FormatSnippet(output, /*indent=*/10);
+    Diagnostics::PrintSnippet(output, converted.loc, /*indent=*/10);
   });
 
   while (auto maybe_node_id = traversal.Next()) {
@@ -456,7 +462,7 @@ auto CheckUnit::CheckRequiredDeclarations() -> void {
   }
 }
 
-auto CheckUnit::CheckRequiredDefinitions() -> void {
+auto CheckUnit::CheckRequiredDefinitions() -> bool {
   CARBON_DIAGNOSTIC(MissingDefinitionInImpl, Error,
                     "no definition found for declaration in impl file");
   for (SemIR::InstId decl_inst_id : context_.definitions_required_by_decl()) {
@@ -499,25 +505,38 @@ auto CheckUnit::CheckRequiredDefinitions() -> void {
       }
     }
   }
+  context_.definitions_required_by_decl().clear();
 
+  auto new_resolution_occurred = false;
   for (auto [loc, specific_id] :
        GrowingRange(context_.definitions_required_by_use())) {
     // This is using the location for the use. We could track the
     // list of enclosing locations if this was used from a generic.
-    if (!ResolveSpecificDefinition(context_, loc, specific_id)) {
-      CARBON_DIAGNOSTIC(MissingGenericFunctionDefinition, Error,
-                        "use of undefined generic function");
-      CARBON_DIAGNOSTIC(MissingGenericFunctionDefinitionHere, Note,
-                        "generic function declared here");
-      auto generic_decl_id =
-          context_.generics()
-              .Get(context_.specifics().Get(specific_id).generic_id)
-              .decl_id;
-      emitter_.Build(loc, MissingGenericFunctionDefinition)
-          .Note(generic_decl_id, MissingGenericFunctionDefinitionHere)
-          .Emit();
+    switch (ResolveSpecificDefinition(context_, loc, specific_id)) {
+      case DefinitionResolution::NewlyResolved:
+        new_resolution_occurred = true;
+        break;
+      case DefinitionResolution::PreviouslyResolved:
+        break;
+      case DefinitionResolution::NotDefinedYet: {
+        CARBON_DIAGNOSTIC(MissingGenericFunctionDefinition, Error,
+                          "use of undefined generic function");
+        CARBON_DIAGNOSTIC_LABEL(MissingGenericFunctionDefinitionHere, Info,
+                                "generic function declared here");
+        auto generic_decl_id =
+            context_.generics()
+                .Get(context_.specifics().Get(specific_id).generic_id)
+                .decl_id;
+        emitter_.Build(loc, MissingGenericFunctionDefinition)
+            .Attach(generic_decl_id, MissingGenericFunctionDefinitionHere)
+            .Emit();
+        break;
+      }
     }
   }
+
+  context_.definitions_required_by_use().clear();
+  return new_resolution_occurred;
 }
 
 auto CheckUnit::CheckPoisonedConcreteImplLookupQueries() -> void {
@@ -568,27 +587,28 @@ auto CheckUnit::CheckPoisonedConcreteImplLookupQueries() -> void {
             "found `impl` that would change the result of an earlier "
             "use of `{0} as {1}`",
             InstIdAsRawType, SpecificInterfaceIdAsRawType);
-        CARBON_DIAGNOSTIC(
-            PoisonedImplLookupConcreteResultNoteBadImpl, Note,
+        CARBON_DIAGNOSTIC_LABEL(
+            PoisonedImplLookupConcreteResultNoteBadImpl, Info,
             "the use would select the `impl` here but it was not found yet");
-        CARBON_DIAGNOSTIC(PoisonedImplLookupConcreteResultNotePreviousImpl,
-                          Note, "the use had selected the `impl` here");
+        CARBON_DIAGNOSTIC_LABEL(
+            PoisonedImplLookupConcreteResultNotePreviousImpl, Info,
+            "the use had selected the `impl` here");
         emitter_
             .Build(poison.loc_id, PoisonedImplLookupConcreteResult,
                    poison.query.query_self_inst_id,
                    poison.query.query_specific_interface_id)
-            .Note(bad_impl.first_decl_id(),
-                  PoisonedImplLookupConcreteResultNoteBadImpl)
-            .Note(prev_impl.first_decl_id(),
-                  PoisonedImplLookupConcreteResultNotePreviousImpl)
+            .Attach(bad_impl.first_decl_id(),
+                    PoisonedImplLookupConcreteResultNoteBadImpl)
+            .Attach(prev_impl.first_decl_id(),
+                    PoisonedImplLookupConcreteResultNotePreviousImpl)
             .Emit();
       } else {
         CARBON_DIAGNOSTIC(
             PoisonedImplLookupCustomResult, Error,
             "found `impl` for {0} as {1}, which has a custom witness",
             InstIdAsRawType, SpecificInterfaceIdAsRawType);
-        CARBON_DIAGNOSTIC(
-            PoisonedImplLookupCustomResultNoteBadImpl, Note,
+        CARBON_DIAGNOSTIC_LABEL(
+            PoisonedImplLookupCustomResultNoteBadImpl, Info,
             "the use tried to select the `impl` here, but a custom "
             "witness was already chosen");
 
@@ -596,8 +616,8 @@ auto CheckUnit::CheckPoisonedConcreteImplLookupQueries() -> void {
             .Build(poison.loc_id, PoisonedImplLookupCustomResult,
                    poison.query.query_self_inst_id,
                    poison.query.query_specific_interface_id)
-            .Note(prev_impl.first_decl_id(),
-                  PoisonedImplLookupCustomResultNoteBadImpl)
+            .Attach(prev_impl.first_decl_id(),
+                    PoisonedImplLookupCustomResultNoteBadImpl)
             .Emit();
       }
     }
@@ -608,13 +628,56 @@ auto CheckUnit::CheckPoisonedConcreteImplLookupQueries() -> void {
 auto CheckUnit::CheckImpls() -> void { ValidateImplsInFile(context_); }
 
 auto CheckUnit::FinishRun() -> void {
+  // TODO: Find a more optimal way to evaluate mutually-dependent interop
+  // between Carbon and C++.
+  //
+  // Experimental evidence indicates that each additional loop doubles the
+  // run-time of `fail_todo_bidirectional_intorp_too_deep.carbon`. That unit
+  // test isn't terribly complex, so we should probably solve this fairly soon.
+  //
+  // We chose 10 loops to make progress without significantly slowing down debug
+  // unit tests.
+  constexpr auto max_resolution_check_cycles = 10;
+  auto num_resolution_check_cycles = 0;
+
+  while (num_resolution_check_cycles++ < max_resolution_check_cycles) {
+    // Finalizes the C++ portion of the compilation, allowing any pending C++
+    // template instantiations to complete Carbon types and generate required
+    // thunks or function calls.
+    FinalizeCppAstFragment(context_);
+
+    if (!CheckRequiredDefinitions()) {
+      break;
+    }
+  }
+
+  if (num_resolution_check_cycles > max_resolution_check_cycles) {
+    // Point at the code that queued the next C++ instantiation, because that
+    // is what needed another pass. If nothing is queued, point at the top of
+    // the file.
+    auto loc_id = SemIR::LocId::None;
+    auto* cpp_context = context_.cpp_context();
+    if (cpp_context && !cpp_context->sema().PendingInstantiations.empty()) {
+      loc_id = AddImportIRInst(
+          context_.sem_ir(),
+          cpp_context->sema().PendingInstantiations.front().second);
+    }
+    context_.TODO(
+        loc_id, llvm::formatv(
+                    "maximum depth for bidirectional Carbon/C++ interop is {}",
+                    max_resolution_check_cycles));
+    if (cpp_context) {
+      cpp_context->sema().PendingInstantiations.clear();
+    }
+  }
+
   CheckRequiredDeclarations();
-  CheckRequiredDefinitions();
   CheckPoisonedConcreteImplLookupQueries();
   CheckImpls();
 
-  // Finalizes the C++ portion of the compilation.
-  FinishAst(context_);
+  // Disconnects AST generation from the context before popping the file-level
+  // scope and destroying the context.
+  DisconnectCppAst(context_);
 
   // Pop information for the file-level scope.
   context_.sem_ir().set_top_inst_block_id(context_.inst_block_stack().Pop());
@@ -637,6 +700,8 @@ auto CheckUnit::FinishRun() -> void {
       block = context_.generated();
     } else if (reserved_id == SemIR::InstBlockId::Imports) {
       block = context_.imports();
+    } else if (reserved_id == SemIR::InstBlockId::ExtraGlobalVariables) {
+      block = context_.extra_global_variables();
     } else {
       CARBON_FATAL("Unexpected reserved InstBlockId: {0}", reserved_id);
     }

toolchain/check/check_unit.h

@@ -175,9 +175,15 @@ class CheckUnit {
 
   // Checks that each required definition is available. If the definition can be
   // generated by resolving a specific, does so, otherwise emits a diagnostic
-  // for each declaration in context.definitions_required_by_decl() and
-  // context.definitions_required_by_use that doesn't have a definition.
-  auto CheckRequiredDefinitions() -> void;
+  // for each declaration in `context.definitions_required_by_decl()` and
+  // `context.definitions_required_by_use()` that doesn't have a definition.
+  // Each call clears both vectors.
+  //
+  // The C++ AST fragment may not be finalized if an element in
+  // `context.definitions_required_by_use()` is resolved by calling this
+  // function. This function returns true if an element was resolved, and false
+  // otherwise.
+  [[nodiscard]] auto CheckRequiredDefinitions() -> bool;
 
   // Re-run every impl lookup with a concrete result and make sure they find the
   // same witnesses.

toolchain/check/class.cpp

@@ -8,6 +8,7 @@
 #include "toolchain/check/convert.h"
 #include "toolchain/check/cpp/export.h"
 #include "toolchain/check/eval.h"
+#include "toolchain/check/full_pattern_stack.h"
 #include "toolchain/check/function.h"
 #include "toolchain/check/generic.h"
 #include "toolchain/check/impl.h"
@@ -60,21 +61,22 @@ static auto CheckCompleteAdapterClassType(
   const auto& class_info = context.classes().Get(class_id);
   if (class_info.base_id.has_value()) {
     CARBON_DIAGNOSTIC(AdaptWithBase, Error, "adapter with base class");
-    CARBON_DIAGNOSTIC(AdaptWithBaseHere, Note, "`base` declaration is here");
+    CARBON_DIAGNOSTIC_LABEL(AdaptWithBaseHere, Info,
+                            "`base` declaration is here");
     context.emitter()
         .Build(class_info.adapt_id, AdaptWithBase)
-        .Note(class_info.base_id, AdaptWithBaseHere)
+        .Attach(class_info.base_id, AdaptWithBaseHere)
         .Emit();
     return SemIR::ErrorInst::InstId;
   }
 
   if (!field_decls.empty()) {
     CARBON_DIAGNOSTIC(AdaptWithFields, Error, "adapter with fields");
-    CARBON_DIAGNOSTIC(AdaptWithFieldHere, Note,
-                      "first field declaration is here");
+    CARBON_DIAGNOSTIC_LABEL(AdaptWithFieldHere, Info,
+                            "first field declaration is here");
     context.emitter()
         .Build(class_info.adapt_id, AdaptWithFields)
-        .Note(field_decls.front(), AdaptWithFieldHere)
+        .Attach(field_decls.front(), AdaptWithFieldHere)
         .Emit();
     return SemIR::ErrorInst::InstId;
   }
@@ -87,11 +89,11 @@ static auto CheckCompleteAdapterClassType(
           SemIR::Function::VirtualModifier::Virtual) {
         CARBON_DIAGNOSTIC(AdaptWithVirtual, Error,
                           "adapter with virtual function");
-        CARBON_DIAGNOSTIC(AdaptWithVirtualHere, Note,
-                          "first virtual function declaration is here");
+        CARBON_DIAGNOSTIC_LABEL(AdaptWithVirtualHere, Info,
+                                "first virtual function declaration is here");
         context.emitter()
             .Build(class_info.adapt_id, AdaptWithVirtual)
-            .Note(inst_id, AdaptWithVirtualHere)
+            .Attach(inst_id, AdaptWithVirtualHere)
             .Emit();
         return SemIR::ErrorInst::InstId;
       }
@@ -316,11 +318,12 @@ static auto BuildVtable(Context& context, Parse::ClassDefinitionId node_id,
     } else if (!implemented_impls.Lookup(fn_decl.function_id)) {
       CARBON_DIAGNOSTIC(OverrideWithoutVirtualInBase, Error,
                         "override without compatible virtual in base class");
-      CARBON_DIAGNOSTIC(OverrideCandidateArityMismatch, Note,
-                        "base class function has {2:more|fewer} parameters "
-                        "({0} vs {1} excluding `self`)",
-                        Diagnostics::IntAsSelect, Diagnostics::IntAsSelect,
-                        Diagnostics::BoolAsSelect);
+      CARBON_DIAGNOSTIC_LABEL(
+          OverrideCandidateArityMismatch, Info,
+          "base class function has {2:more|fewer} parameters "
+          "({0} vs {1} excluding `self`)",
+          Diagnostics::IntAsSelect, Diagnostics::IntAsSelect,
+          Diagnostics::BoolAsSelect);
       auto builder = context.emitter().Build(SemIR::LocId(inst_id),
                                              OverrideWithoutVirtualInBase);
       if (base_vtable_id.has_value()) {
@@ -338,12 +341,12 @@ static auto BuildVtable(Context& context, Parse::ClassDefinitionId node_id,
           const auto& base_fn = context.sem_ir().functions().Get(fn_id);
           switch (CompareVirtualWithOverrider(base_fn, fn)) {
             case OverrideMatchResult::ArityMismatch:
-              builder.Note(base_fn.first_owning_decl_id,
-                           OverrideCandidateArityMismatch,
-                           base_fn.call_param_ranges.explicit_size() - 1,
-                           fn.call_param_ranges.explicit_size() - 1,
-                           base_fn.call_param_ranges.explicit_size() >
-                               fn.call_param_ranges.explicit_size());
+              builder.Attach(base_fn.first_owning_decl_id,
+                             OverrideCandidateArityMismatch,
+                             base_fn.call_param_ranges.explicit_size() - 1,
+                             fn.call_param_ranges.explicit_size() - 1,
+                             base_fn.call_param_ranges.explicit_size() >
+                                 fn.call_param_ranges.explicit_size());
               break;
             case OverrideMatchResult::NameMismatch:
               // TODO: If the name is similar enough and the overrider otherwise
@@ -434,15 +437,10 @@ static auto CheckCompleteClassType(
            context.types().GetTypeInstId(struct_type_id)});
   class_info.complete_type_witness_id = complete_type_witness_id;
 
-  if (foreign_vtable) {
-    if (class_info.generic_id.has_value()) {
-      context.TODO(class_info.first_decl_id(),
-                   "generic class deriving from C++ virtual class");
-    } else {
-      ExportAndCompleteClassToCpp(
-          context,
-          context.types().GetAs<SemIR::ClassType>(class_info.self_type_id));
-    }
+  if (foreign_vtable && !class_info.generic_id.has_value()) {
+    ExportAndCompleteClassToCpp(
+        context,
+        context.types().GetAs<SemIR::ClassType>(class_info.self_type_id));
   }
 
   return complete_type_witness_id;
@@ -460,17 +458,37 @@ auto ComputeClassObjectRepr(Context& context, Parse::ClassDefinitionId node_id,
 }
 
 auto InNonStaticFieldDecl(Context& context) -> bool {
-  return context.full_pattern_stack().IsCurrentKindClassScopeVarDecl() &&
+  return context.full_pattern_stack().CurrentKindIs(
+             FullPatternStack::Kind::ClassScopeVarDecl) &&
          !context.decl_introducer_state_stack()
               .innermost()
               .modifier_set.HasAnyOf(KeywordModifierSet::Static);
 }
 
 auto InStaticClassScopeVar(Context& context) -> bool {
-  return context.full_pattern_stack().IsCurrentKindClassScopeVarDecl() &&
+  return context.full_pattern_stack().CurrentKindIs(
+             FullPatternStack::Kind::ClassScopeVarDecl) &&
          context.decl_introducer_state_stack()
              .innermost()
              .modifier_set.HasAnyOf(KeywordModifierSet::Static);
 }
 
+auto InClassScopeLet(Context& context) -> bool {
+  return context.full_pattern_stack().CurrentKindIs(
+      FullPatternStack::Kind::ClassScopeLetDecl);
+}
+
+auto LookupClassFieldByStructField(Context& context,
+                                   const SemIR::NameScope& class_scope,
+                                   const SemIR::StructTypeField& struct_field)
+    -> std::optional<SemIR::InstStore::GetAsWithIdResult<SemIR::FieldDecl>> {
+  auto entry_id = class_scope.Lookup(struct_field.name_id);
+  if (!entry_id) {
+    return std::nullopt;
+  }
+  auto entry_inst_id = class_scope.GetEntry(*entry_id).result.target_inst_id();
+  LoadImportRef(context, entry_inst_id);
+  return SemIR::GetClassFieldFromScopeEntry(context.sem_ir(), entry_inst_id);
+}
+
 }  // namespace Carbon::Check

toolchain/check/class.h

@@ -5,7 +5,12 @@
 #ifndef CARBON_TOOLCHAIN_CHECK_CLASS_H_
 #define CARBON_TOOLCHAIN_CHECK_CLASS_H_
 
+#include <optional>
+
 #include "toolchain/check/context.h"
+#include "toolchain/sem_ir/name_scope.h"
+#include "toolchain/sem_ir/struct_type_field.h"
+#include "toolchain/sem_ir/typed_insts.h"
 
 namespace Carbon::Check {
 
@@ -29,6 +34,19 @@ auto InNonStaticFieldDecl(Context& context) -> bool;
 // Whether a static class var decl is currently being checked.
 auto InStaticClassScopeVar(Context& context) -> bool;
 
+// Whether a class scope `let` is currently being checked.
+auto InClassScopeLet(Context& context) -> bool;
+
+// Returns the `FieldDecl` in the class scope corresponding to the given field
+// of the class's object representation, or `nullopt` if there is none, such as
+// for a base class. If the class was imported, this loads the field. See
+// also `SemIR::LookupClassFieldByStructField`, which can be used once checking
+// is complete.
+auto LookupClassFieldByStructField(Context& context,
+                                   const SemIR::NameScope& class_scope,
+                                   const SemIR::StructTypeField& struct_field)
+    -> std::optional<SemIR::InstStore::GetAsWithIdResult<SemIR::FieldDecl>>;
+
 }  // namespace Carbon::Check
 
 #endif  // CARBON_TOOLCHAIN_CHECK_CLASS_H_

toolchain/check/context.cpp

@@ -40,7 +40,8 @@ Context::Context(DiagnosticEmitterBase* emitter,
         TODO(loc_id, label);
       }),
       core_identifiers_(&identifiers()),
-      mangle_string_fingerprint_(mangle_string_fingerprint) {
+      mangle_string_fingerprint_(mangle_string_fingerprint),
+      identified_facet_types_(sem_ir->check_ir_id()) {
   // Prepare fields which relate to the number of IRs available for import.
   import_irs().Reserve(imported_ir_count);
   import_ir_constant_values_.reserve(imported_ir_count);
@@ -75,11 +76,12 @@ auto Context::VerifyOnFinish() const -> void {
   region_stack_.VerifyOnFinish();
   CARBON_CHECK(impl_lookup_stack_.empty());
   CARBON_CHECK(declaring_impl_decls_.empty());
+  CARBON_CHECK(eval_impl_lookup_to_identified_witness_ == 0);
   CARBON_CHECK(return_form_expr_ == std::nullopt);
 
 #ifndef NDEBUG
   if (auto verify = sem_ir_->Verify(); !verify.ok()) {
-    CARBON_FATAL("Built invalid SemIR: {1}\n", verify.error());
+    CARBON_FATAL("Built invalid SemIR: {0}\n", verify.error());
   }
 #endif
 }

toolchain/check/context.h

@@ -21,6 +21,7 @@
 #include "toolchain/check/full_pattern_stack.h"
 #include "toolchain/check/generic_region_stack.h"
 #include "toolchain/check/global_init.h"
+#include "toolchain/check/identified_facet_type.h"
 #include "toolchain/check/inst_block_stack.h"
 #include "toolchain/check/node_stack.h"
 #include "toolchain/check/param_and_arg_refs_stack.h"
@@ -33,7 +34,6 @@
 #include "toolchain/parse/tree_and_subtrees.h"
 #include "toolchain/sem_ir/declared_facet_type.h"
 #include "toolchain/sem_ir/file.h"
-#include "toolchain/sem_ir/identified_facet_type.h"
 #include "toolchain/sem_ir/ids.h"
 #include "toolchain/sem_ir/import_ir.h"
 #include "toolchain/sem_ir/inst.h"
@@ -170,6 +170,10 @@ class Context {
 
   auto exports() -> llvm::SmallVector<SemIR::InstId>& { return exports_; }
 
+  auto extra_global_variables() -> llvm::SmallVector<SemIR::InstId>& {
+    return extra_global_variables_;
+  }
+
   using CheckIRToImportIRStore =
       FixedSizeValueStore<SemIR::CheckIRId, SemIR::ImportIRId>;
   auto check_ir_map() -> CheckIRToImportIRStore& { return check_ir_map_; }
@@ -314,11 +318,16 @@ class Context {
   struct DeclaringImplDecl {
     SemIR::ConstantId self_id;
     SemIR::SpecificInterface specific_interface;
+    auto operator==(const DeclaringImplDecl& rhs) const -> bool = default;
   };
   auto declaring_impl_decls() -> llvm::SmallVector<DeclaringImplDecl>& {
     return declaring_impl_decls_;
   }
 
+  auto eval_impl_lookup_to_identified_witness() -> int32_t& {
+    return eval_impl_lookup_to_identified_witness_;
+  }
+
   // Data about a form expression.
   //
   // TODO: consider moving this out of Context.
@@ -358,6 +367,13 @@ class Context {
 
   auto access_context() -> SemIR::NameScopeId& { return access_context_; }
 
+  auto identified_facet_types() -> IdentifiedFacetTypeStore& {
+    return identified_facet_types_;
+  }
+  auto identified_facet_types() const -> const IdentifiedFacetTypeStore& {
+    return identified_facet_types_;
+  }
+
   // --------------------------------------------------------------------------
   // Directly expose SemIR::File data accessors for brevity in calls.
   // --------------------------------------------------------------------------
@@ -405,9 +421,6 @@ class Context {
   auto declared_facet_types() -> SemIR::DeclaredFacetTypeStore& {
     return sem_ir().declared_facet_types();
   }
-  auto identified_facet_types() -> SemIR::IdentifiedFacetTypeStore& {
-    return sem_ir().identified_facet_types();
-  }
   auto impls() -> SemIR::ImplStore& { return sem_ir().impls(); }
   auto specific_interfaces() -> SemIR::SpecificInterfaceStore& {
     return sem_ir().specific_interfaces();
@@ -505,7 +518,8 @@ class Context {
   // - The associated entries witness table, while parsing an interface.
   InstBlockStack args_type_info_stack_;
 
-  // The stack of StructTypeFields for in-progress StructTypeLiterals.
+  // The stack of StructTypeFields for in-progress StructTypeLiterals,
+  // StructLiteral values, and StructPatterns.
   ArrayStack<SemIR::StructTypeField> struct_type_fields_stack_;
 
   // The stack of FieldDecls for in-progress Class definitions.
@@ -547,6 +561,10 @@ class Context {
   // `InstBlockId::Exports`.
   llvm::SmallVector<SemIR::InstId> exports_;
 
+  // Instructions that will be scanned for export as global variables. This
+  // becomes `InstBlockId::ExtraGlobalVariables`.
+  llvm::SmallVector<SemIR::InstId> extra_global_variables_;
+
   // Maps CheckIRId to ImportIRId.
   CheckIRToImportIRStore check_ir_map_;
 
@@ -637,6 +655,13 @@ class Context {
   // that impl being declared (even though it does not yet exist).
   llvm::SmallVector<DeclaringImplDecl> declaring_impl_decls_;
 
+  // Tracks if we are currently identifying a facet type, which causes
+  // LookupImplWitness instructions to not resolve to final witnesses by
+  // redirecting them to form IdentifiedWitness instructions. The caller to
+  // identify must replace any IdentifiedWitness instructions in the
+  // IdentifiedFacetType before using the values there.
+  int32_t eval_impl_lookup_to_identified_witness_ = 0;
+
   // Declared return form for the in-progress function declaration, if any.
   std::optional<FormExpr> return_form_expr_;
 
@@ -651,6 +676,9 @@ class Context {
   // This is updated by `DeclNameStack`. During monomorphization, it is updated
   // by `TryEvalBlockForSpecific`.
   SemIR::NameScopeId access_context_ = SemIR::NameScopeId::None;
+
+  // Storage for identified facet types.
+  IdentifiedFacetTypeStore identified_facet_types_;
 };
 
 inline constexpr Context::FormExpr Context::FormExpr::Error = {

toolchain/check/control_flow.cpp

@@ -204,9 +204,10 @@ auto AddBranchWithCleanups(Context& context, SemIR::LocId loc_id,
 }
 
 auto AddReturnInstWithCleanups(Context& context,
-                               SemIR::LocIdAndInst loc_id_and_inst) -> void {
+                               SemIR::LocIdAndInst loc_id_and_inst)
+    -> SemIR::InstId {
   AddCleanups(context, context.scope_stack().function_cleanup_scope_depth());
-  AddInst(context, loc_id_and_inst);
+  return AddInst(context, loc_id_and_inst);
 }
 
 }  // namespace Carbon::Check

toolchain/check/control_flow.h

@@ -110,18 +110,20 @@ auto AddBranchWithCleanups(Context& context, SemIR::LocId loc_id,
 
 // Adds a return instruction, along with cleanups for all live variables or
 // temporaries in the current function. If no instruction is passed, it
-// defaults to `SemIR::Return{}`.
+// defaults to `SemIR::Return{}`. Returns the return instruction.
 auto AddReturnInstWithCleanups(Context& context,
-                               SemIR::LocIdAndInst loc_id_and_inst) -> void;
+                               SemIR::LocIdAndInst loc_id_and_inst)
+    -> SemIR::InstId;
 
 template <typename LocT>
-auto AddReturnInstWithCleanups(Context& context, LocT loc) -> void {
-  AddReturnInstWithCleanups(context, SemIR::LocIdAndInst(loc, SemIR::Return{}));
+auto AddReturnInstWithCleanups(Context& context, LocT loc) -> SemIR::InstId {
+  return AddReturnInstWithCleanups(context,
+                                   SemIR::LocIdAndInst(loc, SemIR::Return{}));
 }
 template <typename LocT>
 auto AddReturnInstWithCleanups(Context& context, LocT loc,
-                               SemIR::ReturnExpr inst) -> void {
-  AddReturnInstWithCleanups(context, SemIR::LocIdAndInst(loc, inst));
+                               SemIR::ReturnExpr inst) -> SemIR::InstId {
+  return AddReturnInstWithCleanups(context, SemIR::LocIdAndInst(loc, inst));
 }
 
 }  // namespace Carbon::Check

toolchain/check/convert.cpp

@@ -186,19 +186,20 @@ static auto VisitAllTemporaryStorageArgs(
 // Materializes and returns a temporary initialized from the initializer
 // `init_id`. If `init_id` has a storage arg, it must be a `TemporaryStorage`;
 // if not, this function allocates one for it.
-static auto MaterializeTemporary(Context& context, SemIR::InstId init_id)
-    -> SemIR::InstId {
+static auto MaterializeTemporary(Context& context, SemIR::LocId loc_id,
+                                 SemIR::InstId init_id) -> SemIR::InstId {
   auto& sem_ir = context.sem_ir();
   auto category = SemIR::GetExprCategory(sem_ir, init_id);
   CARBON_CHECK(SemIR::IsInitializerCategory(category));
   auto init = sem_ir.insts().Get(init_id);
   auto storage_id = FindStorageArgForInitializer(sem_ir, init_id);
+  auto init_loc_id = GetInstLocationOrFallback(context, init_id, loc_id);
   if (!storage_id.has_value()) {
     CARBON_CHECK(category == SemIR::ExprCategory::ReprInitializing);
     // The initializer has no storage arg, but we want to produce an ephemeral
     // reference, so we need to allocate temporary storage.
-    storage_id = AddInst<SemIR::TemporaryStorage>(
-        context, SemIR::LocId(init_id), {.type_id = init.type_id()});
+    storage_id = AddInst<SemIR::TemporaryStorage>(context, init_loc_id,
+                                                  {.type_id = init.type_id()});
   }
 
   CARBON_CHECK(
@@ -206,7 +207,7 @@ static auto MaterializeTemporary(Context& context, SemIR::InstId init_id)
       "Storage arg for initializer does not contain a temporary; "
       "initialized multiple times? Have {0}",
       sem_ir.insts().Get(storage_id));
-  return AddInstWithCleanup<SemIR::Temporary>(context, SemIR::LocId(init_id),
+  return AddInstWithCleanup<SemIR::Temporary>(context, init_loc_id,
                                               {.type_id = init.type_id(),
                                                .storage_id = storage_id,
                                                .init_id = init_id});
@@ -217,8 +218,8 @@ static auto MaterializeTemporary(Context& context, SemIR::InstId init_id)
 //
 // TODO: We should probably start its lifetime unconditionally, because
 // types with by-copy representations can still have nontrivial destructors.
-static auto DiscardInitializer(Context& context, SemIR::InstId init_id)
-    -> void {
+static auto DiscardInitializer(Context& context, SemIR::LocId loc_id,
+                               SemIR::InstId init_id) -> void {
   auto& sem_ir = context.sem_ir();
   auto storage_id = FindStorageArgForInitializer(sem_ir, init_id);
   if (!storage_id.has_value()) {
@@ -229,16 +230,16 @@ static auto DiscardInitializer(Context& context, SemIR::InstId init_id)
 
   // init_id writes to temporary storage, so we need to materialize a temporary
   // for it.
-  MaterializeTemporary(context, init_id);
+  MaterializeTemporary(context, loc_id, init_id);
 }
 
 // If `expr_id` is an initializer, materializes it and returns the resulting
 // ephemeral reference expression. Otherwise, returns `expr_id`.
-static auto MaterializeIfInitializer(Context& context, SemIR::InstId expr_id)
-    -> SemIR::InstId {
+static auto MaterializeIfInitializer(Context& context, SemIR::LocId loc_id,
+                                     SemIR::InstId expr_id) -> SemIR::InstId {
   if (SemIR::IsInitializerCategory(
           SemIR::GetExprCategory(context.sem_ir(), expr_id))) {
-    return MaterializeTemporary(context, expr_id);
+    return MaterializeTemporary(context, loc_id, expr_id);
   } else {
     return expr_id;
   }
@@ -407,14 +408,15 @@ static auto ConvertAggregateElement(
 // Performs a conversion from a tuple to an array type. This function only
 // converts the type, and does not perform a final conversion to the requested
 // expression category.
-static auto ConvertTupleToArray(Context& context, SemIR::TupleType tuple_type,
+static auto ConvertTupleToArray(Context& context, SemIR::LocId loc_id,
+                                SemIR::TupleType tuple_type,
                                 SemIR::ArrayType array_type,
                                 SemIR::InstId value_id, ConversionTarget target)
     -> SemIR::InstId {
   auto& sem_ir = context.sem_ir();
   auto tuple_elem_types = sem_ir.inst_blocks().Get(tuple_type.type_elements_id);
 
-  SemIR::LocId value_loc_id(value_id);
+  auto value_loc_id = GetInstLocationOrFallback(context, value_id, loc_id);
 
   // If we're initializing from a tuple literal, we will use its elements
   // directly. Otherwise, materialize a temporary if needed and index into the
@@ -422,7 +424,7 @@ static auto ConvertTupleToArray(Context& context, SemIR::TupleType tuple_type,
   auto literal =
       GetAggregateLiteralElements<SemIR::TupleLiteral>(context, value_id);
   if (!literal.has_value()) {
-    value_id = MaterializeIfInitializer(context, value_id);
+    value_id = MaterializeIfInitializer(context, loc_id, value_id);
   }
 
   // Check that the tuple is the right size.
@@ -522,7 +524,8 @@ static auto ConvertFunctionToCppPointer(
 // Performs a conversion from a tuple to a tuple type. This function only
 // converts the type, and does not perform a final conversion to the requested
 // expression category.
-static auto ConvertTupleToTuple(Context& context, SemIR::TupleType src_type,
+static auto ConvertTupleToTuple(Context& context, SemIR::LocId loc_id,
+                                SemIR::TupleType src_type,
                                 SemIR::TupleType dest_type,
                                 SemIR::InstId value_id, ConversionTarget target)
     -> SemIR::InstId {
@@ -530,7 +533,7 @@ static auto ConvertTupleToTuple(Context& context, SemIR::TupleType src_type,
   auto src_elem_types = sem_ir.inst_blocks().Get(src_type.type_elements_id);
   auto dest_elem_types = sem_ir.inst_blocks().Get(dest_type.type_elements_id);
 
-  SemIR::LocId value_loc_id(value_id);
+  auto value_loc_id = GetInstLocationOrFallback(context, value_id, loc_id);
 
   // If we're initializing from a tuple literal, we will use its elements
   // directly. Otherwise, materialize a temporary if needed and index into the
@@ -538,7 +541,7 @@ static auto ConvertTupleToTuple(Context& context, SemIR::TupleType src_type,
   auto literal =
       GetAggregateLiteralElements<SemIR::TupleLiteral>(context, value_id);
   if (!literal.has_value()) {
-    value_id = MaterializeIfInitializer(context, value_id);
+    value_id = MaterializeIfInitializer(context, loc_id, value_id);
   }
 
   // Check that the tuples are the same size.
@@ -763,10 +766,9 @@ static auto PerformVptrAccess(Context& context, SemIR::LocId loc_id,
 // Converts an initializer for a type `partial T` to an initializer for `T` by
 // initializing the vptr if necessary.
 static auto ConvertPartialInitializerToNonPartial(
-    Context& context, ConversionTarget target,
+    Context& context, SemIR::LocId loc_id, ConversionTarget target,
     SemIR::ClassType vtable_class_type, SemIR::InstId result_id)
     -> SemIR::InstId {
-  auto loc_id = SemIR::LocId(result_id);
   auto vptr_id = CreateVtablePtrRef(context, loc_id, vtable_class_type);
   if (!vptr_id.has_value()) {
     // No vtable pointer in this class, nothing to do.
@@ -793,8 +795,9 @@ static auto ConvertPartialInitializerToNonPartial(
 // Common implementation for ConvertStructToStruct and ConvertStructToClass.
 template <typename TargetAccessInstT, typename GetDefault>
 static auto ConvertStructToStructOrClass(
-    Context& context, SemIR::StructType src_type, SemIR::StructType dest_type,
-    SemIR::InstId value_id, ConversionTarget target, GetDefault get_default,
+    Context& context, SemIR::LocId loc_id, SemIR::StructType src_type,
+    SemIR::StructType dest_type, SemIR::InstId value_id,
+    ConversionTarget target, GetDefault get_default,
     SemIR::ClassType* vtable_class_type = nullptr) -> SemIR::InstId {
   static_assert(std::is_same_v<SemIR::ClassElementAccess, TargetAccessInstT> ||
                 std::is_same_v<SemIR::StructAccess, TargetAccessInstT>);
@@ -808,7 +811,7 @@ static auto ConvertStructToStructOrClass(
   auto dest_elem_fields_size =
       dest_elem_fields.size() - (dest_vptr_index.has_value() ? 1 : 0);
 
-  SemIR::LocId value_loc_id(value_id);
+  auto value_loc_id = GetInstLocationOrFallback(context, value_id, loc_id);
 
   // If we're initializing from a struct literal, we will use its elements
   // directly. Otherwise, materialize a temporary if needed and index into the
@@ -816,7 +819,7 @@ static auto ConvertStructToStructOrClass(
   auto literal =
       GetAggregateLiteralElements<SemIR::StructLiteral>(context, value_id);
   if (!literal.has_value()) {
-    value_id = MaterializeIfInitializer(context, value_id);
+    value_id = MaterializeIfInitializer(context, loc_id, value_id);
   }
 
   // Prepare to look up fields in the source by index. Also check for
@@ -1001,7 +1004,7 @@ static auto ConvertStructToStructOrClass(
                                                 .dest_id = target.storage_id});
     if (vtable_class_type) {
       result_id = ConvertPartialInitializerToNonPartial(
-          context, target, *vtable_class_type, result_id);
+          context, value_loc_id, target, *vtable_class_type, result_id);
     }
     return result_id;
   } else if (is_init) {
@@ -1020,32 +1023,35 @@ static auto ConvertStructToStructOrClass(
 // Performs a conversion from a struct to a struct type. This function only
 // converts the type, and does not perform a final conversion to the requested
 // expression category.
-static auto ConvertStructToStruct(Context& context, SemIR::StructType src_type,
+static auto ConvertStructToStruct(Context& context, SemIR::LocId loc_id,
+                                  SemIR::StructType src_type,
                                   SemIR::StructType dest_type,
                                   SemIR::InstId value_id,
                                   ConversionTarget target) -> SemIR::InstId {
   return ConvertStructToStructOrClass<SemIR::StructAccess>(
-      context, src_type, dest_type, value_id, target,
+      context, loc_id, src_type, dest_type, value_id, target,
       /*get_default=*/[](SemIR::NameId) { return SemIR::InstId::None; });
 }
 
 // Performs a conversion from a struct to a class type. This function only
 // converts the type, and does not perform a final conversion to the requested
 // expression category.
-static auto ConvertStructToClass(Context& context, SemIR::StructType src_type,
+static auto ConvertStructToClass(Context& context, SemIR::LocId loc_id,
+                                 SemIR::StructType src_type,
                                  SemIR::ClassType dest_type,
                                  SemIR::InstId value_id,
                                  ConversionTarget target,
                                  bool is_partial = false) -> SemIR::InstId {
   CARBON_CHECK(target.kind != ConversionTarget::InPlaceInitializing ||
                target.storage_id.has_value());
+  auto value_loc_id = GetInstLocationOrFallback(context, value_id, loc_id);
   PendingBlock target_block(&context);
   auto& dest_class_info = context.classes().Get(dest_type.class_id);
   if (!is_partial &&
       dest_class_info.inheritance_kind == SemIR::Class::Abstract) {
     CARBON_DIAGNOSTIC(AbstractTypeInInit, Error,
                       "initialization of abstract class {0}", SemIR::TypeId);
-    context.emitter().Emit(value_id, AbstractTypeInInit, target.type_id);
+    context.emitter().Emit(value_loc_id, AbstractTypeInInit, target.type_id);
   }
   auto object_repr_id =
       dest_class_info.GetObjectRepr(context.sem_ir(), dest_type.specific_id);
@@ -1065,7 +1071,7 @@ static auto ConvertStructToClass(Context& context, SemIR::StructType src_type,
     target.kind = ConversionTarget::Initializing;
     target.storage_access_block = &target_block;
     target.storage_id = target_block.AddInst<SemIR::TemporaryStorage>(
-        SemIR::LocId(value_id), {.type_id = target.type_id});
+        value_loc_id, {.type_id = target.type_id});
   }
 
   const auto& dest_class_scope =
@@ -1112,8 +1118,8 @@ static auto ConvertStructToClass(Context& context, SemIR::StructType src_type,
   };
 
   return ConvertStructToStructOrClass<SemIR::ClassElementAccess>(
-      context, src_type, dest_struct_type, value_id, target, get_default,
-      is_partial ? nullptr : &dest_type);
+      context, loc_id, src_type, dest_struct_type, value_id, target,
+      get_default, is_partial ? nullptr : &dest_type);
 }
 
 // Represents an edge in the inheritance graph, created by a `base` declaration.
@@ -1492,8 +1498,9 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
         // the type (or some part of it) is not copyable.
         Diagnostics::AnnotationScope annotate_diagnostics(
             &context.emitter(), [&](auto& builder) {
-              CARBON_DIAGNOSTIC(InCopy, Note, "in copy of {0}", TypeOfInstId);
-              builder.Note(value_id, InCopy, value_id);
+              CARBON_DIAGNOSTIC_LABEL(InCopy, Info, "in copy of {0}",
+                                      TypeOfInstId);
+              builder.Attach(value_id, InCopy, value_id);
             });
 
         foundation_value_id = PerformBuiltinConversion(
@@ -1576,7 +1583,8 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
                   context.types().TryGetAs<SemIR::ClassType>(
                       unqual_target_type_id)) {
             value_id = ConvertPartialInitializerToNonPartial(
-                context, target, *target_class_type, value_id);
+                context, GetInstLocationOrFallback(context, value_id, loc_id),
+                target, *target_class_type, value_id);
           }
         }
 
@@ -1616,8 +1624,8 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
   if (auto target_tuple_type = target_type_inst.TryAs<SemIR::TupleType>()) {
     if (auto src_tuple_type =
             sem_ir.types().TryGetAs<SemIR::TupleType>(value_type_id)) {
-      return ConvertTupleToTuple(context, *src_tuple_type, *target_tuple_type,
-                                 value_id, target);
+      return ConvertTupleToTuple(context, loc_id, *src_tuple_type,
+                                 *target_tuple_type, value_id, target);
     }
   }
 
@@ -1628,7 +1636,7 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
   if (auto target_struct_type = target_type_inst.TryAs<SemIR::StructType>()) {
     if (auto src_struct_type =
             sem_ir.types().TryGetAs<SemIR::StructType>(value_type_id)) {
-      return ConvertStructToStruct(context, *src_struct_type,
+      return ConvertStructToStruct(context, loc_id, *src_struct_type,
                                    *target_struct_type, value_id, target);
     }
   }
@@ -1646,8 +1654,8 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
           sem_ir.types().TryGetAs<SemIR::TupleType>(value_type_id)) {
     if (auto target_array_type =
             TryGetAsArrayType(context, loc_id, target.type_id)) {
-      return ConvertTupleToArray(context, *src_tuple_type, *target_array_type,
-                                 value_id, target);
+      return ConvertTupleToArray(context, loc_id, *src_tuple_type,
+                                 *target_array_type, value_id, target);
     }
   }
 
@@ -1682,8 +1690,8 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
                .Get(target_class_type->class_id)
                .adapt_id.has_value()) {
         return ConvertStructToClass(
-            context, *src_struct_type, *target_class_type, value_id, target,
-            target_quals.HasAnyOf(SemIR::TypeQualifiers::Partial));
+            context, loc_id, *src_struct_type, *target_class_type, value_id,
+            target, target_quals.HasAnyOf(SemIR::TypeQualifiers::Partial));
       }
     }
   }
@@ -1856,14 +1864,17 @@ static auto PerformBuiltinConversion(Context& context, SemIR::LocId loc_id,
 
 // Given a value expression, form a corresponding initializer that copies from
 // that value to the specified target, if it is possible to do so.
-static auto PerformCopy(Context& context, SemIR::InstId expr_id,
-                        const ConversionTarget& target) -> SemIR::InstId {
+static auto PerformCopy(Context& context, SemIR::LocId loc_id,
+                        SemIR::InstId expr_id, const ConversionTarget& target)
+    -> SemIR::InstId {
   auto copy_id = BuildUnaryOperator(
-      context, SemIR::LocId(expr_id), {.interface_name = CoreIdentifier::Copy},
-      expr_id, target.diagnose, [&](auto& builder) {
-        CARBON_DIAGNOSTIC(CopyOfUncopyableType, Context,
-                          "cannot copy value of type {0}", TypeOfInstId);
-        builder.Context(expr_id, CopyOfUncopyableType, expr_id);
+      context, GetInstLocationOrFallback(context, expr_id, loc_id),
+      {.interface_name = CoreIdentifier::Copy}, expr_id, target.diagnose,
+      [&](auto& builder) {
+        CARBON_DIAGNOSTIC_CONTEXT(CopyOfUncopyableType,
+                                  "cannot copy value of type {0}",
+                                  TypeOfInstId);
+        builder.Attach(expr_id, CopyOfUncopyableType, expr_id);
       });
   return copy_id;
 }
@@ -1871,8 +1882,8 @@ static auto PerformCopy(Context& context, SemIR::InstId expr_id,
 // Tries to form a `ValueAsRef` conversion that extracts the pointer value from
 // a value expression with a pointer value representation. Returns the converted
 // expression, or None if the conversion was not applicable.
-static auto TryMakeValueAsRef(Context& context, SemIR::InstId expr_id)
-    -> SemIR::InstId {
+static auto TryMakeValueAsRef(Context& context, SemIR::LocId loc_id,
+                              SemIR::InstId expr_id) -> SemIR::InstId {
   auto expr = context.insts().Get(expr_id);
 
   // If the expression has a pointer value representation, extract that and use
@@ -1880,7 +1891,7 @@ static auto TryMakeValueAsRef(Context& context, SemIR::InstId expr_id)
   if (SemIR::ValueRepr::ForType(context.sem_ir(), expr.type_id()).kind ==
       SemIR::ValueRepr::Pointer) {
     return AddInst<SemIR::ValueAsRef>(
-        context, SemIR::LocId(expr_id),
+        context, GetInstLocationOrFallback(context, expr_id, loc_id),
         {.type_id = expr.type_id(), .value_id = expr_id});
   }
 
@@ -1924,25 +1935,25 @@ static auto PerformUserDefinedConversion(Context& context, SemIR::LocId loc_id,
             : target.kind == ConversionTarget::ExplicitUnsafeAs ? 2
                                                                 : 0;
         if (context.types().Is<SemIR::FacetType>(target.type_id)) {
-          CARBON_DIAGNOSTIC(
-              ConversionFailureNonTypeToFacet, Context,
+          CARBON_DIAGNOSTIC_CONTEXT(
+              ConversionFailureNonTypeToFacet,
               "cannot{0:=0: implicitly|:} convert non-type value of type {1} "
               "{2:to|into type implementing} {3}"
               "{0:=1: with `as`|=2: with `unsafe as`|:}",
               Diagnostics::IntAsSelect, TypeOfInstId, Diagnostics::BoolAsSelect,
               SemIR::TypeId);
-          builder.Context(loc_id, ConversionFailureNonTypeToFacet,
-                          target_kind_for_diag, expr_id,
-                          target.type_id == SemIR::TypeType::TypeId,
-                          target.type_id);
+          builder.Attach(loc_id, ConversionFailureNonTypeToFacet,
+                         target_kind_for_diag, expr_id,
+                         target.type_id == SemIR::TypeType::TypeId,
+                         target.type_id);
         } else {
-          CARBON_DIAGNOSTIC(
-              ConversionFailure, Context,
+          CARBON_DIAGNOSTIC_CONTEXT(
+              ConversionFailure,
               "cannot{0:=0: implicitly|:} convert expression of type "
               "{1} to {2}{0:=1: with `as`|=2: with `unsafe as`|:}",
               Diagnostics::IntAsSelect, TypeOfInstId, SemIR::TypeId);
-          builder.Context(loc_id, ConversionFailure, target_kind_for_diag,
-                          expr_id, target.type_id);
+          builder.Attach(loc_id, ConversionFailure, target_kind_for_diag,
+                         expr_id, target.type_id);
         }
       });
 
@@ -2073,7 +2084,7 @@ auto CategoryConverter::DoStep(const SemIR::InstId expr_id,
       }
 
       if (target_.kind == ConversionTarget::Discarded) {
-        DiscardInitializer(context_, expr_id);
+        DiscardInitializer(context_, loc_id_, expr_id);
         return Done{expr_id};
       } else if (IsValidExprCategoryForConversionTarget(category,
                                                         target_.kind)) {
@@ -2084,8 +2095,9 @@ auto CategoryConverter::DoStep(const SemIR::InstId expr_id,
         // already a value representation.
         // TODO: If the target is DurableRef, materialize a VarStorage instead
         // of a TemporaryStorage to lifetime-extend.
-        return NextStep{.expr_id = MaterializeTemporary(context_, expr_id),
-                        .category = SemIR::ExprCategory::EphemeralRef};
+        return NextStep{
+            .expr_id = MaterializeTemporary(context_, loc_id_, expr_id),
+            .category = SemIR::ExprCategory::EphemeralRef};
       }
 
     case SemIR::ExprCategory::DurableRef:
@@ -2108,10 +2120,11 @@ auto CategoryConverter::DoStep(const SemIR::InstId expr_id,
 
       // If we have a reference and don't want one, form a value binding.
       // TODO: Support types with custom value representations.
-      return NextStep{.expr_id = AddInst<SemIR::AcquireValue>(
-                          context_, SemIR::LocId(expr_id),
-                          {.type_id = target_.type_id, .value_id = expr_id}),
-                      .category = SemIR::ExprCategory::Value};
+      return NextStep{
+          .expr_id = AddInst<SemIR::AcquireValue>(
+              context_, GetInstLocationOrFallback(context_, expr_id, loc_id_),
+              {.type_id = target_.type_id, .value_id = expr_id}),
+          .category = SemIR::ExprCategory::Value};
 
     case SemIR::ExprCategory::Value:
       if (target_.kind == ConversionTarget::DurableRef) {
@@ -2139,7 +2152,7 @@ auto CategoryConverter::DoStep(const SemIR::InstId expr_id,
       // When initializing a C++ thunk parameter, try to pass a value "by
       // reference".
       if (target_.kind == ConversionTarget::CppThunkRef) {
-        if (auto result_id = TryMakeValueAsRef(context_, expr_id);
+        if (auto result_id = TryMakeValueAsRef(context_, loc_id_, expr_id);
             result_id.has_value()) {
           return Done{result_id};
         }
@@ -2149,7 +2162,7 @@ auto CategoryConverter::DoStep(const SemIR::InstId expr_id,
       // When initializing from a value, perform a copy.
       if (target_.is_initializer() ||
           target_.kind == ConversionTarget::CppThunkRef) {
-        auto copy_id = PerformCopy(context_, expr_id, target_);
+        auto copy_id = PerformCopy(context_, loc_id_, expr_id, target_);
         if (copy_id == SemIR::ErrorInst::InstId) {
           return Done{SemIR::ErrorInst::InstId};
         }
@@ -2422,7 +2435,8 @@ auto PerformAction(Context& context, SemIR::SpecificId specific_id,
         if (!result_ids[next_result_index].has_value()) {
           result_ids[next_result_index] =
               AddInstInNoBlock<SemIR::TemporaryStorage>(
-                  context, SemIR::LocId(storage_arg_id),
+                  context,
+                  GetInstLocationOrFallback(context, storage_arg_id, loc_id),
                   {.type_id = context.insts().Get(storage_arg_id).type_id()});
         }
         ++next_result_index;
@@ -2480,7 +2494,9 @@ auto Convert(Context& context, SemIR::LocId loc_id, SemIR::InstId expr_id,
     if (target.diagnose) {
       CARBON_DIAGNOSTIC(UseOfNonExprAsValue, Error,
                         "expression cannot be used as a value");
-      context.emitter().Emit(expr_id, UseOfNonExprAsValue);
+      context.emitter().Emit(
+          GetInstLocationOrFallback(context, expr_id, loc_id),
+          UseOfNonExprAsValue);
     }
     return SemIR::ErrorInst::InstId;
   }
@@ -2497,7 +2513,9 @@ auto Convert(Context& context, SemIR::LocId loc_id, SemIR::InstId expr_id,
       if (target.diagnose) {
         CARBON_DIAGNOSTIC(RefTagNoRefParam, Error,
                           "`ref` tag is not an argument to a `ref` parameter");
-        context.emitter().Emit(expr_id, RefTagNoRefParam);
+        context.emitter().Emit(
+            GetInstLocationOrFallback(context, expr_id, loc_id),
+            RefTagNoRefParam);
       }
       return SemIR::ErrorInst::InstId;
     }
@@ -2507,7 +2525,9 @@ auto Convert(Context& context, SemIR::LocId loc_id, SemIR::InstId expr_id,
     if (target.diagnose) {
       CARBON_DIAGNOSTIC(RefParamNoRefTag, Error,
                         "argument to `ref` parameter not marked with `ref`");
-      context.emitter().Emit(expr_id, RefParamNoRefTag);
+      context.emitter().Emit(
+          GetInstLocationOrFallback(context, expr_id, loc_id),
+          RefParamNoRefTag);
     }
     return SemIR::ErrorInst::InstId;
   }
@@ -2523,17 +2543,17 @@ auto Convert(Context& context, SemIR::LocId loc_id, SemIR::InstId expr_id,
                     !target.is_initializer(),
                     "Initialization of incomplete types is expected to be "
                     "caught elsewhere.");
-                CARBON_DIAGNOSTIC(IncompleteTypeInValueConversion, Context,
-                                  "forming value of incomplete type {0}",
-                                  SemIR::TypeId);
-                CARBON_DIAGNOSTIC(IncompleteTypeInConversion, Context,
-                                  "invalid use of incomplete type {0}",
-                                  SemIR::TypeId);
-                builder.Context(loc_id,
-                                target.kind == ConversionTarget::Value
-                                    ? IncompleteTypeInValueConversion
-                                    : IncompleteTypeInConversion,
-                                target.type_id);
+                CARBON_DIAGNOSTIC_CONTEXT(
+                    IncompleteTypeInValueConversion,
+                    "forming value of incomplete type {0}", SemIR::TypeId);
+                CARBON_DIAGNOSTIC_CONTEXT(IncompleteTypeInConversion,
+                                          "invalid use of incomplete type {0}",
+                                          SemIR::TypeId);
+                builder.Attach(loc_id,
+                               target.kind == ConversionTarget::Value
+                                   ? IncompleteTypeInValueConversion
+                                   : IncompleteTypeInConversion,
+                               target.type_id);
               })) {
         return SemIR::ErrorInst::InstId;
       }
@@ -2704,7 +2724,8 @@ auto ConvertForExplicitAs(Context& context, Parse::NodeId as_node,
 }
 
 // TODO: Consider moving this to pattern_match.h.
-auto ConvertCallArgs(Context& context, SemIR::InstId self_id,
+auto ConvertCallArgs(Context& context, SemIR::LocId loc_id,
+                     SemIR::InstId self_id,
                      llvm::ArrayRef<SemIR::InstId> arg_refs,
                      SemIR::InstId return_arg_id, const SemIR::Function& callee,
                      SemIR::SpecificId callee_specific_id, bool is_desugared)
@@ -2719,9 +2740,10 @@ auto ConvertCallArgs(Context& context, SemIR::InstId self_id,
       !return_arg_id.has_value() || callee.call_param_ranges.return_size() > 0,
       "Return argument provided for a callee with no return parameter");
 
-  return CallerPatternMatch(context, callee_specific_id, callee.self_param_id,
-                            callee.param_patterns_id, callee.return_pattern_id,
-                            self_id, arg_refs, return_arg_id, is_desugared);
+  return CallerPatternMatch(context, loc_id, callee_specific_id,
+                            callee.self_param_id, callee.param_patterns_id,
+                            callee.return_pattern_id, self_id, arg_refs,
+                            return_arg_id, is_desugared);
 }
 
 auto TypeExpr::ForUnsugared(Context& context, SemIR::TypeId type_id)
@@ -2858,7 +2880,7 @@ auto UnsafeUndoConvert(Context& context, SemIR::LocId loc_id,
     case SemIR::ExprCategory::DurableRef:
       break;
     case SemIR::ExprCategory::Value:
-      if (auto ref_expr_id = TryMakeValueAsRef(context, expr_id);
+      if (auto ref_expr_id = TryMakeValueAsRef(context, loc_id, expr_id);
           ref_expr_id.has_value()) {
         expr_id = ref_expr_id;
       } else {

toolchain/check/convert.h

@@ -209,7 +209,8 @@ auto ConvertForExplicitAs(Context& context, Parse::NodeId as_node,
 // argument values for runtime parameters. `is_desugared` indicates that this
 // call was produced by desugaring, not written as a function call in user code,
 // so arguments to `ref` parameters aren't required to have `ref` tags.
-auto ConvertCallArgs(Context& context, SemIR::InstId self_id,
+auto ConvertCallArgs(Context& context, SemIR::LocId loc_id,
+                     SemIR::InstId self_id,
                      llvm::ArrayRef<SemIR::InstId> arg_refs,
                      SemIR::InstId return_arg_id, const SemIR::Function& callee,
                      SemIR::SpecificId callee_specific_id, bool is_desugared)
@@ -227,6 +228,11 @@ struct TypeExpr {
   SemIR::TypeInstId inst_id;
   // The corresponding type, or `ErrorInst::TypeId`.
   SemIR::TypeId type_id;
+
+  auto SetError() {
+    inst_id = SemIR::ErrorInst::TypeInstId;
+    type_id = SemIR::ErrorInst::TypeId;
+  }
 };
 
 inline constexpr TypeExpr TypeExpr::None = {.inst_id = SemIR::TypeInstId::None,
@@ -247,9 +253,13 @@ auto ExprAsType(Context& context, SemIR::LocId loc_id, SemIR::InstId value_id,
 
 // Converts an expression in a form position for use as a form.
 //
+// Form positions include return-forms (following `->?`) and form binding
+// patterns.
+//
 // Note that the right-hand side of a `->` return type declaration is normally
 // a type expression, not a form, and should be handled by `ReturnExprAsForm`.
-// It's only a form if it begins with a primitive form keyword such as `ref`.
+// It's only a form if it begins with a primitive form keyword: `val`, `ref`,
+// or `var`.
 //
 // `diagnose` has the same effect as in `ExprAsType`.
 auto FormExprAsForm(Context& context, SemIR::LocId loc_id,

toolchain/check/core_identifier.def

@@ -39,7 +39,7 @@ CARBON_CORE_IDENTIFIER(CppUnsafeDeref)
 CARBON_CORE_IDENTIFIER(Dec)
 CARBON_CORE_IDENTIFIER(Default)
 CARBON_CORE_IDENTIFIER(DefaultOrUnformed)
-CARBON_CORE_IDENTIFIER(Destroy)
+CARBON_CORE_IDENTIFIER(DestroyImpl)
 CARBON_CORE_IDENTIFIER(DivAssignWith)
 CARBON_CORE_IDENTIFIER(DivWith)
 CARBON_CORE_IDENTIFIER(EqWith)
@@ -53,6 +53,7 @@ CARBON_CORE_IDENTIFIER(HasValue)
 CARBON_CORE_IDENTIFIER(ImplicitAs)
 CARBON_CORE_IDENTIFIER(Inc)
 CARBON_CORE_IDENTIFIER(IndexWith)
+CARBON_CORE_IDENTIFIER(IndirectIndexWith)
 CARBON_CORE_IDENTIFIER(Int)
 CARBON_CORE_IDENTIFIER(IntFitsIn)
 CARBON_CORE_IDENTIFIER(Iterate)
@@ -75,6 +76,7 @@ CARBON_CORE_IDENTIFIER(NullptrT)
 CARBON_CORE_IDENTIFIER(Op)
 CARBON_CORE_IDENTIFIER(Optional)
 CARBON_CORE_IDENTIFIER(OrderedWith)
+CARBON_CORE_IDENTIFIER(Ref)
 CARBON_CORE_IDENTIFIER(RightShiftAssignWith)
 CARBON_CORE_IDENTIFIER(RightShiftWith)
 CARBON_CORE_IDENTIFIER(SelfDestruct)

toolchain/check/cpp/call.cpp

@@ -52,6 +52,7 @@ static auto SplitCallArgumentList(Context& context,
 }
 
 auto PerformCallToCppFunction(Context& context, SemIR::LocId loc_id,
+                              SemIR::InstId callee_expr_id,
                               SemIR::CppOverloadSetId overload_set_id,
                               SemIR::InstId self_id,
                               llvm::ArrayRef<SemIR::InstId> arg_ids,
@@ -59,8 +60,9 @@ auto PerformCallToCppFunction(Context& context, SemIR::LocId loc_id,
   auto [template_arg_ids, function_arg_ids] =
       SplitCallArgumentList(context, arg_ids);
   auto callee_id = PerformCppOverloadResolution(
-      context, loc_id, context.cpp_overload_sets().Get(overload_set_id),
-      template_arg_ids, self_id, function_arg_ids);
+      context, loc_id, callee_expr_id,
+      context.cpp_overload_sets().Get(overload_set_id), template_arg_ids,
+      self_id, function_arg_ids);
   SemIR::Callee callee = GetCallee(context.sem_ir(), callee_id);
   CARBON_KIND_SWITCH(callee) {
     case CARBON_KIND(SemIR::CalleeError _): {

toolchain/check/cpp/call.h

@@ -31,6 +31,9 @@ auto ConvertArgsToTemplateArgs(Context& context,
 // function call in user code, so arguments to `ref` parameters aren't required
 // to have `ref` tags.
 //
+// `callee_expr_id` is the expression naming the overload set, which is what a
+// diagnostic about the call marks.
+//
 // Chooses the best viable C++ function by performing Clang overloading
 // resolution over the overload set.
 //
@@ -44,6 +47,7 @@ auto ConvertArgsToTemplateArgs(Context& context,
 // possible. Keeping the same behavior here for consistency and supporting
 // migrations so that the migrated callers from C++ remain valid.
 auto PerformCallToCppFunction(Context& context, SemIR::LocId loc_id,
+                              SemIR::InstId callee_expr_id,
                               SemIR::CppOverloadSetId overload_set_id,
                               SemIR::InstId self_id,
                               llvm::ArrayRef<SemIR::InstId> arg_ids,

toolchain/check/cpp/context.h

@@ -9,15 +9,18 @@
 
 #include "clang/Basic/SourceLocation.h"
 #include "common/check.h"
+#include "common/map.h"
 #include "llvm/ADT/SmallVector.h"
 #include "toolchain/check/cpp/diagnostic_listener.h"
 #include "toolchain/sem_ir/cpp_domain.h"
+#include "toolchain/sem_ir/ids.h"
 
 namespace clang {
 class ASTContext;
 class CompilerInstance;
 class FunctionDecl;
 class MangleContext;
+class NamespaceDecl;
 class Parser;
 class Sema;
 }  // namespace clang
@@ -48,6 +51,17 @@ class CppContext {
     return carbon_file_locations_;
   }
 
+  auto carbon_lookup_namespace() const -> clang::NamespaceDecl* {
+    return carbon_lookup_namespace_;
+  }
+  void set_carbon_lookup_namespace(clang::NamespaceDecl* decl) {
+    carbon_lookup_namespace_ = decl;
+  }
+
+  auto lookup_namespaces() -> Map<SemIR::NameScopeId, clang::NamespaceDecl*>& {
+    return lookup_namespaces_;
+  }
+
   auto placement_new_decl() const -> clang::FunctionDecl* {
     return placement_new_decl_;
   }
@@ -59,6 +73,12 @@ class CppContext {
   // The C++ compilation domain.
   SemIR::CppDomain* domain_;
 
+  // The per-file lookup-only `::Carbon` namespace and child namespaces.
+  // These are only used for C++ name lookup within this Carbon file, and are
+  // never used as the parent `DeclContext` for exported Carbon entities.
+  clang::NamespaceDecl* carbon_lookup_namespace_ = nullptr;
+  Map<SemIR::NameScopeId, clang::NamespaceDecl*> lookup_namespaces_;
+
   // TODO: All of the below state that is not specific to a particular
   // Check::Context or SemIR::File should be moved into CppDomain.
 

toolchain/check/cpp/diagnostic_consumer.cpp

@@ -8,11 +8,11 @@
 #include <string>
 
 #include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticSema.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Frontend/CompilerInvocation.h"
-#include "clang/Frontend/TextDiagnostic.h"
+#include "clang/Lex/Lexer.h"
 #include "common/check.h"
-#include "common/raw_string_ostream.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
@@ -21,31 +21,27 @@
 #include "toolchain/check/cpp/location.h"
 #include "toolchain/diagnostics/diagnostic.h"
 #include "toolchain/diagnostics/emitter.h"
+#include "toolchain/sem_ir/diagnostic_loc_converter.h"
 
 namespace Carbon::Check {
 
 class CarbonClangDiagnosticConsumer;
 
-// A diagnostic emitter that maps Clang SourceLocations to Carbon diagnostic
+// A diagnostic emitter that maps Clang source ranges to Carbon diagnostic
 // locations.
 class ClangLocDiagnosticEmitter
-    : public Diagnostics::Emitter<clang::SourceLocation> {
+    : public Diagnostics::Emitter<clang::CharSourceRange> {
  public:
   explicit ClangLocDiagnosticEmitter(Diagnostics::Consumer* consumer,
                                      const clang::SourceManager* source_manager)
       : Emitter(consumer), source_manager_(source_manager) {}
 
  protected:
-  auto ConvertLoc(clang::SourceLocation loc, ContextFnT /*context_fn*/) const
+  auto ConvertLoc(clang::CharSourceRange range, ContextFnT /*context_fn*/) const
       -> Diagnostics::ConvertedLoc override {
     Diagnostics::Loc result_loc;
-    if (source_manager_ && loc.isValid()) {
-      clang::PresumedLoc presumed_loc = source_manager_->getPresumedLoc(loc);
-      if (presumed_loc.isValid()) {
-        result_loc.filename = presumed_loc.getFilename();
-        result_loc.line_number = presumed_loc.getLine();
-        result_loc.column_number = presumed_loc.getColumn();
-      }
+    if (source_manager_) {
+      result_loc = SemIR::ConvertClangRangeToLoc(*source_manager_, range);
     }
     return {.loc = result_loc, .last_byte_offset = -1};
   }
@@ -54,6 +50,34 @@ class ClangLocDiagnosticEmitter
   const clang::SourceManager* source_manager_;
 };
 
+// Attaches a label saying what Clang suggests be done to a range. Clang words
+// these as an edit to make, so they read as an instruction rather than as a
+// description of the code.
+//
+// TODO: Carry fix-its as data -- span, replacement, confidence -- in a form
+// Carbon's own diagnostics can share, rather than wording the edit into a
+// label here, and render both as a unified diff following GCC's example
+// rather than Clang's. See future work in
+// /toolchain/docs/diagnostics_rendering.md.
+template <typename BuilderT, typename LocT>
+static auto AttachFixIt(BuilderT& builder, LocT loc,
+                        const CppDiagnosticListener::FixIt& fix_it) -> void {
+  if (fix_it.text.empty()) {
+    CARBON_DIAGNOSTIC_LABEL(CppInteropFixItRemoval, Info, "remove this");
+    builder.Attach(loc, CppInteropFixItRemoval);
+    return;
+  }
+  if (fix_it.range.getBegin() == fix_it.range.getEnd()) {
+    CARBON_DIAGNOSTIC_LABEL(CppInteropFixItInsertion, Info, "insert `{0}` here",
+                            std::string);
+    builder.Attach(loc, CppInteropFixItInsertion, fix_it.text);
+    return;
+  }
+  CARBON_DIAGNOSTIC_LABEL(CppInteropFixItReplacement, Info,
+                          "replace with `{0}`", std::string);
+  builder.Attach(loc, CppInteropFixItReplacement, fix_it.text);
+}
+
 // Returns the diagnostic to use for a given Clang diagnostic level.
 static auto GetDiagnostic(clang::DiagnosticsEngine::Level level)
     -> const Diagnostics::DiagnosticBase<std::string>& {
@@ -63,8 +87,13 @@ static auto GetDiagnostic(clang::DiagnosticsEngine::Level level)
       break;
     }
     case clang::DiagnosticsEngine::Note: {
-      CARBON_DIAGNOSTIC(CppInteropParseNote, Note, "{0}", std::string);
-      return CppInteropParseNote;
+      // A note explains the diagnostic it follows and is attached to that one
+      // as a label. One can still lead a buffer -- a flush can land between an
+      // error and its trailing notes, and Clang can emit a stray note -- and
+      // however wrong that is of Clang, it must not be why a compiler dies
+      // while reporting a problem.
+      CARBON_DIAGNOSTIC(CppInteropStrayNote, Warning, "{0}", std::string);
+      return CppInteropStrayNote;
     }
     case clang::DiagnosticsEngine::Remark:
     case clang::DiagnosticsEngine::Warning: {
@@ -81,6 +110,158 @@ static auto GetDiagnostic(clang::DiagnosticsEngine::Level level)
   }
 }
 
+// Returns whether `range` holds `loc`, which needs both to be spelled in the
+// same file: two offsets from different files are unordered, and comparing them
+// would report a containment that isn't one.
+static auto Contains(const clang::SourceManager& source_manager,
+                     clang::CharSourceRange range, clang::SourceLocation loc)
+    -> bool {
+  if (range.isInvalid()) {
+    return false;
+  }
+  CARBON_CHECK(range.isCharRange(), "Widen a token range before comparing.");
+  auto [file_id, offset] = source_manager.getDecomposedSpellingLoc(loc);
+  auto [begin_file_id, begin] =
+      source_manager.getDecomposedSpellingLoc(range.getBegin());
+  auto [end_file_id, end] =
+      source_manager.getDecomposedSpellingLoc(range.getEnd());
+  return begin_file_id == file_id && end_file_id == file_id &&
+         begin <= offset && offset < end;
+}
+
+// Returns whether `diag_id` is a Clang diagnostic whose text is written as
+// `<what was considered>: <why it was not viable>`, with its first range
+// marking the source that second half is about.
+//
+// These are the overload-resolution candidate notes, and each one is listed
+// rather than being recognized by shape. Splitting on the first `: ` in any
+// note that happens to carry a range would cut the wrong messages in the wrong
+// place: a type or a path in the text supplies its own colon.
+//
+// TODO: Try to do this upstream instead. Clang has the structure here and
+// loses it: `DiagnoseBadConversion` and its siblings know which half is the
+// candidate and which is the reason, and know which argument the reason is
+// about, and then format both halves into one string. A note carrying its two
+// halves separately would serve Clang's own rendering as well, and would leave
+// nothing here to take apart. Until then this list has to be revisited
+// whenever the wording of one of these notes changes.
+static auto SplitsIntoCandidateAndReason(unsigned diag_id) -> bool {
+  switch (diag_id) {
+    case clang::diag::note_ovl_candidate_arity:
+    case clang::diag::note_ovl_candidate_arity_one:
+    case clang::diag::note_ovl_candidate_bad_base_to_derived_conv:
+    case clang::diag::note_ovl_candidate_bad_conv:
+    case clang::diag::note_ovl_candidate_bad_conv_incomplete:
+    case clang::diag::note_ovl_candidate_bad_cvr:
+    case clang::diag::note_ovl_candidate_bad_cvr_this:
+    case clang::diag::note_ovl_candidate_bad_list_argument:
+    case clang::diag::note_ovl_candidate_bad_value_category:
+      return true;
+    default:
+      return false;
+  }
+}
+
+// Returns the parameter list of the declaration named at `loc`, which is what
+// a note marks when it has nothing of its own to mark.
+//
+// Clang leaves a function's parameter-list range empty when the function
+// declares no parameters, so a note about how many arguments it takes marks
+// nothing even though the list is written. The list is the parentheses after
+// the name, holding at most `void`, and it is where the count comes from.
+//
+// Returns an invalid range where those parentheses aren't right there: the name
+// may be an implicit function's, which has no list at all, or be followed by
+// template arguments, and reaching further would mark something that isn't the
+// list.
+static auto FindEmptyParameterList(const clang::SourceManager& source_manager,
+                                   const clang::LangOptions& lang_opts,
+                                   clang::SourceLocation loc)
+    -> clang::CharSourceRange {
+  if (loc.isInvalid() || loc.isMacroID()) {
+    return clang::CharSourceRange();
+  }
+  // `operator()`'s name holds parentheses of its own, and the parameter list
+  // is past them; the token after the name would be the name's `(`, not the
+  // list's.
+  clang::Token name;
+  if (clang::Lexer::getRawToken(loc, name, source_manager, lang_opts,
+                                /*IgnoreWhiteSpace=*/true) ||
+      (name.is(clang::tok::raw_identifier) &&
+       name.getRawIdentifier() == "operator")) {
+    return clang::CharSourceRange();
+  }
+  auto open = clang::Lexer::findNextToken(loc, source_manager, lang_opts);
+  if (!open || !open->is(clang::tok::l_paren)) {
+    return clang::CharSourceRange();
+  }
+  auto close = clang::Lexer::findNextToken(open->getLocation(), source_manager,
+                                           lang_opts);
+  // An empty list is written as `()` or as `(void)`. The lexing here is raw, so
+  // `void` arrives as an identifier rather than as the keyword it is.
+  if (close && close->is(clang::tok::raw_identifier) &&
+      close->getRawIdentifier() == "void") {
+    close = clang::Lexer::findNextToken(close->getLocation(), source_manager,
+                                        lang_opts);
+  }
+  if (!close || !close->is(clang::tok::r_paren)) {
+    return clang::CharSourceRange();
+  }
+  return clang::CharSourceRange::getCharRange(open->getLocation(),
+                                              close->getEndLoc());
+}
+
+// Returns the label to attach for a Clang note, which says more about the
+// diagnostic it follows rather than reporting a problem of its own.
+static auto GetNoteLabel() -> const Diagnostics::LabelBase<std::string>& {
+  CARBON_DIAGNOSTIC_LABEL(CppInteropParseNote, Info, "{0}", std::string);
+  return CppInteropParseNote;
+}
+
+// Returns the label to attach for what a note says about the source it marks,
+// as opposed to the declaration the note names.
+static auto GetNoteReasonLabel() -> const Diagnostics::LabelBase<std::string>& {
+  CARBON_DIAGNOSTIC_LABEL(CppInteropParseNoteReason, Info, "{0}", std::string);
+  return CppInteropParseNoteReason;
+}
+
+// Attaches what `info` marks: the ranges Clang would underline, which say
+// nothing the message doesn't and so only mark, and the changes it suggests,
+// which explain rather than state and so carry words.
+//
+// A range belonging to the leading diagnostic is part of the problem; one
+// belonging to a note explains it, the same as the note itself does.
+//
+// `to_loc` maps a Clang range onto whatever locations the builder takes.
+template <typename BuilderT, typename ToLocT>
+static auto AttachMarks(BuilderT& builder,
+                        const CppDiagnosticListener::Diagnostic& info,
+                        bool is_note, ToLocT to_loc) -> void {
+  for (clang::CharSourceRange range : info.ranges) {
+    builder.Attach(to_loc(range), is_note
+                                      ? Diagnostics::LabelCategory::Info
+                                      : Diagnostics::LabelCategory::Primary);
+  }
+  for (const CppDiagnosticListener::FixIt& fix_it : info.fix_its) {
+    AttachFixIt(builder, to_loc(fix_it.range), fix_it);
+  }
+}
+
+// Attaches `note_info`, a note trailing the diagnostic being built: its words
+// on the declaration it names, its reason half -- when it split into one --
+// on the source that half is about, and then whatever it marks.
+template <typename BuilderT, typename ToLocT>
+static auto AttachNote(BuilderT& builder,
+                       const CppDiagnosticListener::Diagnostic& note_info,
+                       ToLocT to_loc) -> void {
+  builder.Attach(to_loc(note_info.location), GetNoteLabel(), note_info.message);
+  if (!note_info.reason.empty()) {
+    builder.Attach(to_loc(note_info.reason_range), GetNoteReasonLabel(),
+                   note_info.reason);
+  }
+  AttachMarks(builder, note_info, /*is_note=*/true, to_loc);
+}
+
 // A listener that emits Clang diagnostics directly to a provided Carbon
 // Diagnostics::Consumer when no Carbon Context is active.
 class FallbackDiagnosticListener : public CppDiagnosticListener {
@@ -97,19 +278,16 @@ class FallbackDiagnosticListener : public CppDiagnosticListener {
     }
     ClangLocDiagnosticEmitter emitter(carbon_consumer_,
                                       diags[0].source_manager);
+    auto as_range = [](clang::CharSourceRange range) { return range; };
     for (size_t i = 0; i != diags.size(); ++i) {
       const Diagnostic& info = diags[i];
       auto builder =
           emitter.Build(info.location, GetDiagnostic(info.level), info.message);
-      builder.OverrideSnippet(info.snippet);
+      AttachMarks(builder, info, /*is_note=*/false, as_range);
       for (; i + 1 < diags.size() &&
              diags[i + 1].level == clang::DiagnosticsEngine::Note;
            ++i) {
-        const Diagnostic& note_info = diags[i + 1];
-        builder
-            .Note(note_info.location, GetDiagnostic(note_info.level),
-                  note_info.message)
-            .OverrideSnippet(note_info.snippet);
+        AttachNote(builder, diags[i + 1], as_range);
       }
       builder.Emit();
     }
@@ -142,17 +320,14 @@ class ContextDiagnosticListener : public CppDiagnosticListener {
       auto builder =
           context_->emitter().Build(SemIR::LocId(import_ir_inst_id),
                                     GetDiagnostic(info.level), info.message);
-      builder.OverrideSnippet(info.snippet);
+      auto as_loc_id = [&](clang::CharSourceRange range) {
+        return SemIR::LocId(AddImportIRInst(context_->sem_ir(), range));
+      };
+      AttachMarks(builder, info, /*is_note=*/false, as_loc_id);
       for (; i + 1 < diags.size() &&
              diags[i + 1].level == clang::DiagnosticsEngine::Note;
            ++i) {
-        const Diagnostic& note_info = diags[i + 1];
-        SemIR::ImportIRInstId note_import_ir_inst_id =
-            AddImportIRInst(context_->sem_ir(), note_info.location);
-        builder
-            .Note(SemIR::LocId(note_import_ir_inst_id),
-                  GetDiagnostic(note_info.level), note_info.message)
-            .OverrideSnippet(note_info.snippet);
+        AttachNote(builder, diags[i + 1], as_loc_id);
       }
       // TODO: This will apply all current Carbon annotation functions. We
       // should instead track how Clang's context notes and Carbon's annotation
@@ -227,63 +402,118 @@ class CarbonClangDiagnosticConsumer : public clang::DiagnosticConsumer {
     llvm::SmallString<256> message;
     info.FormatDiagnostic(message);
 
-    // Render a code snippet including any highlighted ranges and fixit hints.
-    // TODO: Also include the #include stack and macro expansion stack in the
-    // diagnostic output in some way.
-    RawStringOstream snippet_stream;
-    if (!info.hasSourceManager()) {
-      // If we don't have a source manager, this is an error from early in the
-      // frontend. Don't produce a snippet.
-      CARBON_CHECK(info.getLocation().isInvalid());
-    } else {
-      CodeContextRenderer(snippet_stream, invocation_->getLangOpts(),
-                          invocation_->getDiagnosticOpts())
-          .emitDiagnostic(
-              clang::FullSourceLoc(info.getLocation(), info.getSourceManager()),
-              diag_level, message, info.getRanges(), info.getFixItHints());
-    }
-
+    // What Clang would underline and what it suggests changing, kept as ranges
+    // rather than as text: the toolchain marks and labels them itself, so that
+    // a diagnostic from C++ reads like one from Carbon. The `#include` and
+    // macro-expansion stacks reach the location as `included from` and
+    // `expanded from macro defined at` steps, drawn like any other path a
+    // location was reached by.
     const clang::SourceManager* source_manager =
         info.hasSourceManager() ? &info.getSourceManager() : nullptr;
+
+    // Clang's token ranges reach to the start of their last token rather than
+    // past it, so they are widened here. Everything downstream then measures a
+    // range by subtracting its ends, and an empty one is an insertion point
+    // rather than a range that happens to name one token.
+    auto as_char_range = [&](clang::CharSourceRange range) {
+      if (!source_manager) {
+        return range;
+      }
+      return clang::Lexer::getAsCharRange(range, *source_manager,
+                                          invocation_->getLangOpts());
+    };
+    llvm::SmallVector<clang::CharSourceRange, 2> ranges;
+    for (clang::CharSourceRange range : info.getRanges()) {
+      ranges.push_back(as_char_range(range));
+    }
+    llvm::SmallVector<CppDiagnosticListener::FixIt, 0> fix_its;
+    for (const clang::FixItHint& hint : info.getFixItHints()) {
+      // A hint copying code from elsewhere carries it in `InsertFromRange`,
+      // which nothing here reads; presented as the removal its empty
+      // `CodeToInsert` looks like, it would instruct the opposite edit.
+      // TODO: Map such a hint to an insertion of the text `InsertFromRange`
+      // names.
+      if (hint.isNull() || hint.InsertFromRange.isValid()) {
+        continue;
+      }
+      // Widening fails for a token range ending inside a macro; Clang's own
+      // rendering drops such a fix-it too.
+      clang::CharSourceRange range = as_char_range(hint.RemoveRange);
+      if (range.isInvalid()) {
+        continue;
+      }
+      fix_its.push_back({.range = range, .text = hint.CodeToInsert});
+    }
+
+    // A Clang location names a point, and is marked as the one column Clang's
+    // own caret marks. How far it reaches is said only by a range attached
+    // alongside it, and a location can name the middle of a token, as one in
+    // a format string does.
+    clang::SourceLocation begin = info.getLocation();
+    clang::CharSourceRange location =
+        clang::CharSourceRange::getCharRange(begin, begin);
+
+    // Clang draws a range holding the caret as part of the caret's own mark
+    // rather than beside it -- the `~~~~` of a `^~~~~` -- so such a range is
+    // what the message is about rather than something else it points at. The
+    // rest stay marks of their own.
+    //
+    // TODO: Also mark where the caret is within the range, as Clang's
+    // `~~~^~~~` does. That needs the renderer to be able to mark a point
+    // inside a range it underlines.
+    if (source_manager && begin.isValid()) {
+      auto* found = llvm::find_if(ranges, [&](clang::CharSourceRange range) {
+        return Contains(*source_manager, range, begin);
+      });
+      if (found != ranges.end()) {
+        location = *found;
+        ranges.erase(found);
+      }
+    }
+
+    if (!source_manager) {
+      // Without a source manager this is an error from early in the frontend,
+      // and there is nothing to mark.
+      CARBON_CHECK(info.getLocation().isInvalid());
+      ranges.clear();
+      fix_its.clear();
+    }
+
+    // A note that says one thing about the declaration it names and another
+    // about the source it marks becomes a label on each, rather than one
+    // sentence hung off the name. Which notes are written that way is stated
+    // by `SplitsIntoCandidateAndReason` rather than guessed at from the text.
+    llvm::StringRef head = message.str();
+    llvm::StringRef reason;
+    clang::CharSourceRange reason_range;
+    if (source_manager && SplitsIntoCandidateAndReason(info.getID()) &&
+        !ranges.empty()) {
+      if (ranges.front().isInvalid()) {
+        ranges.front() = FindEmptyParameterList(
+            *source_manager, invocation_->getLangOpts(), begin);
+      }
+      auto [before, after] = head.split(": ");
+      if (ranges.front().isValid() && !after.empty()) {
+        head = before;
+        reason = after;
+        reason_range = ranges.front();
+        ranges.erase(ranges.begin());
+      }
+    }
+    llvm::erase_if(
+        ranges, [](clang::CharSourceRange range) { return range.isInvalid(); });
+
     diagnostic_infos_.push_back({.level = diag_level,
-                                 .location = info.getLocation(),
+                                 .location = location,
                                  .source_manager = source_manager,
-                                 .message = message.str().str(),
-                                 .snippet = snippet_stream.TakeStr()});
+                                 .message = head.str(),
+                                 .reason = reason.str(),
+                                 .reason_range = reason_range,
+                                 .ranges = std::move(ranges),
+                                 .fix_its = std::move(fix_its)});
   }
 
  private:
-  // A diagnostics renderer based on clang's TextDiagnostic that captures just
-  // the code context (the snippet).
-  class CodeContextRenderer : public clang::TextDiagnostic {
-   protected:
-    using TextDiagnostic::TextDiagnostic;
-
-    void emitDiagnosticMessage(
-        clang::FullSourceLoc /*loc*/, clang::PresumedLoc /*ploc*/,
-        clang::DiagnosticsEngine::Level /*level*/, llvm::StringRef /*message*/,
-        llvm::ArrayRef<clang::CharSourceRange> /*ranges*/,
-        clang::DiagOrStoredDiag /*info*/) override {}
-    void emitDiagnosticLoc(
-        clang::FullSourceLoc /*loc*/, clang::PresumedLoc /*ploc*/,
-        clang::DiagnosticsEngine::Level /*level*/,
-        llvm::ArrayRef<clang::CharSourceRange> /*ranges*/) override {}
-
-    // emitCodeContext is inherited from clang::TextDiagnostic.
-
-    void emitIncludeLocation(clang::FullSourceLoc /*loc*/,
-                             clang::PresumedLoc /*ploc*/) override {}
-    void emitImportLocation(clang::FullSourceLoc /*loc*/,
-                            clang::PresumedLoc /*ploc*/,
-                            llvm::StringRef /*module_name*/) override {}
-    void emitBuildingModuleLocation(clang::FullSourceLoc /*loc*/,
-                                    clang::PresumedLoc /*ploc*/,
-                                    llvm::StringRef /*module_name*/) override {}
-
-    // beginDiagnostic and endDiagnostic are inherited from
-    // clang::TextDiagnostic in case it wants to do any setup / teardown work.
-  };
-
   llvm::SmallVector<CppDiagnosticListener::Diagnostic> diagnostic_infos_;
   llvm::SmallVector<CppDiagnosticListener*, 2> listeners_;
   FallbackDiagnosticListener fallback_listener_;

toolchain/check/cpp/diagnostic_listener.h

@@ -10,6 +10,7 @@
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/SourceLocation.h"
 #include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/SmallVector.h"
 
 namespace clang {
 class SourceManager;
@@ -26,12 +27,34 @@ class CarbonClangDiagnosticConsumer;
 // itself on destruction.
 class CppDiagnosticListener {
  public:
+  // A change Clang suggests: what to put in `range`, which is empty for an
+  // insertion and where `text` is empty for a deletion.
+  struct FixIt {
+    clang::CharSourceRange range;
+    std::string text;
+  };
+
   struct Diagnostic {
     clang::DiagnosticsEngine::Level level;
-    clang::SourceLocation location;
+    // Where the diagnostic points: the one column its location names, or the
+    // range holding that column when one was attached, which Clang draws as
+    // the caret's own `^~~~`.
+    clang::CharSourceRange location;
     const clang::SourceManager* source_manager = nullptr;
     std::string message;
-    std::string snippet;
+    // What a note says about `reason_range` rather than about the declaration
+    // it names, split off from `message`, or empty for a note that says only
+    // one thing. Clang words a candidate note as "<which candidate>: <what is
+    // wrong with it>" and marks the source the second half is about, which is
+    // two labels here rather than one sentence hung off the name.
+    std::string reason;
+    clang::CharSourceRange reason_range;
+    // What Clang would underline, which becomes the ranges the diagnostic
+    // marks. Clang's own rendering of them is not kept: the toolchain draws
+    // these the way it draws its own. `reason_range` is not among them, since
+    // it carries words instead.
+    llvm::SmallVector<clang::CharSourceRange, 2> ranges;
+    llvm::SmallVector<FixIt, 0> fix_its;
   };
 
   explicit CppDiagnosticListener(CarbonClangDiagnosticConsumer& consumer);

toolchain/check/cpp/export.cpp

@@ -12,7 +12,10 @@
 #include "clang/Sema/EnterExpressionEvaluationContext.h"
 #include "clang/Sema/Sema.h"
 #include "llvm/Support/Casting.h"
+#include "toolchain/base/kind_switch.h"
+#include "toolchain/check/class.h"
 #include "toolchain/check/cpp/access.h"
+#include "toolchain/check/cpp/context.h"
 #include "toolchain/check/cpp/import.h"
 #include "toolchain/check/cpp/location.h"
 #include "toolchain/check/cpp/type_mapping.h"
@@ -90,13 +93,115 @@ static auto ExportClassToCppInDeclContext(Context& context,
   return record_decl;
 }
 
+auto CreateSemanticNamespace(Context& context,
+                             clang::DeclContext* parent_context,
+                             clang::IdentifierInfo* identifier)
+    -> clang::NamespaceDecl* {
+  // TODO: Provide a source location.
+  auto* namespace_decl = clang::NamespaceDecl::Create(
+      context.ast_context(), parent_context, /*Inline=*/false,
+      clang::SourceLocation(), clang::SourceLocation(), identifier,
+      /*PrevDecl=*/nullptr, /*Nested=*/false);
+  // This declaration should never be made visible to name lookups. Clear its
+  // set of associated identifier namespaces, which indicates that it should not
+  // be found by any kind of lookup.
+  namespace_decl->clearIdentifierNamespace();
+  parent_context->addHiddenDecl(namespace_decl);
+  return namespace_decl;
+}
+
+auto ExportNamespaceToCpp(Context& context, clang::DeclContext* parent_context,
+                          clang::IdentifierInfo* identifier)
+    -> clang::NamespaceDecl* {
+  auto& namespaces = context.cpp_context()->domain().semantic_namespaces();
+  auto result = namespaces.Insert(std::pair{parent_context, identifier}, [&] {
+    return CreateSemanticNamespace(context, parent_context, identifier);
+  });
+  return result.value();
+}
+
+auto ExportLookupNamespaceToCpp(Context& context, SemIR::LocId loc_id,
+                                SemIR::NameScopeId name_scope_id)
+    -> clang::DeclContext* {
+  auto& lookup_namespaces = context.cpp_context()->lookup_namespaces();
+  llvm::SmallVector<SemIR::NameScopeId> name_scope_ids_to_create;
+
+  // Collect all the enclosing Carbon namespaces that don't have a corresponding
+  // C++ namespace.
+  clang::DeclContext* decl_context = nullptr;
+  while (true) {
+    if (auto lookup = lookup_namespaces.Lookup(name_scope_id)) {
+      decl_context = lookup.value();
+      break;
+    }
+
+    const auto& name_scope = context.name_scopes().Get(name_scope_id);
+    if (name_scope.is_cpp_scope()) {
+      decl_context = GetClangDeclContextForScope(context, name_scope_id);
+      CARBON_CHECK(decl_context);
+      break;
+    }
+
+    CARBON_CHECK(context.name_scopes().InstIs<SemIR::Namespace>(name_scope_id),
+                 "Parent of namespace is non-namespace {0}",
+                 context.name_scopes().GetInstIfValid(name_scope_id).second);
+
+    name_scope_ids_to_create.push_back(name_scope_id);
+    name_scope_id = name_scope.parent_scope_id();
+    CARBON_CHECK(
+        name_scope_id.has_value(),
+        "Reached the top level without finding a scope mapped into C++");
+  }
+
+  // Build each new lookup namespace, from the outermost one inwards. The
+  // enclosing namespaces aren't necessary semantically, but they allow Clang to
+  // produce diagnostics that match the way we want the contents of namespace
+  // `Carbon` to be named.
+  for (auto name_scope_id : llvm::reverse(name_scope_ids_to_create)) {
+    const auto& name_scope = context.name_scopes().Get(name_scope_id);
+
+    auto* identifier_info =
+        GetClangIdentifierInfo(context, name_scope.name_id());
+    if (!identifier_info) {
+      // TODO: Handle keyword package names like `Cpp` and `Core`. These can
+      // be named from C++ via an alias.
+      context.TODO(loc_id, "interop with non-identifier package name");
+      return nullptr;
+    }
+
+    // TODO: Provide a source location.
+    auto* namespace_decl = clang::NamespaceDecl::Create(
+        context.ast_context(), decl_context, /*Inline=*/false,
+        clang::SourceLocation(), clang::SourceLocation(), identifier_info,
+        /*PrevDecl=*/nullptr, /*Nested=*/false);
+
+    // Add the declaration to the enclosing scope. We are providing lookup
+    // results in that enclosing scope, so don't add it to Clang's name lookup
+    // table.
+    decl_context->addHiddenDecl(namespace_decl);
+
+    // Notify Clang that we are providing lookup results in the new namespace
+    // too.
+    namespace_decl->setHasExternalVisibleStorage();
+
+    auto key = SemIR::ClangDeclKey::ForNonFunctionDecl(namespace_decl);
+    context.clang_decls().Add({.key = key, .inst_id = name_scope.inst_id()});
+    lookup_namespaces.Insert(name_scope_id, namespace_decl);
+
+    decl_context = namespace_decl;
+  }
+
+  return decl_context;
+}
+
 auto ExportNameScopeToCpp(Context& context, SemIR::LocId loc_id,
                           SemIR::NameScopeId name_scope_id)
     -> clang::DeclContext* {
   llvm::SmallVector<SemIR::NameScopeId> name_scope_ids_to_create;
 
   // Walk through the parent scopes, looking for one that's already mapped into
-  // C++. We already mapped the package scope to ::Carbon, so we must find one.
+  // C++. We already mapped the package scope to ::Carbon::<PackageName>, so we
+  // must find one.
   clang::DeclContext* decl_context = nullptr;
   while (true) {
     // If this name scope was produced by importing a C++ declaration or has
@@ -109,7 +214,14 @@ auto ExportNameScopeToCpp(Context& context, SemIR::LocId loc_id,
 
     // Otherwise, continue to the parent and create a scope for it first.
     name_scope_ids_to_create.push_back(name_scope_id);
-    name_scope_id = context.name_scopes().Get(name_scope_id).parent_scope_id();
+    const auto& name_scope = context.name_scopes().Get(name_scope_id);
+    if (name_scope.is_imported_package()) {
+      // Imported packages are exported as sub-namespaces of `::Carbon`, not of
+      // the current package's sub-namespace.
+      decl_context = context.cpp_context()->domain().carbon_namespace();
+      break;
+    }
+    name_scope_id = name_scope.parent_scope_id();
 
     // TODO: What should happen if there's an intervening function scope?
     CARBON_CHECK(
@@ -135,12 +247,8 @@ auto ExportNameScopeToCpp(Context& context, SemIR::LocId loc_id,
         return nullptr;
       }
 
-      // TODO: Provide a source location.
-      auto* namespace_decl = clang::NamespaceDecl::Create(
-          context.ast_context(), decl_context, false, clang::SourceLocation(),
-          clang::SourceLocation(), identifier_info, nullptr, false);
-      decl_context->addHiddenDecl(namespace_decl);
-      decl_context = namespace_decl;
+      decl_context =
+          ExportNamespaceToCpp(context, decl_context, identifier_info);
     } else if (auto class_type =
                    context.insts().TryGetAs<SemIR::ClassType>(const_inst_id)) {
       const auto& class_info = context.classes().Get(class_type->class_id);
@@ -151,8 +259,6 @@ auto ExportNameScopeToCpp(Context& context, SemIR::LocId loc_id,
       return nullptr;
     }
 
-    decl_context->setHasExternalVisibleStorage();
-
     auto key = SemIR::ClangDeclKey::ForNonFunctionDecl(
         cast<clang::Decl>(decl_context));
     auto clang_decl_id = context.clang_decls().Add(
@@ -254,8 +360,17 @@ static auto ExportClassSpecificToCpp(Context& context, SemIR::LocId loc_id,
     // TODO: also handle non-type args. Such args can't happen here yet, since
     // the `ExportGenericClassToCpp` call above already checks for them.
 
-    auto cpp_type = MapToCppType(
-        context, context.types().GetTypeIdForTypeInstId(specific_arg_inst_id));
+    // Extract the access type from constrained types.
+    SemIR::TypeId specific_arg_type_id = SemIR::TypeId::None;
+    if (context.types().IsConstrainedFacetType(
+            context.insts().GetAttachedType(specific_arg_inst_id))) {
+      specific_arg_type_id = GetFacetAccessType(context, specific_arg_inst_id);
+    } else {
+      specific_arg_type_id =
+          context.types().GetTypeIdForTypeInstId(specific_arg_inst_id);
+    }
+
+    auto cpp_type = MapToCppType(context, specific_arg_type_id);
     if (cpp_type.isNull()) {
       context.TODO(loc_id, "failed to map specific type arg to C++");
       return nullptr;
@@ -570,8 +685,8 @@ auto ExportAllFieldsToCpp(Context& context,
 
   for (const auto& struct_field : class_info.GetStructTypeFields(
            context.sem_ir(), class_type.specific_id)) {
-    auto class_field = LookupClassFieldByStructField(context.sem_ir(),
-                                                     class_scope, struct_field);
+    auto class_field =
+        LookupClassFieldByStructField(context, class_scope, struct_field);
     if (!class_field) {
       continue;
     }
@@ -630,9 +745,12 @@ auto ExportFieldToCpp(Context& context, SemIR::InstId field_inst_id,
       GetClassTypeInstId(context, class_type.class_id, specific_id);
   ExportAllFieldsToCpp(context, class_type_inst_id);
 
-  // Get the exported `clang::FieldDecl`.
-  if (const auto* clang_decl =
-          context.clang_decls().Lookup(field_inst_id, specific_id)) {
+  // Get the exported `clang::FieldDecl`. Exported fields are keyed by the
+  // `FieldDecl` instruction, but `field_inst_id` is an import ref if the field
+  // was imported, so look up the instruction it resolves to.
+  if (const auto* clang_decl = context.clang_decls().Lookup(
+          context.constant_values().GetConstantInstId(field_inst_id),
+          specific_id)) {
     if (!clang_decl->decl()->isInvalidDecl()) {
       return cast<clang::FieldDecl>(clang_decl->decl());
     }
@@ -643,11 +761,13 @@ auto ExportFieldToCpp(Context& context, SemIR::InstId field_inst_id,
 namespace {
 struct FunctionInfo {
   struct Param {
-    Param(Context& context, SemIR::InstId param_inst_id)
+    Param(Context& context, SemIR::InstId param_inst_id,
+          SemIR::ConstantId default_value_const_id)
         : pattern_inst_id(param_inst_id),
           type_id(ExtractScrutineeType(
               context.sem_ir(), context.insts().Get(param_inst_id).type_id())),
-          kind(GetParamPatternKind(context, param_inst_id)) {}
+          kind(GetParamPatternKind(context, param_inst_id)),
+          default_value_const_id(default_value_const_id) {}
 
     // The parameter's pattern type.
     SemIR::InstId pattern_inst_id;
@@ -657,13 +777,19 @@ struct FunctionInfo {
 
     // Kind of the parameter pattern.
     ParamPatternKind kind;
+
+    // ConstantId for a default value for the pattern, if specified, or `None`
+    // if the parameter pattern has no default value.
+    SemIR::ConstantId default_value_const_id;
   };
 
   explicit FunctionInfo(Context& context, SemIR::FunctionId function_id,
+                        SemIR::SpecificId specific_id,
                         const SemIR::Function& function,
                         clang::DeclContext* decl_context,
                         bool export_as_constructor)
       : function_id(function_id),
+        specific_id(specific_id),
         function(function),
         decl_context(decl_context),
         return_type_id(function.GetDeclaredReturnType(context.sem_ir())),
@@ -672,20 +798,30 @@ struct FunctionInfo {
         context.inst_blocks().Get(function.call_param_patterns_id);
     const auto& ranges = function.call_param_ranges;
     auto explicit_begin = ranges.explicit_begin().index;
+    auto param_start = explicit_begin;
 
     // Get the function's `self` parameter, if present. `self` is the first
     // explicit call parameter. (The lowered call parameters are leaf patterns
     // without binding names, so we rely on `self_param_id` and the positional
     // convention rather than inspecting the pattern.)
     if (function.self_param_id.has_value()) {
-      CARBON_CHECK(explicit_begin != ranges.explicit_end().index);
-      self_param = Param(context, function_params[explicit_begin]);
-      ++explicit_begin;
+      CARBON_CHECK(param_start != ranges.explicit_end().index);
+      self_param =
+          Param(context, function_params[param_start], SemIR::ConstantId::None);
+      ++param_start;
     }
 
+    auto param_patterns =
+        context.inst_blocks().GetOrEmpty(function.param_patterns_id);
     // The remaining explicit parameters are the caller-provided arguments.
-    for (auto i = explicit_begin; i != ranges.explicit_end().index; ++i) {
-      explicit_params.push_back(Param(context, function_params[i]));
+    for (auto i = param_start; i != ranges.explicit_end().index; ++i) {
+      // We only calculated default values for the explicit parameters, so
+      // we need to adjust the index within `parameter_defaults` by any elements
+      // in `function_params` that may have preceded them.
+      auto default_value_const_id =
+          ExtractParameterDefault(context, param_patterns[i - explicit_begin]);
+      explicit_params.push_back(
+          Param(context, function_params[i], default_value_const_id));
     }
   }
 
@@ -728,6 +864,7 @@ struct FunctionInfo {
   }
 
   SemIR::FunctionId function_id;
+  SemIR::SpecificId specific_id;
   const SemIR::Function& function;
 
   // Parent scope in the C++ AST where a C++ thunk for this function can
@@ -770,7 +907,8 @@ static auto MapToCppThunkParamType(Context& context, SemIR::TypeId type_id)
 // Build FunctionInfo for an export of the given Carbon function. Exports the
 // name scope if necessary. Returns `nullopt` if an error was diagnosed.
 static auto BuildFunctionInfo(Context& context, SemIR::LocId loc_id,
-                              SemIR::FunctionId callee_function_id)
+                              SemIR::FunctionId callee_function_id,
+                              SemIR::SpecificId callee_specific_id)
     -> std::optional<FunctionInfo> {
   const SemIR::Function& callee = context.functions().Get(callee_function_id);
 
@@ -818,22 +956,128 @@ static auto BuildFunctionInfo(Context& context, SemIR::LocId loc_id,
     }
   }
 
-  return FunctionInfo(context, callee_function_id, callee, decl_context,
-                      export_as_constructor);
+  return FunctionInfo(context, callee_function_id, callee_specific_id, callee,
+                      decl_context, export_as_constructor);
+}
+
+// Synthesizes a global variable with a value of `value_id` and exports it to
+// clang. To generate a unique name, `parameter_index` should be the index of
+// the parameter within the function. `pattern_id` is the parameter pattern.
+// Returns a `clang::Expr*` suitable for conversion into an argument default
+// value to provide to clang, or `nullptr` if `value_id` is None.
+static auto BuildDefaultValueExpr(Context& context, SemIR::LocId loc_id,
+                                  const FunctionInfo& target,
+                                  int32_t parameter_index,
+                                  SemIR::InstId pattern_id,
+                                  SemIR::ConstantId value_id) -> clang::Expr* {
+  if (!value_id.has_value()) {
+    return nullptr;
+  }
+
+  // Generic function decls must provide an evaluated context for synthesis
+  // of these default values.
+  CARBON_CHECK(!context.clang_sema().isUnevaluatedContext());
+
+  // Synthesize a global variable, initialized to the default value, for export
+  // to C++.
+  // TODO: making the mangler is expensive, we should cache and reuse it.
+  SemIR::Mangler mangler(context.sem_ir(), context.total_ir_count(),
+                         context.mangle_string_fingerprint());
+  auto mangled_fn_name = mangler.Mangle(target.function_id, target.specific_id);
+  llvm::SmallString<64> var_name = llvm::formatv(
+      "__carbon_default_value_{}_{}", mangled_fn_name, parameter_index);
+
+  // We should not be exporting the same variable twice.
+  CARBON_CHECK(!context.identifiers().Lookup(var_name).has_value());
+
+  auto& ident = context.ast_context().Idents.get(var_name);
+  auto name_id =
+      SemIR::NameId::ForIdentifier(context.identifiers().Add(ident.getName()));
+
+  // First we synthesize the variable definition by creating an EntityName from
+  // the generated name stored in `var_name`. We want to generate a
+  // `SemIR::VarStorage` instruction in the `InstBlockId::ExtraGlobalVariables`
+  // special block. During lowering this block is scanned for `VarStorage`
+  // instructions, and each one encountered is exported as a global variable
+  // with the associated typed storage.
+  auto pattern_type_id = context.insts().Get(pattern_id).type_id();
+
+  // TODO: we are not storing the original TypeInstIds of the parameter patterns
+  // with the function, so the location of those instructions are lost in this
+  // entity. If we ever need to know that location (e.g. for diagnostics), we
+  // will need to preserve them with functions and then pipe that through here.
+  SemIR::EntityName entity_name = {
+      .name_id = name_id,
+      .parent_scope_id = SemIR::NameScopeId::Package,
+      .type_inst_id = context.types().GetTypeInstId(pattern_type_id),
+  };
+
+  auto entity_name_id = context.entity_names().AddCanonical(entity_name);
+
+  auto binding_pattern_id = AddInstInNoBlock<SemIR::RefBindingPattern>(
+      context, loc_id,
+      {.type_id = pattern_type_id, .entity_name_id = entity_name_id});
+  context.extra_global_variables().push_back(binding_pattern_id);
+
+  auto var_pattern_id = AddInstInNoBlock(
+      context, SemIR::LocIdAndInst::RuntimeVerified(
+                   context.sem_ir(), loc_id,
+                   SemIR::VarPattern({.type_id = pattern_type_id,
+                                      .subpattern_id = binding_pattern_id})));
+  context.extra_global_variables().push_back(var_pattern_id);
+  // Note that as this is a global variable, we will not be pushing any cleanup
+  // instructions.
+  auto var_storage_id = AddInstInNoBlock(
+      context,
+      SemIR::LocIdAndInst(
+          loc_id, SemIR::VarStorage({.type_id = ExtractScrutineeType(
+                                         context.sem_ir(), pattern_type_id),
+                                     .pattern_id = var_pattern_id})));
+  context.extra_global_variables().push_back(var_storage_id);
+
+  // Add the variable initialization commands to the global initialization
+  // section so that it will have its value set before being referenced by the
+  // C++ code. Because we are exporting variables (vs. constants), we don't
+  // export the value of that default value, only the location of the global
+  // storage for that value. As a result, we must add instructions to
+  // the global initialization table to actually initialize that storage
+  // at binary startup time, so that when the C++ code dereferences
+  // it, the value is correct.
+  context.global_init().Resume();
+  auto init_id = InitializeExisting(
+      context, loc_id, var_storage_id,
+      context.constant_values().GetInstId(value_id), /*for_return=*/false);
+  AddInst<SemIR::Assign>(context, loc_id,
+                         {.lhs_id = var_storage_id, .rhs_id = init_id});
+  context.global_init().Suspend();
+
+  auto* var_decl =
+      ExportVarToCpp(context, var_storage_id,
+                     context.insts().GetAs<SemIR::VarStorage>(var_storage_id));
+  auto decl_ref_expr = context.clang_sema().BuildDeclarationNameExpr(
+      clang::CXXScopeSpec(),
+      clang::DeclarationNameInfo(var_decl->getDeclName(),
+                                 clang::SourceLocation()),
+      var_decl);
+  return decl_ref_expr.get();
 }
 
 // Create a `clang::FunctionDecl` with the given parameter types and
 // return type.
 //
 // The function's name will match the one referenced by `function_name_id`,
-// and the function will be added to the given `decl_context`.
-static auto BuildCppFunctionDecl(Context& context,
-                                 clang::DeclContext* decl_context,
-                                 SemIR::LocId loc_id,
-                                 clang::DeclarationName declaration_name,
-                                 clang::ArrayRef<clang::QualType> param_types,
-                                 clang::QualType return_type,
-                                 bool export_as_constructor) {
+// and the function will be added to the given `decl_context`. If provided,
+// `default_value_exprs` must be index-aligned with `param_types`, and at every
+// index must have either a `clang::Expr*` suitable for use as a default value
+// for that parameter, or `nullptr` if no default value is specified.
+static auto BuildCppFunctionDecl(
+    Context& context, clang::DeclContext* decl_context, SemIR::LocId loc_id,
+    clang::DeclarationName declaration_name,
+    clang::ArrayRef<clang::QualType> param_types,
+    std::optional<clang::ArrayRef<clang::Expr*>> default_value_exprs,
+    clang::QualType return_type, bool export_as_constructor) {
+  CARBON_CHECK(!default_value_exprs.has_value() ||
+               default_value_exprs->size() == param_types.size());
   auto clang_loc = GetCppLocation(context, loc_id);
 
   auto cpp_function_type = context.ast_context().getFunctionType(
@@ -869,6 +1113,15 @@ static auto BuildCppFunctionDecl(Context& context,
         context.ast_context(), function_decl, /*StartLoc=*/clang_loc,
         /*IdLoc=*/clang_loc, /*Id=*/nullptr, type, param_tinfo, clang::SC_None,
         /*DefArg=*/nullptr);
+    auto* default_value_expr =
+        default_value_exprs ? default_value_exprs.value()[i] : nullptr;
+    if (default_value_expr) {
+      auto converted_result = context.clang_sema().ConvertParamDefaultArgument(
+          param, default_value_expr, clang_loc);
+      CARBON_CHECK(!converted_result.isInvalid());
+      context.clang_sema().SetParamDefaultArgument(
+          param, converted_result.get(), clang_loc);
+    }
     param_var_decls.push_back(param);
   }
   function_decl->setParams(param_var_decls);
@@ -921,7 +1174,8 @@ static auto BuildCppFunctionDeclForNonGenericCarbonFn(Context& context,
                            : context.ast_context().getTranslationUnitDecl();
   auto* function_decl = BuildCppFunctionDecl(
       context, decl_context, loc_id, target.GetCppName(context),
-      cpp_param_types, cpp_return_type, target.export_as_constructor);
+      cpp_param_types, std::nullopt, cpp_return_type,
+      target.export_as_constructor);
 
   // Mangle the function name and attach it to the `FunctionDecl`.
   SemIR::Mangler m(context.sem_ir(), context.total_ir_count(),
@@ -952,6 +1206,7 @@ static auto BuildCppFunctionDeclForGenericCarbonFn(Context& context,
   // irrelevant, and the parameter should instead map to something that will
   // guide C++ template argument deduction into doing the right thing.
   llvm::SmallVector<clang::QualType> cpp_param_types;
+  llvm::SmallVector<clang::Expr*> cpp_default_values;
   if (callee.self_param) {
     auto cpp_type = MapToCppThunkParamType(context, callee.self_param->type_id);
     if (cpp_type.isNull()) {
@@ -959,14 +1214,18 @@ static auto BuildCppFunctionDeclForGenericCarbonFn(Context& context,
       return nullptr;
     }
     cpp_param_types.push_back(cpp_type);
+    cpp_default_values.push_back(nullptr);
   }
-  for (auto param : callee.explicit_params) {
+  for (auto [i, param] : llvm::enumerate(callee.explicit_params)) {
     auto cpp_type = MapToCppThunkParamType(context, param.type_id);
     if (cpp_type.isNull()) {
       context.TODO(loc_id, "failed to map Carbon type to C++");
       return nullptr;
     }
     cpp_param_types.push_back(cpp_type);
+    cpp_default_values.push_back(
+        BuildDefaultValueExpr(context, loc_id, callee, i, param.pattern_inst_id,
+                              param.default_value_const_id));
   }
 
   clang::QualType cpp_return_type = context.ast_context().VoidTy;
@@ -985,7 +1244,8 @@ static auto BuildCppFunctionDeclForGenericCarbonFn(Context& context,
                            : context.ast_context().getTranslationUnitDecl();
   return BuildCppFunctionDecl(context, decl_context, loc_id,
                               callee.GetCppName(context), cpp_param_types,
-                              cpp_return_type, callee.export_as_constructor);
+                              cpp_default_values, cpp_return_type,
+                              callee.export_as_constructor);
 }
 
 // Returns whether the given Carbon parameter should be passed as a C++ const
@@ -1071,6 +1331,30 @@ static auto BuildCppToCarbonThunkFunctionType(Context& context,
       ->getAs<clang::FunctionProtoType>();
 }
 
+// Look up the `SemIR::Class` associated with a `clang::Decl`. The mapping is
+// required to exist.
+static auto GetClassInfoForClangDecl(Context& context, clang::Decl* decl)
+    -> const SemIR::Class& {
+  auto clang_decl_id = context.clang_decls().LookupId(
+      SemIR::ClangDeclKey::ForNonFunctionDecl(decl));
+  const auto& clang_decl = context.clang_decls().Get(clang_decl_id);
+  auto class_id = SemIR::ClassId::None;
+  // Regular classes are mapped in `clang_decls` to a `ClassDecl`
+  // inst. Specific classes are mapped to a `ClassType` plus a specific.
+  if (auto class_decl =
+          context.insts().TryGetAs<SemIR::ClassDecl>(clang_decl.inst_id)) {
+    class_id = class_decl->class_id;
+
+  } else if (auto class_type = context.insts().TryGetAs<SemIR::ClassType>(
+                 clang_decl.inst_id)) {
+    class_id = class_type->class_id;
+
+  } else {
+    CARBON_FATAL("failed to map clang decl to a Carbon class");
+  }
+  return context.classes().Get(class_id);
+}
+
 // Create the declaration of the C++ thunk.
 static auto BuildCppToCarbonThunkDecl(Context& context, SemIR::LocId loc_id,
                                       const FunctionInfo& target,
@@ -1129,8 +1413,13 @@ static auto BuildCppToCarbonThunkDecl(Context& context, SemIR::LocId loc_id,
           tinfo, target.GetStorageClass(), uses_fp_intrin, inline_specified,
           constexpr_kind, clang_loc, trailing_requires_clause);
     }
-    // TODO: Map Carbon access to C++ access.
-    thunk_function_decl->setAccess(clang::AS_public);
+
+    // Map Carbon access to C++ access.
+    const auto& class_info = GetClassInfoForClangDecl(context, parent_class);
+    const auto& class_scope = context.name_scopes().Get(class_info.scope_id);
+    SetCppClassMemberAccess(class_scope, target.function.name_id,
+                            thunk_function_decl);
+
     // Carbon overriders are non-virtual in C++; only the corresponding thunk is
     // virtual.
     thunk_function_decl->setVirtualAsWritten(
@@ -1151,10 +1440,27 @@ static auto BuildCppToCarbonThunkDecl(Context& context, SemIR::LocId loc_id,
 
   llvm::SmallVector<clang::ParmVarDecl*> param_var_decls;
   for (auto [i, type] : llvm::enumerate(thunk_function_type->param_types())) {
+    const auto& param = target.explicit_params[i];
     clang::ParmVarDecl* thunk_param = clang::ParmVarDecl::Create(
         ast_context, thunk_function_decl, /*StartLoc=*/clang_loc,
         /*IdLoc=*/clang_loc, /*Id=*/nullptr, type,
         /*TInfo=*/nullptr, clang::SC_None, /*DefArg=*/nullptr);
+    if (param.default_value_const_id.has_value()) {
+      clang::Sema& sema = context.clang_sema();
+      clang::Sema::ContextRAII context_raii(sema, thunk_function_decl);
+      // Ensure that the evaluation context is not `Unevaluated`, as that
+      // would cause code generation to fail.
+      clang::EnterExpressionEvaluationContext evaluated(
+          sema, clang::Sema::ExpressionEvaluationContext::PotentiallyEvaluated);
+      auto* default_value_expr = BuildDefaultValueExpr(
+          context, loc_id, target, i, param.pattern_inst_id,
+          param.default_value_const_id);
+      auto converted_result = context.clang_sema().ConvertParamDefaultArgument(
+          thunk_param, default_value_expr, clang_loc);
+      CARBON_CHECK(!converted_result.isInvalid());
+      context.clang_sema().SetParamDefaultArgument(
+          thunk_param, converted_result.get(), clang_loc);
+    }
     param_var_decls.push_back(thunk_param);
   }
   thunk_function_decl->setParams(param_var_decls);
@@ -1334,8 +1640,10 @@ static auto BuildCarbonToCarbonThunk(Context& context, SemIR::LocId loc_id,
   // If this thunk will be exposed as a C++ constructor, we put the output
   // parameter first to match the Itanium constructor ABI.
   //
-  // TODO: use `clang::CodeGen::CGCXXABI::HasThisReturn` to determine if the
-  // constructor's `this` should be a return value instead of an output param.
+  // Some C++ ABIs also return `this` from constructors. We don't model that
+  // here; instead, lowering defines the thunk in the LLVM function Clang
+  // creates for its C++ declaration, and returns `this` if Clang's signature
+  // does. See `FileContext::GetCppThunkLLVMFunction`.
   if (target.export_as_constructor) {
     CARBON_CHECK(target.return_type_id != SemIR::TypeId::None);
     std::rotate(thunk_param_type_ids.begin(), thunk_param_type_ids.end() - 1,
@@ -1353,6 +1661,9 @@ static auto BuildCarbonToCarbonThunk(Context& context, SemIR::LocId loc_id,
            .param_type_ids = thunk_param_type_ids,
            .param_kinds = thunk_param_kinds})
           .second;
+  context.functions()
+      .Get(carbon_thunk_function_id)
+      .SetCppThunk(target.function.first_decl_id());
 
   BuildThunkDefinitionForExport(
       context, carbon_thunk_function_id, target.function_id,
@@ -1360,6 +1671,7 @@ static auto BuildCarbonToCarbonThunk(Context& context, SemIR::LocId loc_id,
       target.function.first_decl_id(), target.export_as_constructor);
 
   return FunctionInfo(context, carbon_thunk_function_id,
+                      SemIR::SpecificId::None,
                       context.functions().Get(carbon_thunk_function_id),
                       target.decl_context, target.export_as_constructor);
 }
@@ -1376,7 +1688,7 @@ auto ExportVirtualFunctionDeclToCpp(Context& context, SemIR::LocId loc_id,
                                     clang::CXXRecordDecl* parent,
                                     SemIR::FunctionId function_id)
     -> clang::CXXMethodDecl* {
-  FunctionInfo target(context, function_id,
+  FunctionInfo target(context, function_id, SemIR::SpecificId::None,
                       context.functions().Get(function_id), parent,
                       /*export_as_constructor=*/false);
   return cast_or_null<clang::CXXMethodDecl>(
@@ -1399,6 +1711,20 @@ static auto BuildCppToCarbonThunk(Context& context, SemIR::LocId loc_id,
     return;
   }
 
+  // Map the Carbon thunk to its C++ declaration, so that lowering can define
+  // the thunk in the same LLVM function that Clang uses to call it.
+  SemIR::ClangDeclSignature carbon_function_signature{
+      .kind = SemIR::ClangDeclSignature::Normal,
+      .num_params = static_cast<int32_t>(carbon_function_decl->getNumParams())};
+  carbon_function_signature.passing_modes.assign(
+      carbon_function_signature.num_params,
+      SemIR::ClangDeclSignature::PassingMode::ByValue);
+  context.clang_decls().Add(
+      {.key = SemIR::ClangDeclKey::ForFunctionDecl(
+           carbon_function_decl, context.clang_decl_signatures().Add(
+                                     std::move(carbon_function_signature))),
+       .inst_id = carbon_thunk_target.function.first_decl_id()});
+
   // Build the thunk function body.
   clang::Sema& sema = context.clang_sema();
   clang::Sema::ContextRAII context_raii(sema, thunk_function_decl);
@@ -1420,7 +1746,8 @@ auto DefineExportedVirtualFunction(Context& context, SemIR::LocId loc_id,
                                    SemIR::FunctionId callee_function_id,
                                    clang::CXXMethodDecl* method_decl) -> void {
   const SemIR::Function& callee = context.functions().Get(callee_function_id);
-  FunctionInfo target_function_info(context, callee_function_id, callee,
+  FunctionInfo target_function_info(context, callee_function_id,
+                                    SemIR::SpecificId::None, callee,
                                     method_decl->getDeclContext(),
                                     /*export_as_constructor=*/false);
   BuildCppToCarbonThunk(context, loc_id, target_function_info, method_decl, "");
@@ -1431,9 +1758,9 @@ auto DefineExportedVirtualFunction(Context& context, SemIR::LocId loc_id,
 // name.
 //
 // Returns nullptr if an error occurs.
-auto ExportNonGenericFunctionToCpp(Context& context, SemIR::LocId loc_id,
-                                   const FunctionInfo& target,
-                                   std::string_view extra_name = "")
+static auto ExportNonGenericFunctionToCpp(Context& context, SemIR::LocId loc_id,
+                                          const FunctionInfo& target,
+                                          std::string_view extra_name = "")
     -> clang::FunctionDecl* {
   auto* thunk_function_decl =
       ExportNonGenericFunctionDeclToCpp(context, loc_id, target);
@@ -1462,20 +1789,25 @@ auto ExportFunctionSpecializationToCpp(
   auto target_function =
       context.functions().Get(target_function_decl.function_id);
 
-  auto* decl_context = function_template_decl->getDeclContext();
-  FunctionInfo target(context, target_function_decl.function_id,
-                      target_function, decl_context,
-                      llvm::isa<clang::CXXConstructorDecl>(
-                          function_template_decl->getTemplatedDecl()));
-  SemIR::LocId loc_id(target.function.first_decl_id());
+  SemIR::LocId loc_id(target_function.first_decl_id());
 
   // Create a specific, and use that to convert return type and
   // parameters with symbolic types to concrete types.
   auto specific_id = MakeSpecificForTemplateArgs(
-      context, loc_id, target.function.generic_id, template_args);
+      context, loc_id, target_function.generic_id, template_args);
   if (specific_id == SemIR::SpecificId::None) {
     return false;
   }
+  auto* decl_context = function_template_decl->getDeclContext();
+  FunctionInfo target(context, target_function_decl.function_id, specific_id,
+                      target_function, decl_context,
+                      llvm::isa<clang::CXXConstructorDecl>(
+                          function_template_decl->getTemplatedDecl()));
+
+  if (!ResolveReturnTypeInSpecific(context, loc_id, target.function,
+                                   specific_id)) {
+    return false;
+  }
   // This name is appended to the thunk name to disambiguate between
   // specializations.
   SemIR::Mangler m(context.sem_ir(), context.total_ir_count(),
@@ -1539,7 +1871,16 @@ static auto ExportGenericFunctionToCpp(Context& context, SemIR::LocId loc_id,
 static auto ExportFunctionToCpp(Context& context, SemIR::LocId loc_id,
                                 SemIR::FunctionId callee_function_id)
     -> clang::NamedDecl* {
-  auto target = BuildFunctionInfo(context, loc_id, callee_function_id);
+  if (context.functions()
+          .Get(callee_function_id)
+          .has_pending_deduced_return_type()) {
+    context.TODO(loc_id,
+                 "export of function before its return type is deduced");
+    return nullptr;
+  }
+
+  auto target = BuildFunctionInfo(context, loc_id, callee_function_id,
+                                  SemIR::SpecificId::None);
   if (!target) {
     return nullptr;
   }
@@ -1640,12 +1981,13 @@ auto ExportFunctionToCppPointerConversion(
           ExportedFunctionPtrTypeMismatch, Error,
           "can't convert exported {0:member |}function pointer to `{1}`",
           Diagnostics::BoolAsSelect, CppType);
-      CARBON_DIAGNOSTIC(ExportedFromFunction, Note,
-                        "function exported with pointer type `{0}`", CppType);
+      CARBON_DIAGNOSTIC_LABEL(ExportedFromFunction, Info,
+                              "function exported with pointer type `{0}`",
+                              CppType);
       context.emitter()
           .Build(src_id, ExportedFunctionPtrTypeMismatch,
                  src_ptr_type->isMemberFunctionPointerType(), dest_ptr_type)
-          .Note(function.first_decl_id(), ExportedFromFunction, src_ptr_type)
+          .Attach(function.first_decl_id(), ExportedFromFunction, src_ptr_type)
           .Emit();
     }
     return false;
@@ -1733,7 +2075,7 @@ auto ExportDestructorToCpp(Context& context, const SemIR::Class& class_info,
   // TODO: Once we support exporting specific classes, export the specific
   // destructor here rather than a generic one.
   auto thunk_function_id = BuildDestroyThunk(context, loc_id, class_info);
-  FunctionInfo thunk_target(context, thunk_function_id,
+  FunctionInfo thunk_target(context, thunk_function_id, SemIR::SpecificId::None,
                             context.functions().Get(thunk_function_id),
                             record_decl, /*export_as_constructor=*/false);
   auto* cpp_function_decl =

toolchain/check/cpp/export.h

@@ -17,6 +17,33 @@ class CXXRecordDecl;
 
 namespace Carbon::Check {
 
+// Creates a namespace to use as a semantic parent for exported C++ entities.
+// This is used for the root `Carbon` namespace, the package namespaces within
+// it, and any nested namespaces within those packages. Does not register the
+// namespace as corresponding to any particular Carbon entity.
+auto CreateSemanticNamespace(Context& context,
+                             clang::DeclContext* parent_context,
+                             clang::IdentifierInfo* identifier)
+    -> clang::NamespaceDecl*;
+
+// Exports a Carbon namespace into C++ as a semantic sub-namespace of
+// `parent_context` (used as the parent `DeclContext` of exported Carbon
+// entities), or returns the existing namespace if already exported in this C++
+// domain.
+auto ExportNamespaceToCpp(Context& context, clang::DeclContext* parent_context,
+                          clang::IdentifierInfo* identifier)
+    -> clang::NamespaceDecl*;
+
+// Exports a Carbon namespace into C++ as a per-file lookup-only
+// `NamespaceDecl`, or returns the C++ declaration context if it was imported
+// from C++. Lookup-only namespaces are only used for C++ name lookup in the
+// current Carbon file, and are never the parent `DeclContext` of anything
+// other than child lookup-only namespaces, so that references to Carbon
+// entities are mangled consistently.
+auto ExportLookupNamespaceToCpp(Context& context, SemIR::LocId loc_id,
+                                SemIR::NameScopeId name_scope_id)
+    -> clang::DeclContext*;
+
 // Exports a Carbon name scope into C++ as a namespace or class, or returns the
 // C++ namespace or class declaration that it was imported from.
 //

toolchain/check/cpp/generate_ast.cpp

@@ -9,10 +9,12 @@
 
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/DeclContextInternals.h"
 #include "clang/AST/Mangle.h"
 #include "clang/Basic/DiagnosticParse.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/Module.h"
+#include "clang/Basic/TargetOptions.h"
 #include "clang/CodeGen/ModuleBuilder.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
@@ -102,6 +104,7 @@ class ShallowCopyCompilerInvocation : public clang::CompilerInvocation {
     // Make a deep copy of options that we modify.
     FrontendOpts = std::make_shared<clang::FrontendOptions>(*FrontendOpts);
     PPOpts = std::make_shared<clang::PreprocessorOptions>(*PPOpts);
+    TargetOpts = std::make_shared<clang::TargetOptions>(*TargetOpts);
   }
 };
 
@@ -213,8 +216,8 @@ auto CarbonExternalASTSource::MapInstIdToClangDeclOrType(LookupResult lookup)
   CARBON_KIND_SWITCH(target_inst) {
     case CARBON_KIND(SemIR::Namespace namespace_info): {
       auto* decl_context =
-          ExportNameScopeToCpp(*context_, SemIR::LocId(target_inst_id),
-                               namespace_info.name_scope_id);
+          ExportLookupNamespaceToCpp(*context_, SemIR::LocId(target_inst_id),
+                                     namespace_info.name_scope_id);
       if (!decl_context) {
         return nullptr;
       }
@@ -257,57 +260,82 @@ auto CarbonExternalASTSource::BuildCarbonNamespace() -> void {
   auto& ast_context = context_->ast_context();
   auto* identifier = &ast_context.Idents.get(carbon_namespace_name);
 
-  auto* decl_context = ast_context.getTranslationUnitDecl();
+  auto* tu_decl = ast_context.getTranslationUnitDecl();
 
-  // Check if it already exists.
-  clang::NamespaceDecl* carbon_cpp_namespace = nullptr;
-  auto lookup_result = decl_context->lookup(identifier);
-  if (!lookup_result.empty()) {
-    carbon_cpp_namespace = cast<clang::NamespaceDecl>(lookup_result.front());
-  } else {
-    // Create it if it doesn't exist.
-    carbon_cpp_namespace = clang::NamespaceDecl::Create(
-        ast_context, decl_context, /*Inline=*/false, clang::SourceLocation(),
-        clang::SourceLocation(), identifier, /*PrevDecl=*/nullptr,
-        /*Nested=*/false);
-    decl_context->addDecl(carbon_cpp_namespace);
-
-    // We provide custom lookup results within this namespace.
-    carbon_cpp_namespace->setHasExternalVisibleStorage();
+  // Create or find the semantic `::Carbon` namespace for this C++ domain, which
+  // is used as the parent `DeclContext` for package sub-namespaces.
+  auto& domain = context_->cpp_context()->domain();
+  auto* semantic_carbon_namespace = domain.carbon_namespace();
+  if (!semantic_carbon_namespace) {
+    semantic_carbon_namespace =
+        CreateSemanticNamespace(*context_, tu_decl, identifier);
+    domain.set_carbon_namespace(semantic_carbon_namespace);
   }
 
-  // Register this file's package scope as corresponding to the `Carbon`
-  // namespace in C++.
-  // TODO: For mangling purposes, include the package as a sub-namespace.
-  auto key = SemIR::ClangDeclKey::ForNonFunctionDecl(carbon_cpp_namespace);
-  auto clang_decl_id = context_->clang_decls().Add(
-      {.key = key, .inst_id = SemIR::Namespace::PackageInstId});
-  context_->name_scopes()
-      .Get(SemIR::NameScopeId::Package)
-      .set_clang_decl_context_id(clang_decl_id, /*is_cpp_scope=*/false);
+  // Create or find the semantic sub-namespace for this file's package, which is
+  // used as the parent `DeclContext` for package-scope entities so that the
+  // package name is included in C++ mangled names.
+  auto& package_scope =
+      context_->name_scopes().Get(SemIR::NameScopeId::Package);
+  auto package_name = context_->names().GetIRBaseName(package_scope.name_id());
+  auto* package_identifier = &ast_context.Idents.get(package_name);
+  auto* package_cpp_namespace = ExportNamespaceToCpp(
+      *context_, semantic_carbon_namespace, package_identifier);
+
+  auto package_key =
+      SemIR::ClangDeclKey::ForNonFunctionDecl(package_cpp_namespace);
+  auto package_clang_decl_id = context_->clang_decls().Add(
+      {.key = package_key, .inst_id = SemIR::Namespace::PackageInstId});
+  package_scope.set_clang_decl_context_id(package_clang_decl_id,
+                                          /*is_cpp_scope=*/false);
+
+  // Create a per-file lookup-only `::Carbon` namespace, mapped to the package
+  // scope so that C++ lookups in `Carbon` find package-scope entities directly
+  // (and do not find the current package name), and are isolated to this file.
+  auto* lookup_carbon_namespace = clang::NamespaceDecl::Create(
+      ast_context, tu_decl, /*Inline=*/false, clang::SourceLocation(),
+      clang::SourceLocation(), identifier, /*PrevDecl=*/nullptr,
+      /*Nested=*/false);
+  tu_decl->addDecl(lookup_carbon_namespace);
+  lookup_carbon_namespace->setHasExternalVisibleStorage();
+
+  context_->cpp_context()->set_carbon_lookup_namespace(lookup_carbon_namespace);
+  context_->cpp_context()->lookup_namespaces().Insert(
+      SemIR::NameScopeId::Package, lookup_carbon_namespace);
+
+  auto carbon_key =
+      SemIR::ClangDeclKey::ForNonFunctionDecl(lookup_carbon_namespace);
+  context_->clang_decls().Add(
+      {.key = carbon_key, .inst_id = SemIR::Namespace::PackageInstId});
 }
 
 auto CarbonExternalASTSource::FindExternalVisibleDeclsByName(
-    const clang::DeclContext* decl_context, clang::DeclarationName decl_name,
-    const clang::DeclContext* /*OriginalDC*/) -> bool {
-  // Find the Carbon declaration corresponding to this Clang declaration.
-  auto* decl = cast<clang::Decl>(
-      const_cast<clang::DeclContext*>(decl_context->getPrimaryContext()));
-  if (isa<clang::FunctionDecl>(decl)) {
-    // Functions don't meaningfully have visible decls, but bail out early since
-    // we can't form a `ClangDeclKey` for a function in the abstract.
+    const clang::DeclContext* const_decl_context,
+    clang::DeclarationName decl_name, const clang::DeclContext* /*OriginalDC*/)
+    -> bool {
+  // Clang expects us to modify the DeclContext but passes it as `const`.
+  // TODO: Fix clang to pass in a mutable DeclContext.
+  auto* decl_context = const_cast<clang::DeclContext*>(const_decl_context);
+
+  auto clang_decl = GetCarbonOwnedDeclAsLookupContext(decl_context);
+  if (!clang_decl) {
     return false;
   }
-  auto key = SemIR::ClangDeclKey::ForNonFunctionDecl(decl);
-  auto decl_id = context_->clang_decls().LookupId(key);
-  if (!decl_id.has_value()) {
-    return false;
-  }
-  auto clang_decl = context_->clang_decls().Get(decl_id);
-  if (clang_decl.is_imported) {
-    // This is imported from C++, presumably from a Clang AST file, so it's not
-    // our responsibility to provide its name lookup results.
-    return false;
+
+  auto* primary_context = decl_context->getPrimaryContext();
+  if (auto* lookup_map = primary_context->getLookupPtr()) {
+    if (auto it = lookup_map->find(decl_name);
+        it != lookup_map->end() && !it->second.isNull()) {
+      // We already exported declarations for this name, and may have been
+      // asked again after `completeVisibleDeclsMap` re-enabled external
+      // visible storage. Mark external lookup complete without re-exporting.
+      //
+      // Note that this only sets that there are no external decls from an AST
+      // file, despite its name. External visible decls already added by other
+      // AST sources, such as this one, are retained.
+      SetNoExternalVisibleDeclsForName(primary_context, decl_name);
+      return true;
+    }
   }
 
   llvm::SmallVector<Check::LookupScope> lookup_scopes;
@@ -316,7 +344,7 @@ auto CarbonExternalASTSource::FindExternalVisibleDeclsByName(
   // here - completeness should've been checked by clang before this point.
   if (!AppendLookupScopesForConstant(
           *context_, SemIR::LocId::None,
-          context_->constant_values().Get(clang_decl.inst_id),
+          context_->constant_values().Get(clang_decl->inst_id),
           SemIR::ConstantId::None, /*extended_scope=*/false, &lookup_scopes)) {
     return false;
   }
@@ -335,25 +363,47 @@ auto CarbonExternalASTSource::FindExternalVisibleDeclsByName(
       break;
     }
     default:
+      SetNoExternalVisibleDeclsForName(primary_context, decl_name);
       return false;
   }
 
   auto name_id = AddIdentifierName(*context_, identifier->getName());
 
-  // `required=false` so Carbon doesn't diagnose a failure, let Clang diagnose
-  // it or even SFINAE.
-  LookupResult result =
-      LookupQualifiedName(*context_, SemIR::LocId::None, name_id, lookup_scopes,
-                          /*required=*/false);
+  LookupResult result{
+      .specific_id = SemIR::SpecificId::None,
+      .scope_result = SemIR::ScopeLookupResult::MakeNotFound(),
+  };
+  for (auto lookup_scope : lookup_scopes) {
+    auto scope_id = lookup_scope.name_scope_id;
+    result.scope_result =
+        LookupNameInExactScope(*context_, GetCurrentCppLocId(), name_id,
+                               scope_id, context_->name_scopes().Get(scope_id));
+    result.specific_id = lookup_scope.specific_id;
+    // TODO: If the name is found in multiple scopes, it should be treated as
+    // ambiguous, possibly by adding all the results to the name lookup table
+    // so Clang reports the ambiguity.
+    if (result.scope_result.is_found()) {
+      break;
+    }
+  }
+
   if (!result.scope_result.is_found()) {
+    // Avoid caching a negative lookup, as a declaration with this name may be
+    // added in Carbon later.
+    if (auto* lookup_map = primary_context->getLookupPtr()) {
+      if (auto it = lookup_map->find(decl_name);
+          it != lookup_map->end() && it->second.isNull()) {
+        lookup_map->erase(it);
+      }
+    }
     return false;
   }
 
   // Map the found Carbon entity to a Clang NamedDecl.
   CARBON_KIND_SWITCH(MapInstIdToClangDeclOrType(result)) {
-    case CARBON_KIND(clang::NamedDecl* clang_decl): {
-      if (clang_decl) {
-        SetExternalVisibleDeclsForName(decl_context, decl_name, {clang_decl});
+    case CARBON_KIND(clang::NamedDecl* mapped_decl): {
+      if (mapped_decl) {
+        SetExternalVisibleDeclsForName(decl_context, decl_name, {mapped_decl});
         return true;
       } else {
         SetNoExternalVisibleDeclsForName(decl_context, decl_name);
@@ -362,15 +412,31 @@ auto CarbonExternalASTSource::FindExternalVisibleDeclsByName(
     }
 
     case CARBON_KIND(clang::QualType type): {
-      // Create a typedef declaration to model the type result.
+      // Create a typedef declaration to model the type result. We create the
+      // typedef declaration in whatever name context the lookup was performed
+      // in, since we don't have a Carbon declaration to take a name scope from.
+      // But if that context was a namespace, map from the lookup-only namespace
+      // to the corresponding semantic namespace so that lookup-only
+      // NamespaceDecls are never the parent DeclContext of exported entities.
       // TODO: If the type is a tag type that was declared with this name in
       // this context, use the tag decl directly.
+      auto* semantic_decl_context = decl_context;
+      if (auto namespace_info = context_->insts().TryGetAs<SemIR::Namespace>(
+              clang_decl->inst_id)) {
+        semantic_decl_context = ExportNameScopeToCpp(
+            *context_, SemIR::LocId(result.scope_result.target_inst_id()),
+            namespace_info->name_scope_id);
+        if (!semantic_decl_context) {
+          SetNoExternalVisibleDeclsForName(decl_context, decl_name);
+          return false;
+        }
+      }
       auto& ast_context = context_->ast_context();
       auto loc = GetCppLocation(
           *context_, SemIR::LocId(result.scope_result.target_inst_id()));
       auto* typedef_decl = clang::TypedefDecl::Create(
-          ast_context, const_cast<clang::DeclContext*>(decl_context), loc, loc,
-          identifier, ast_context.getTrivialTypeSourceInfo(type, loc));
+          ast_context, semantic_decl_context, loc, loc, identifier,
+          ast_context.getTrivialTypeSourceInfo(type, loc));
       if (isa<clang::CXXRecordDecl>(decl_context)) {
         typedef_decl->setAccess(
             MapToCppAccess(result.scope_result.access_kind()));
@@ -509,10 +575,37 @@ auto CarbonExternalASTSource::CompleteType(clang::TagDecl* tag_decl) -> void {
   class_decl->completeDefinition();
 
   // Now the class is complete, we can define the virtual function thunks.
-  for (auto virtual_fn : pending_virtual_functions) {
-    DefineExportedVirtualFunction(*context_, virtual_fn.loc_id,
-                                  virtual_fn.function_id,
-                                  virtual_fn.method_decl);
+  if (llvm::isa<clang::ClassTemplateSpecializationDecl>(class_decl) &&
+      !pending_virtual_functions.empty()) {
+    context_->TODO(class_info.first_decl_id(),
+                   "generic class overriding methods from a C++ virtual class");
+  } else {
+    for (auto virtual_fn : pending_virtual_functions) {
+      DefineExportedVirtualFunction(*context_, virtual_fn.loc_id,
+                                    virtual_fn.function_id,
+                                    virtual_fn.method_decl);
+    }
+  }
+
+  // Export friends.
+  for (auto friend_scope : class_info.friend_scopes.entries()) {
+    auto friend_inst_id = context_->constant_values().GetInstId(friend_scope);
+    if (auto friend_class_type =
+            context_->insts().TryGetAs<SemIR::ClassType>(friend_inst_id)) {
+      const auto& friend_class_info =
+          context_->classes().Get(friend_class_type->class_id);
+      if (const auto* friend_clang_decl = context_->clang_decls().Lookup(
+              friend_class_info.first_decl_id())) {
+        auto* friend_tag_decl =
+            llvm::cast<clang::TagDecl>(friend_clang_decl->decl());
+        auto* friend_tsi = context_->ast_context().getTrivialTypeSourceInfo(
+            context_->ast_context().getCanonicalTagType(friend_tag_decl));
+        clang::FriendDecl::Create(context_->ast_context(), class_decl,
+                                  /*L=*/clang::SourceLocation(), friend_tsi,
+                                  /*FL=*/clang::SourceLocation());
+      }
+    }
+    // TODO: support other kinds of friends.
   }
 }
 
@@ -1038,13 +1131,42 @@ auto InjectAstFromInlineCode(Context& context, SemIR::LocId loc_id,
   ParseInjectedTokens(*cpp_context);
 }
 
-auto FinishAst(Context& context) -> void {
+auto FinalizeCppAstFragment(Context& context) -> void {
+  // `FinalizeCppAst` operates very close to where AST fragment is
+  // nullified. Builds may call this function with a null AST fragment,
+  // so we exit early in the case where it's null.
+  //
+  // TODO: investigate if we can avoid AST fragment resolution when the
+  // fragment is null.
+  if (!context.cpp_context()) {
+    return;
+  }
+
+  // Finalize the per-Context AST fragment. The final ActOnEndOfTranslationUnit
+  // call for the CppDomain is performed in FinalizeCppDomain once all files
+  // sharing the domain have been checked.
+  context.cpp_context()->sema().ActOnEndOfTranslationUnitFragment(
+      clang::TUFragmentKind::Normal);
+  FlushDiagnosticConsumer(
+      *context.cpp_context()->sema().getDiagnostics().getClient());
+  context.emitter().Flush();
+}
+
+auto DisconnectCppAst(Context& context) -> void {
+  // `DisconnectCppAst` operates very close to where AST fragment is
+  // nullified. Builds may call this function with a null AST fragment,
+  // so we exit early in the case where it's null.
+  //
+  // TODO: investigate if we can avoid AST fragment disconnection when
+  // the fragment is null.
   if (!context.cpp_context()) {
     return;
   }
 
   // Leave the module we entered to encapsulate the contents of this Carbon
-  // file.
+  // file. This happens here rather than in `FinalizeCppAstFragment`, because
+  // that can run more than once for a file, but the module is only entered
+  // once.
   auto end_loc_id =
       SemIR::LocId(*(context.sem_ir().parse_tree().postorder().end() - 1));
   // Shuffle the end of file location back by one character to work around a
@@ -1056,14 +1178,20 @@ auto FinishAst(Context& context) -> void {
   LeaveModule(*context.cpp_context(),
               GetCppLocation(context, end_loc_id).getLocWithOffset(-1));
 
-  // Finalize the per-Context AST fragment. The final ActOnEndOfTranslationUnit
-  // call for the CppDomain is performed in FinalizeCppDomain once all files
-  // sharing the domain have been checked.
-  context.cpp_context()->sema().ActOnEndOfTranslationUnitFragment(
-      clang::TUFragmentKind::Normal);
-  FlushDiagnosticConsumer(
-      *context.cpp_context()->sema().getDiagnostics().getClient());
-  context.emitter().Flush();
+  // Hide the per-file lookup-only `::Carbon` namespace from the further lookup
+  // results and remove it from the translation unit's name lookup table if it's
+  // already there, so that subsequent files in a shared C++ AST domain do not
+  // see it.
+  if (auto* lookup_ns = context.cpp_context()->carbon_lookup_namespace()) {
+    lookup_ns->clearIdentifierNamespace();
+    auto* tu_decl = context.ast_context().getTranslationUnitDecl();
+    if (auto* lookup = tu_decl->getLookupPtr()) {
+      if (auto it = lookup->find(lookup_ns->getDeclName());
+          it != lookup->end()) {
+        it->second.remove(lookup_ns);
+      }
+    }
+  }
 
   // Remove the `CarbonExternalASTSource` installed in `GenerateAst` and
   // replace it with a `ReadOnlyASTSource`. This is necessary because

toolchain/check/cpp/generate_ast.h

@@ -52,9 +52,15 @@ auto GenerateAst(Context& context,
 auto InjectAstFromInlineCode(Context& context, SemIR::LocId loc_id,
                              llvm::StringRef source_code) -> void;
 
-// Finishes AST generation for the given checking context. Performs end of file
-// steps such as template instantiation and warning on unused declarations.
-auto FinishAst(Context& context) -> void;
+// Finalizes the C++ AST fragment for the given checking context. Performs end
+// of file steps such as template instantiation and warning on unused
+// declarations. This may be called more than once for the same context, to
+// perform template instantiations that later Carbon checking requires.
+auto FinalizeCppAstFragment(Context& context) -> void;
+
+// Disconnects C++ AST generation from the given checking context at the end of
+// checking.
+auto DisconnectCppAst(Context& context) -> void;
 
 // Finalizes a C++ domain at the end of checking all files.
 auto FinalizeCppDomain(SemIR::CppDomain& domain) -> void;

toolchain/check/cpp/impl_lookup.cpp

@@ -147,11 +147,12 @@ static auto BuildCopyWitness(Context& context, SemIR::LocId loc_id,
     auto class_type_id = SemIR::TypeId::ForTypeConstant(query_self_const_id);
     if (!Check::RequireCompleteType(
             context, class_type_id, SemIR::LocId::None, [&](auto& builder) {
-              CARBON_DIAGNOSTIC(IncompleteTypeInCopyWitness, Context,
-                                "argument to C++ call has incomplete type {0}",
-                                SemIR::TypeId);
-              builder.Context(loc_id, IncompleteTypeInCopyWitness,
-                              class_type_id);
+              CARBON_DIAGNOSTIC_CONTEXT(
+                  IncompleteTypeInCopyWitness,
+                  "argument to C++ call has incomplete type {0}",
+                  SemIR::TypeId);
+              builder.Attach(loc_id, IncompleteTypeInCopyWitness,
+                             class_type_id);
             })) {
       return SemIR::ErrorInst::InstId;
     }
@@ -247,7 +248,7 @@ static auto BuildDefaultWitness(
                             query_specific_interface, {fn_id});
 }
 
-static auto BuildCppDestroyWitness(
+static auto BuildCppSubobjectDestroyWitness(
     Context& context, SemIR::LocId loc_id,
     SemIR::ConstantId query_self_const_id,
     SemIR::SpecificInterface query_specific_interface) -> SemIR::InstId {
@@ -259,8 +260,8 @@ static auto BuildCppDestroyWitness(
   }
   auto* class_decl = dyn_cast<clang::CXXRecordDecl>(tag_decl);
   if (!class_decl) {
-    return BuildTrivialDestroyWitness(context, loc_id, query_self_const_id,
-                                      query_specific_interface);
+    return BuildTrivialSubobjectDestroyWitness(
+        context, loc_id, query_self_const_id, query_specific_interface);
   }
   SemIR::ClangDeclSignatureId signature_id = MakeSignature(context, {});
 
@@ -270,11 +271,8 @@ static auto BuildCppDestroyWitness(
   if (fn_id == SemIR::ErrorInst::InstId || fn_id == SemIR::InstId::None) {
     return fn_id;
   }
-  return BuildDestroyWitness(
-      context, loc_id,
-      GetFacetAccessType(
-          context, context.constant_values().GetInstId(query_self_const_id)),
-      query_self_const_id, query_specific_interface, {fn_id});
+  return BuildCustomWitness(context, loc_id, query_self_const_id,
+                            query_specific_interface, {fn_id});
 }
 
 // Attempts to build a witness table entry for a C++ unary operator.
@@ -618,9 +616,9 @@ auto LookupCppImpl(Context& context, SemIR::LocId loc_id,
     case SemIR::CoreInterface::Default:
       return BuildDefaultWitness(context, loc_id, query_self_const_id,
                                  query_specific_interface);
-    case SemIR::CoreInterface::Destroy:
-      return BuildCppDestroyWitness(context, loc_id, query_self_const_id,
-                                    query_specific_interface);
+    case SemIR::CoreInterface::SubobjectDestroy:
+      return BuildCppSubobjectDestroyWitness(
+          context, loc_id, query_self_const_id, query_specific_interface);
 
     case SemIR::CoreInterface::CppRangeForIterate:
       return BuildCppRangeForIterateWitness(

toolchain/check/cpp/import.cpp

@@ -1015,6 +1015,33 @@ static auto BuildClassDefinition(Context& context,
                                                .vtable_id = vtable_id}));
   }
 
+  // Import friend declarations.
+  for (const auto* f : clang_def->friends()) {
+    if (const auto* type_source_info = f->getFriendType()) {
+      auto [friend_type_inst_id, friend_type_id] =
+          ImportCppType(context, SemIR::LocId(import_ir_inst_id),
+                        type_source_info->getType());
+
+      if (!friend_type_id.has_value()) {
+        // TODO: For now, just skip over friends whose types we can't map.
+        continue;
+      }
+
+      class_info.friend_scopes.Insert(
+          context.types().GetConstantId(friend_type_id));
+    } else {
+      auto* friend_decl = f->getFriendDecl();
+      // TODO: support function friend decls.
+      if (!friend_decl->isFunctionOrFunctionTemplate()) {
+        auto key = SemIR::ClangDeclKey::ForNonFunctionDecl(friend_decl);
+        auto inst_id =
+            ImportCppDecl(context, SemIR::LocId(import_ir_inst_id), key);
+
+        class_info.friend_scopes.Insert(context.constant_values().Get(inst_id));
+      }
+    }
+  }
+
   class_info.body_block_id = context.inst_block_stack().Pop();
 }
 
@@ -1844,8 +1871,9 @@ static auto CreateFunctionSignatureInsts(
   // C++ functions have no implicit parameters. Use `None` rather than an empty
   // block so that the signature matches a Carbon function with no implicit
   // parameter list.
-  auto match_results = CalleePatternMatch(context, SemIR::InstBlockId::None,
-                                          param_patterns_id, return_pattern_id);
+  auto match_results =
+      CalleePatternMatch(context, loc_id, SemIR::InstBlockId::None,
+                         param_patterns_id, return_pattern_id);
 
   return {{.implicit_param_patterns_id = SemIR::InstBlockId::None,
            .param_patterns_id = param_patterns_id,
@@ -2071,9 +2099,9 @@ static auto ImportFunctionDecl(Context& context, SemIR::LocId loc_id,
   if (IsCppThunkRequired(context, callee_info)) {
     Diagnostics::AnnotationScope annotate_diagnostics(
         &context.emitter(), [&](auto& builder) {
-          CARBON_DIAGNOSTIC(InCppThunk, Note,
-                            "in thunk for C++ function used here");
-          builder.Note(loc_id, InCppThunk);
+          CARBON_DIAGNOSTIC_LABEL(InCppThunk, Info,
+                                  "in thunk for C++ function used here");
+          builder.Attach(loc_id, InCppThunk);
         });
     DefineAsThunkCall(context, loc_id, callee_info, imported_function);
   } else {
@@ -2138,9 +2166,10 @@ auto ImportFunctionPointerInvoke(
   }
   Diagnostics::AnnotationScope annotate_diagnostics(
       &context.emitter(), [&](auto& builder) {
-        CARBON_DIAGNOSTIC(InCppFunctionPointerThunk, Note,
-                          "in thunk for C++ function pointer type used here");
-        builder.Note(loc_id, InCppFunctionPointerThunk);
+        CARBON_DIAGNOSTIC_LABEL(
+            InCppFunctionPointerThunk, Info,
+            "in thunk for C++ function pointer type used here");
+        builder.Attach(loc_id, InCppFunctionPointerThunk);
       });
 
   CppCalleeFunctionInfo callee_info(context, info.clang_type);
@@ -2744,9 +2773,10 @@ auto ImportNameFromCpp(Context& context, SemIR::LocId loc_id,
     -> SemIR::ScopeLookupResult {
   Diagnostics::AnnotationScope annotate_diagnostics(
       &context.emitter(), [&](auto& builder) {
-        CARBON_DIAGNOSTIC(InCppNameLookup, Note,
-                          "in `Cpp` name lookup for `{0}`", SemIR::NameId);
-        builder.Note(loc_id, InCppNameLookup, name_id);
+        CARBON_DIAGNOSTIC_LABEL(InCppNameLookup, Info,
+                                "in `Cpp` name lookup for `{0}`",
+                                SemIR::NameId);
+        builder.Attach(loc_id, InCppNameLookup, name_id);
       });
   if (IsIncompleteClass(context, scope_id)) {
     return SemIR::ScopeLookupResult::MakeError();

toolchain/check/cpp/location.cpp

@@ -6,6 +6,7 @@
 
 #include "clang/AST/ASTContext.h"
 #include "clang/Basic/SourceManager.h"
+#include "toolchain/lex/tokenized_buffer.h"
 #include "toolchain/sem_ir/absolute_node_ref.h"
 #include "toolchain/sem_ir/ids.h"
 
@@ -68,8 +69,10 @@ auto GetCppLocation(Context& context, SemIR::LocId loc_id)
       SemIR::GetAbsoluteNodeRef(&context.sem_ir(), loc_id);
   const auto& final_node = absolute_node_refs.back();
   if (final_node.is_cpp()) {
-    return final_node.file()->clang_source_locs().Get(
-        final_node.clang_source_loc_id());
+    return final_node.file()
+        ->clang_source_locs()
+        .Get(final_node.clang_source_loc_id())
+        .getBegin();
   }
 
   if (!final_node.node_id().has_value()) {
@@ -88,11 +91,55 @@ auto GetCppLocation(Context& context, SemIR::LocId loc_id)
   return start_loc.getLocWithOffset(offset);
 }
 
-auto AddImportIRInst(SemIR::File& file, clang::SourceLocation clang_source_loc)
+auto GetCppRange(Context& context, SemIR::LocId loc_id)
+    -> clang::CharSourceRange {
+  if (!context.sem_ir().cpp_file()) {
+    return clang::CharSourceRange();
+  }
+
+  llvm::SmallVector<SemIR::AbsoluteNodeRef> absolute_node_refs =
+      SemIR::GetAbsoluteNodeRef(&context.sem_ir(), loc_id);
+  const auto& final_node = absolute_node_refs.back();
+  if (final_node.is_cpp()) {
+    auto loc = GetCppLocation(context, loc_id);
+    return clang::CharSourceRange::getTokenRange(loc, loc);
+  }
+  if (!final_node.node_id().has_value()) {
+    return clang::CharSourceRange();
+  }
+
+  auto [ir, start_loc] = GetFileInfo(context, final_node.check_ir_id());
+  const auto& tokens = ir->parse_tree().tokens();
+  auto token = ir->parse_tree().node_token(final_node.node_id());
+  // Only the file being checked has its subtrees available, since a `Context`
+  // holds the one `TreeAndSubtrees`.
+  Lex::InclusiveTokenRange token_range = {.begin = token, .end = token};
+  if (final_node.check_ir_id() == context.sem_ir().check_ir_id()) {
+    token_range = context.parse_tree_and_subtrees().GetSubtreeTokenRange(
+        final_node.node_id());
+  }
+  return clang::CharSourceRange::getCharRange(
+      start_loc.getLocWithOffset(tokens.GetByteOffset(token_range.begin)),
+      start_loc.getLocWithOffset(tokens.GetByteOffset(token_range.end) +
+                                 tokens.GetTokenText(token_range.end).size()));
+}
+
+auto AddImportIRInst(SemIR::File& file, clang::CharSourceRange clang_range)
     -> SemIR::ImportIRInstId {
+  // Stored as its two ends: every range reaching here is a character range, so
+  // `CharSourceRange`'s token-or-character bit would always say the same thing
+  // and cost a word to say it. Rebuilt as one where it is read.
+  CARBON_CHECK(!clang_range.isTokenRange(),
+               "A token range would lose which it was; widen it first.");
   SemIR::ClangSourceLocId clang_source_loc_id =
-      file.clang_source_locs().Add(clang_source_loc);
+      file.clang_source_locs().Add(clang_range.getAsRange());
   return file.import_ir_insts().Add(SemIR::ImportIRInst(clang_source_loc_id));
 }
 
+auto AddImportIRInst(SemIR::File& file, clang::SourceLocation clang_source_loc)
+    -> SemIR::ImportIRInstId {
+  return AddImportIRInst(file, clang::CharSourceRange::getCharRange(
+                                   clang_source_loc, clang_source_loc));
+}
+
 }  // namespace Carbon::Check

toolchain/check/cpp/location.h

@@ -14,8 +14,26 @@ namespace Carbon::Check {
 auto GetCppLocation(Context& context, SemIR::LocId loc_id)
     -> clang::SourceLocation;
 
-// Adds an `ImportIRInst` referring to the given source location and returns a
-// corresponding `ImportIRInstId` that can be used to construct a `LocId`.
+// Maps a Carbon source location into the Clang range covering the source a
+// Carbon diagnostic marked with it would underline, which is everything the
+// node spans rather than the one token it names.
+//
+// In Carbon source this is a character range, since Clang finds the end of a
+// token range by lexing the token there as C++. Only the file being checked has
+// its subtrees available, so a location in an imported file gives the range of
+// the one token `GetCppLocation` names. A location that is already in C++ gives
+// a token range at that token, which Clang can measure.
+auto GetCppRange(Context& context, SemIR::LocId loc_id)
+    -> clang::CharSourceRange;
+
+// Adds an `ImportIRInst` referring to the given source range and returns a
+// corresponding `ImportIRInstId` that can be used to construct a `LocId`. The
+// range is what a diagnostic marking the location underlines.
+auto AddImportIRInst(SemIR::File& file, clang::CharSourceRange clang_range)
+    -> SemIR::ImportIRInstId;
+
+// The same for a Clang location, which is a point and marks the one column it
+// names.
 auto AddImportIRInst(SemIR::File& file, clang::SourceLocation clang_source_loc)
     -> SemIR::ImportIRInstId;
 

toolchain/check/cpp/operators.cpp

@@ -17,6 +17,7 @@
 #include "toolchain/check/custom_witness.h"
 #include "toolchain/check/function.h"
 #include "toolchain/check/inst.h"
+#include "toolchain/check/interface.h"
 #include "toolchain/check/literal.h"
 #include "toolchain/check/name_lookup.h"
 #include "toolchain/check/pattern.h"
@@ -39,7 +40,7 @@ static auto GetClangOperatorKind(Context& context, SemIR::LocId loc_id,
     -> std::optional<clang::OverloadedOperatorKind> {
   switch (interface_name) {
       // Unary operators.
-    case CoreIdentifier::Destroy:
+    case CoreIdentifier::DestroyImpl:
     case CoreIdentifier::SubobjectDestroy:
     case CoreIdentifier::SelfDestruct:
     case CoreIdentifier::As:
@@ -200,6 +201,12 @@ static auto GetClangOperatorKind(Context& context, SemIR::LocId loc_id,
       return clang::OO_Subscript;
     }
 
+    // C++ `operator[]` is found through `IndexWith`, which subscripting falls
+    // back to.
+    case CoreIdentifier::IndirectIndexWith: {
+      return std::nullopt;
+    }
+
     default: {
       context.TODO(loc_id, llvm::formatv("Unsupported operator interface `{0}`",
                                          interface_name));
@@ -575,31 +582,6 @@ static auto GetBuiltinOperatorInfo(clang::OverloadedOperatorKind kind)
   return OpTable[kind];
 }
 
-static auto GetCoreInterfaceId(Context& context, SemIR::LocId loc_id,
-                               CoreIdentifier interface_name)
-    -> SemIR::InterfaceId {
-  auto inst_id = LookupNameInCore(context, loc_id, interface_name);
-
-  // Non-generic interfaces.
-  if (auto facet_type = context.insts().TryGetAs<SemIR::FacetType>(inst_id)) {
-    const auto& declared =
-        context.declared_facet_types().Get(facet_type->declared_facet_type_id);
-    auto single = declared.TryAsSingleExtend();
-    CARBON_KIND_SWITCH(*single) {
-      case CARBON_KIND(SemIR::SpecificInterface si): {
-        return si.interface_id;
-      }
-      case CARBON_KIND(SemIR::SpecificNamedConstraint _): {
-        CARBON_FATAL("Operators in named constraints are not yet needed");
-      }
-    }
-  }
-
-  auto type_id = context.insts().Get(inst_id).type_id();
-  auto generic = context.types().GetAs<SemIR::GenericInterfaceType>(type_id);
-  return generic.interface_id;
-}
-
 // Builds a Carbon builtin function declaration corresponding to an overload
 // candidate that selected a C++ builtin operator. Returns None if no
 // corresponding builtin function could or should be built.
@@ -673,12 +655,12 @@ struct DiagnoseIncompleteOperandTypeInCppOperatorLookup {
   SemIR::LocId loc_id;
 
   void operator()(auto& builder) const {
-    CARBON_DIAGNOSTIC(
-        IncompleteOperandTypeInCppOperatorLookup, Context,
+    CARBON_DIAGNOSTIC_CONTEXT(
+        IncompleteOperandTypeInCppOperatorLookup,
         "looking up a C++ operator with incomplete operand type {0}",
         SemIR::TypeId);
-    builder.Context(loc_id, IncompleteOperandTypeInCppOperatorLookup,
-                    arg_type_id);
+    builder.Attach(loc_id, IncompleteOperandTypeInCppOperatorLookup,
+                   arg_type_id);
   }
 };
 }  // namespace