Converting an integer value to a destination type that is not a valid
integer type -- such as `i8388609` or `i16777216`, whose bit widths are
diagnosed as invalid -- hit a CHECK failure in
`TypeStore::GetIntTypeInfo` during constant evaluation of `int.convert`
/ `int.convert_checked`:
```
CHECK failure at toolchain/sem_ir/type.cpp:189: int_info: Type type(...) is not an integer type
```
The width error is already diagnosed when forming the type
(`IntWidthNotMultipleOf8` / `IntWidthTooLarge`), so `PerformIntConvert`
and `PerformCheckedIntConvert` now use `TryGetIntTypeInfo` and produce
an error value instead of crashing, following the existing
`SemIR::ErrorInst::ConstantId` convention in `eval.cpp`.
Added a file test covering all three reproducers from the issue
(`i8388609`, `i16777216`, `Core.Int(8388609)`); it crashes without this
change. The full `//toolchain/testing:file_test` suite (1578 tests)
passes.
Fixes#7278.
I have reviewed this change and take responsibility for it.
Assisted-by: Claude
In generate_ast.cpp, an `CarbonExternalASTSource` is installed that has
a `Check::Context` pointer. During lowering, this `ExternalASTSource` is
still installed, and using it can cause a crash if the now-invalid
pointer is dereferenced.
Fix by adding a new `ReadOnlyASTSource` in sem_ir, and using that during
lowering.
`CarbonExternalASTSource` now inherits from `ReadOnlyASTSource` to avoid
some code duplication.
In generate_ast.cpp, we now always install a multiplex source, even if
there's only one child source. Clang internally keeps pointers to the
top-level `ExternalASTSource` installed via `setExternalSource`, and
those pointers aren't updated if `setExternalSource` is called again. By
using `MultiplexExternalSemaSource`, we can keep the top-level
`ExternalASTSource` pointer the same, and only update its children.
Using `MultiplexExternalSemaSource` this way requires a new constructor
and a method to modify its child sources; added a new LLVM patch adding
those.
https://github.com/carbon-language/carbon-lang/issues/7142
Malformed `base` declarations with an omitted colon need two different
recovery paths. For `extend base`, the consumed `extend` modifier
requires the parse tree to retain its `BaseColon` and base expression
children, so this synthesizes an errored `BaseColon` and continues
parsing the expression.
Other malformed forms, such as `base calss X {}`, now use the standard
declaration-error recovery: emit `ExpectedAfterBase`, skip past the
likely declaration end, and form an errored `BaseDecl` without inventing
a colon or cascading diagnostics.
The regression covers `extend base Foo;`, bare `base;`, and the reviewer
counterexample `base calss X {}`.
Tests:
- `prek run --files toolchain/parse/handle_base.cpp
toolchain/parse/testdata/class/fail_base.carbon`
- `./scripts/run_bazelisk.py test -c dbg //toolchain/parse/...`
- `./scripts/run_bazelisk.py test -c dbg //toolchain/testing:file_test`
AI assistance: OpenAI Codex helped inspect the parser recovery path,
implement the change, and run verification. The operator reviewed and
authorized the contribution.
Assisted-by: OpenAI Codex
Because we now support calling a function with a `self` parameter
directly, we can unconditionally call `operator$(lhs, rhs)` rather than
calling `lhs.operator$(rhs)` if the selected operator function happens
to be a member function.
This makes the logic a bit simpler and the SemIR a bit smaller.
We can't do the same for Carbon operators, unfortunately, as we use the
member access to trigger impl lookup.
A binding can have a constant value that is an `ErrorInst` during error
recovery. In that case, deduction would crash when attempting to
diagnose that the binding had no deduced value.
This is based on an idea suggested during review of the original type
erasure PR.
The type-erased check failure path lowers each check through
`CheckFailFormat<Ts...>`, which builds one format adapter per value,
down to the out-of-line `CheckFailImpl`, which takes an array of
base-class `format_adapter*` pointers. Previously a separate variadic
`CheckFailWithAdapters<Adapters...>` template sat in between: it existed
only to bind the adapter temporaries to named parameters so that
pointers to their base class could be collected into a `std::array` and
outlive the call to `CheckFailImpl`.
This removes that layer. `CheckFailFormat` now builds the pointer array
directly, in the braced-init-list of the `CheckFailImpl` call, using a
small `CheckFailFormatAdapterAddr<T>` helper to take the base-class
address of each adapter. The adapter temporaries are materialized as
named parameters of that helper within the same full-expression as the
`CheckFailImpl` call, so they remain alive across the call without a
dedicated function for that purpose.
The win is in debug info, not code. `CheckFailWithAdapters<Adapters...>`
was instantiated once per distinct adapter-type sequence in every
translation unit that uses `CARBON_CHECK`, and each instantiation
carried its own DWARF records -- type entries, string-table offsets, and
range lists. Dropping it removes those records from every such
translation unit. The generated machine code is unchanged.
Measured impact (fastbuild, the inputs to the inner-loop links):
- First-party object files shrink by 173,728 bytes (-0.233%) across the
`carbon-busybox` link inputs; 232 of 270 objects get smaller and none
grow. The largest reductions are in the check-densest translation units
(`type_completion`, `type`, `import_ref`, `function_context`,
`constant`, ...).
- Per `bloaty`, the reduction is almost entirely DWARF (`.debug_info`,
`.debug_str`, `.rela.debug_str_offsets`, `.debug_rnglists`); the loaded
code (VM size) is unchanged.
Assisted-by: Claude Code
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Split out the `prek` tool usage instructions into a separate skill. This
should make the agent more likely to realize the skill is relevant to a
particular task and consult it. Extend the skill to include instructions
for using `prek` in a jj workspace, and add a helper script for that
situation.
Add a `jj` skill, with the main purpose being to instruct the agent to
use `jj` not `git`, and to use `--no-pager` when running it.
Extend the `bazel` tool description slightly to more strongly encourage
agents to read and follow it.
Assisted-by: Gemini via Antigravity
Adds support for arithmetic and comparison operators on
`Core.CharLiteral`s, as well as conversions between `CharLiteral` and
integer types.
Make some minor tweaks to fix skill issues encountered while making this
change.
Assisted-by: Gemini via Antigravity
Implements proposal #7016: `self` moves from the deduced implicit list
(`fn F[self: Self]()`) to the front of the explicit list. Its type may
be written explicitly (`fn F(self: Self)`) or omitted, in which case it
defaults to `Self` (`fn F(self)`, `fn F(ref self)`); `self` in the
implicit list is rejected.
Throughout checking, `self` is modeled as the first explicit parameter.
Because a method is just a function whose first parameter is `self`, it
can also be called as an ordinary function with the receiver passed
explicitly (`Type.M(obj, ...)`), not only as `obj.M(...)`. A new
`SemIR::CallArgParamPatterns` helper chooses the parameters matched
against the explicit arguments, excluding a leading `self` only when it
is supplied as a method-call receiver; arity checking, conversion, and
generic deduction use it. The resulting SemIR and lowering are
unchanged: `self` is still `call_param0`, and witnesses, thunks, and
vtables are unaffected.
An omitted `self` type is parsed as a `SelfBindingPattern` node with no
type expression; checking synthesizes the `Self` type so it behaves
exactly like `self: Self`. However, the exact spelling used must match
between a forward declaration and a definition, following #3763's rules
around declaration matching.
Generated functions, thunks, and C++ interop import/export build `self`
as the first explicit parameter, and the `self`-type override (e.g.
Derived->Base for a virtual override) applies to the explicit `self`.
Placement is validated by new diagnostics: `SelfInImplicitParamList`,
`SelfNotFirstParam`, and `SelfOutsideParamList`. The benchmark source
generator and the documentation adopt the `(self)` shorthand; the
prelude, the examples, and the test data are migrated in the following
commits.
Assisted-by: Claude Code with Claude Opus 4.7
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
This is only valid when the operand is an initializing expression that
holds a copy of the value, but we were incorrectly also forming it when
the operand was an in-place initializing expression.
Fixes a crash in lowering when attempting to lower an invalid
`value_of_initializer`.
`CARBON_CHECK` and `CARBON_FATAL` messages are formatted with
`llvm::formatv`. Previously each check site that had a message
instantiated its own copy of the formatv machinery -- a `formatv_object`
over a tuple of per-argument format adapters, plus that tuple -- in
every translation unit, keyed on the site's file, line, condition, and
format strings. A translation unit with many checks paid for that
machinery over and over.
This restructures check failure so that the formatting machinery is
compiled exactly once, and only a single small adapter is instantiated
per distinct value type per translation unit:
- `CheckFailImpl` (out-of-line) now takes the message's format string
and an array of already-type-erased `format_adapter`s, and renders the
whole failure message -- prefix plus the extra message -- directly into
one stream. The extra message is rendered in place, so no separate
string is ever materialized for it.
- `FormatvInto` (in the `.cpp`) renders a format string over that
adapter array. Rather than instantiate `llvm::formatv`, it drives the
formatv replacement loop over the public
`formatv_object_base::parseFormatString`, so this rendering code exists
exactly once. (A TODO notes that we should add a type-erased entry point
upstream in LLVM rather than reimplement the loop here.)
- The lowering from the macro down to that out-of-line call is split so
that the only per-check-site instantiation is trivial:
- `CheckFail<...>` is instantiated once per site, since its file, line,
condition, and format template-string parameters are unique to the site.
It just lowers those compile-time strings to ordinary arguments and
forwards to `CheckFailFormat`.
- `CheckFailFormat<Ts...>` is instantiated once per distinct value-type
sequence and shared across sites; it builds one type-erased adapter per
value.
- `CheckFailWithAdapters<Adapters...>` collects pointers to those
adapters into an array and calls `CheckFailImpl`. It is a distinct
function so the adapter temporaries stay alive while pointers to their
base class are in flight.
Format semantics, including runtime format-string validation, are
unchanged, and the rendered message is byte-for-byte identical.
For `DCHECK` in optimized builds the check is dead code; its arguments
are now routed through a trivial `IgnoreDeadCheckArgs` no-op rather than
`CheckFail`. This still type-checks the arguments so they cannot bitrot,
without instantiating any formatting machinery for them and without
provoking unused-variable warnings.
Measured full-rebuild impact (353 first-party translation units,
fastbuild): -112.6s CPU, -6.9% relative to trunk.
Assisted-by: Claude
Adapters were erroneously satisfying `Core.Destroy` because we were
directly getting the object's representation without consideration for
abstract and adapted types. This change ensures that adapted types'
representations are used instead of the adapter types.
This helps us move away from the clone-with-modifications approach to
thunking, which gets unwieldy as signatures get more complex.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This is the first draft of the implementation of
[p6333](https://docs.carbon-lang.dev/proposals/p6333.html).
The `build` subcommand shared logic with the `compile` and `link`
subcommands,
so I've moved some of the functionality in `compile` and `link` to
shared
`CompileDriver` and `LinkDriver` classes, respectively. This also
required exposing the
`CompileOptions` and `LinkOptions` subcommand structs for re-use.
There's still some work to do on the proposal, most notably the package
include automatic path resolution and import, and the refactors to
`carbon compile`.
Mostly these errors were around `dict` missing arguments, and they are
almost always `[str, Any]`.
But a real thorn here was `xml.etree.ElementTree.Element`. `ty` insists
that this is a generic type, and indeed it appears to be one, or
becoming one, in some python version. But it is not generic in python
3.12. So we are stuck in an unsolvable land where:
- `ty` gives an error unless you write `[str]` on the type, because it
thinks it is generic.
- python3.12 gives an error if you do write `[str]` on the type, because
it thinks it is not generic.
Forcing `ty` to target exactly python 3.12 does not help. So I have just
used a linter-ignore comment on that line.
When building Carbon on Fedora, clang reports the
resource directory as `/usr/bin/../lib/clang/22`.
This fails to string match against `/usr/lib/clang/22`
and so bazel reports an error.
This PR canonicalizes the path returned by clang
so that it will string match successfully.
These were defined inline in headers reached by most of the toolchain,
and as non-template functions their bodies -- including some very
expensive template instantiations -- are compiled in every including TU
even though they are only cold debug/dump paths:
- BundleStore's YAML output entry points instantiate IdAndKind::Dispatch
over all ~50 ID kinds (two ~365ms instantiations per TU) plus
Yaml::OutputScalar/std::function machinery; moved to a new bundle.cpp.
- TypeStore, ConstantValueStore, NameStoreWrapper, and SharedValueStores
OutputYaml bodies wrap capturing lambdas in std::function via
Yaml::OutputMapping; moved to their existing .cpp files.
This change appears to be worth another 10% compile time reduction.
Assisted-by: Claude
Fundamentally, this uses forward declarations of Clang types to reduce
the overall compile time cost of Clang headers across the codebase.
Tracing and profiling showed ~2s of every check TU's ~8-12s compile time
going just to parsing Clang frontend and AST headers pulled in via a few
sem_ir and check headers that only use the Clang types by pointer or
reference:
- sem_ir/cpp_file.h (reached via sem_ir/file.h by ~150 TUs) included
clang/Frontend/CompilerInstance.h, clang/CodeGen/ModuleBuilder.h,
clang/AST/Mangle.h, and llvm/IR/Module.h. CppFile's accessors move out
of line to a new cpp_file.cpp and the header now forward-declares the
Clang types.
- check/cpp/context.h (reached via check/context.h by ~100 TUs) included
clang/Frontend/FrontendAction.h and clang/Parse/Parser.h, pulling in
clang's Sema.h and ASTUnit.h.
- sem_ir/clang_decl.h included clang/AST/Decl.h; the three small
functions that need complete Clang types move out of line.
- sem_ir/cpp_overload_set.h included clang/Sema/Overload.h solely for
the three-field OverloadCandidateSet::OperatorRewriteInfo, which is now
mirrored as CppOverloadSet::OperatorRewriteInfo, and clang/AST/Decl.h
solely for a pointer.
- sem_ir/name_scope.h's clang/AST/DeclBase.h include was vestigial.
TUs (and more narrowly included headers) that genuinely use the Clang
definitions now include the Clang headers directly.
Representative compile times (fastbuild, aarch64), combined with the
preceding instantiation-cost changes, relative to trunk:
- check/eval.cpp: 11.85s -> 6.94s (-41%)
- check/handle_operator.cpp: 7.71s -> 3.30s (-57%)
- language_server.cpp: 6.68s -> 3.16s (-53%)
- lower/handle.cpp: 6.75s -> 3.66s (-46%)
- sem_ir/file.cpp: 8.60s -> 6.11s (-29%)
- driver.cpp: 6.68s -> 4.78s (-28%)
Measured full-rebuild impact (316 first-party TUs, fastbuild): -689.5s
CPU, -29.9% relative to trunk.
Assisted-by: Claude
Instead of silently producing an `ErrorInst::InstId` when looking up a
private qualified name in the current package, bypass the access check.
We don't need it -- private names from other libraries are filtered out
by the import logic.
Also fix `DiagnoseInvalidQualifiedNameAccess` to actually always produce
a diagnostic, instead of silently ignoring access control failures in
non-class types. This is a no-op after the fix to the access logic,
since we only allow access control at class and namespace scope
currently, but should avoid this issue from recurring when that changes.
Assisted-by: Gemini via Antigravity
We exposed `Core.IntLiteral()`, `Core.FloatLiteral()`,
`Core.CharLiteral()`, and `Core.Bool()` as functions as a workaround,
because we had no way to provide the type names without parentheses that
the design requests. But now we can do so, by using an alias. Switch all
of these over from being functions to simply being names of the
corresponding types.
Assisted-by: Gemini via Antigravity
Change `Lookup` by InstId to return a ClangDecl pointer. All callers
were immediately calling `Get` anyway, so this makes call sites a little
shorter. The other `Lookup` method, by ClangDeclKey, is sometimes called
without calling `Get`, so left that as-is, but renamed to `LookupId`.
Also add a `decl` method to ClangDecl so that the commonly repeated
`clang_decl->key.decl` can be written `clang_decl->decl()`.
My understanding is that `auto` is encouraged even when there's no
applicable naming convention for the variable, and that suffixes like
`_id` are encouraged where applicable, even if the type is explicit, so
there's really no connection between the two policies.
This ensures the clang_decls map is used as a cache - without this,
visiting the same entity twice could cause it to be
re-exported/duplicated. See attached test case.
Replace all uses of CppGlobalVarStore store with ClangDeclStore.
Adding a VarStorage->VarDecl mapping to ClangDeclStore is now done with
the `AddVar` method, which takes an extra `pattern_id` arg. While the
corresponding `ClangDecl` is unchanged from before, the reverse mapping
in `inst_id_to_clang_decl_id_` now uses the `pattern_id` as the key.
This is necessary because in some places the original VarStorage
instructions gets replaced (e.g. by a call to `Convert`). The
`pattern_id` remains stable in those cases.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This proposal clarifies some unclear aspects of the interop support for
object-like macros. In particular:
- Carbon supports importing an object-like macro if its definition can
be evaluated as a constant expression, without further restrictions on
that definition.
- When importing the result of that evaluation, C++ lvalues are imported
as references, and rvalues are imported as values.
Removing the clang_decl_id on SemIR::Function - using only the
clang_decls map to create the association between SemIR::Function and
clang::FunctionDecls.
This adds an `is_external` flag to ClangDecl to indicate whether the
entity originated from Carbon or was imported from another language.
(I'm open to names - I guess for now we mostly use "is this from C++" to
be more specific than "is this external" - eg: NameScope::is_cpp_scope)
Fix import logic to make all imported packages be children of the
`NameScopeId::Package` scope. Previously, indirectly-imported packages
would end up as children of their importing package's scope, which
resulted in them not being treated as packages at all, and in particular
not being fingerprinted as packages.
Fixing that caused a failure in the fingerprinting logic as we started
to encounter packages with no correspoding import scopes. Instead of
looking for import scopes, use a simpler mechanism to map packages to
their package names, and clean up.
Unfortunately the latter change churns all the fingerprints again :(
Hopefully this is the last time for a while.
Fix a lowering crash when lowering a return by reference of a type with
an in-place initializing representation. We previously misinterpreted
this as an in-place initializing return.
This is addressed by changing lowering to interpret a `ReturnExpr` of a
reference expression as a reference return. However, that exposes
another issue: `return var;` produces a `ReturnExpr` of a reference
expression in the case where it returns in place! To fix that, we switch
`return var;` to producing a `ReturnExpr` of a value expression
regardless of whether the function has a return slot. This makes the
representation of `return var;` more uniform:
* If the expression is a reference, we're performing a `ref` return.
* If the expression is an initializing expression, we're performing a
normal by-initialization return.
* If the expression is a value expression, we're performing a `return
var;`.
Implement support for floating-point <-> integer type conversions as
described in #820 and #845, extended to support `unsafe as` conversions
for the conversions that can't be expressed as either implicit
conversions or `as` conversions.
One tricky part here is conversions from floating-point literals to
integer types. Such literals may have both a very large mantissa and a
corresponding somewhat large negative exponent, and still produce a
result that is in the range of values that a small integer type can
represent. In order to support that while avoiding building very large
2^N or 10^N constants in general, we first compute a conservative
approximation of the number of bits necessary to represent the integer
result, with an early exit if the number is either definitely too large
or definitely zero. The remaining cases have a reasonable bound on the
size of integer necessary to compute the base^exponent multiplicand.
Assisted-by: Gemini via Antigravity
Add `ExportVarToCpp`. This checks the `clang_decls` mapping and returns
an existing decl if found. Otherwise, it creates a new `VarDecl` and
adds it to the `clang_decls` mapping.
When lowering, in `FileContext::BuildGlobalVariableDecl`, the
`clang_decls` mapping is used to lookup an existing
`llvm::GlobalVariable` for the instruction. If found, use that rather
than creating a new one to avoid an unwanted second definition in the
llvm IR.
This provides a native harness with several advantages over
`pre-commit`:
- Faster when initializing the cache
- Smaller cache sizes: under 43mb compared to over 62mb
- Better integration with `uv` for Python usage
Steps for migrating for existing contributors:
1. Install `prek` following instructions in the updated docs.
2. Replace hooks in an existing checkout with a special flag:
```sh
prek install --overwrite
```
The `--overwrite` flag is what removes the old hooks.
If you used the pre-push variant:
```sh
prek install --hook-type pre-push --overwrite
```
3. Optional cleanups:
```sh
rm -rf ~/.cache/pre-commit # reclaim the old hook-environment cache
pipx uninstall pre-commit # if installed via pipx; or `brew uninstall
pre-commit`
```
Assisted-by: Antigravity with Gemini
Not sure how these got missed when moving other things to `uv`, but this
should clean them up.
The bump to Python 3.12 is so that we can use `@override` with the
simple import from `typing`. This is needed by the newest versions of
`ty` to do type checking. Added the relevant `@override` annotations.
Assisted-by: Antigravity with Gemini
Nicholas Bishop
mstr-six
Christopher Di Bella
Richard Smith
Chandler CarruthandGeoff Romer
Lucile Rose Nihlen
David Blaikie