We can lower the same variable from multiple `SemIR::File`s if we reach
it through a specific for a generic in a different file. When this
happens, reuse the same global variable in lowering. This matches how we
handle the same problem for functions.
Assisted-by: Claude via Antigravity
When building an expression referencing the result of name lookup, if
the lookup result is non-constant then check we can access it from the
current scope.
* If it is a global, then use a new `GlobalRef` instruction to get the
value of that global. In lowering, create global variables to hold the
values of global `let` bindings so that `GlobalRef` can access those
values.
* If it is a local, walk the lexical scopes between it and the point of
use and ensure it is usable through from all of them. Error if there's
an intervening declarative scope such as a class, add a TODO to handle
captures for intervening functions, and allow `typeof(...)` to name
things it can't read the value of since it's unevaluated.
Global `let`s (which includes `let`s at class scope) now also have their
initializers collected into the global inits block. If the initializer
is non-constant, the wrapper binding also lives in the global inits
block; otherwise it continues to live in the file block and the variable
emitted for it is a global constant.
Assisted-by: Gemini and Claude via Antigravity
Rumdl has grown the ability to format python snippets. That's not what
we want in an example whose purpose is to show off some
weirdly-formatted python code! So turn it off. Also turn it off for the
weirdly formatted Swift and Kotlin code here which rumdl doesn't
reformat today, but might reformat tomorrow.
Treat the function as having a `-> auto` return type. Reject calls to
functions that still have a non-deduced `-> auto` return type, and
rewrite the function to have the actual return type once it is known.
Assisted-by: Gemini via Antigravity
Split the `Carbon::` namespace into one semantic namespace tree, plus
one separate lookup-only namespace tree per Carbon file. Use the
semantic namespaces as the parents of entities, but use the lookup-only
namespaces when C++ name lookup names a Carbon namespace, in order to
use a different C++ name lookup table in each Carbon file.
Assisted-by: Gemini via Antigravity
When a deduced return type depends on the function's generic parameters,
it's computed by the definition's eval block, so resolve the specific's
definition at the call. Calling a specific during its own instantiation,
before the point where its return type is deduced, is diagnosed like any
other use of the function before its return type is known. Also allow
exporting such functions to C++.
Resolving such a specific from within a checked generic can perform
template actions with symbolic arguments. Perform delayed actions in a
discarded generic region so the instructions they create aren't attached
to the generic being checked, and check that evaluating a specific
doesn't add anything to an enclosing generic region.
Assisted-by: Claude via Antigravity
Address some review comments from the review of PR#7935 on the changes
from PR#7933.
Consolidate `const_cast`s to a single place with an explanatory comment.
Improve test clarity and organization. Rename lookup-specific helper
function to avoid suggesting it is more broadly applicable.
Importing a function with a `-> val` return form, such as `fn F() -> val
i32`, crashed because `TryResolveInstCanonical` had no case for
`ValueForm`. Also, an imported `RefForm` was given type `type` rather
than `Core.Form`.
Import all three primitive forms with a single function template.
Assisted-by: Claude via Antigravity
When exporting `X` from Carbon package `Foo`, expose it to C++ with
fully-qualified name `Carbon::Foo::X`, not simply `Carbon::X`. This
ensures that C++ mangled names properly distinguish between same-named
Carbon entities in different packages.
Name *lookup* into `Carbon::` still uses the same rules as before: it
finds what a `package.` lookup would find in Carbon. So `X` is named
from C++ as `Carbon::X` still, and a lookup for `Carbon::Foo` fails,
even though `X`'s "true name" is now `Carbon::Foo::X`.
Assisted-by: Gemini via Antigravity
When unqualified lookup finds a name from an enclosing generic, create a
`specific_constant` if it's a constant, or a `specific_inst` otherwise,
to refer to that name from the inner generic. This allows consumers of
generic instructions to always use a specific of the current scope's
generic when accessing the constant value or type of a dependent
instruction.
Fixes crashes in dominance and in `GetExprCategory` caused by using
specifics for the wrong generic.
Assisted-by: Gemini via Antigravity
The return type is deduced from the function's `return` statements
(currently required to all have the same type), or from its first
`returned var`. Calls to generic functions with deduced return types are
not yet supported.
When a function has a deduced return type, we defer the return type
portion of the callee pattern match until deduction, and during
deduction we extend the decl block and the pattern block with the
return-related instructions.
Conversions for `return expr;` statements are deferred until the return
type has been deduced. The design requires that we take the common type
of the returned types as the deduced return type, so we can't in general
perform any conversions for returned expressions until we reach the end
of the function. This is accomplished by inserting a `splice_block`
where the conversions will go, and later updating that instruction to
contain the conversion.
Assisted-by: Claude Opus 5.5 via Antigravity
If Clang encounters a lookup failure, it caches that result, creating
problems with `-share-cpp-ast` where a lookup failure in one file
effectively poisons the same name when looked up in a different file.
Also, if Clang triggers loading a complete name table (which happens,
for example, during typo correction, but Clang could do that any time it
wants to), it clears out the "has external visible results" flag, which
means that introducing new names in Carbon and then attempting to use
them from C++ after that point does not work.
Fix both of these problems by re-enabling the "has external visible
results" flag on callbacks that indicate Clang just cleared it, and by
not caching failed name lookups into Carbon scopes in Clang's lookup
table. (They still get cached in *our* lookup table.)
Assisted-by: Gemini via Antigravity
We previously discarded a `generic` applied to `self`, resulting in
silently treating `generic self: Self` as a runtime parameter and
producing a weird error on `template self: Self`.
Instead of forcibly making all `self` parameters non-runtime, change the
*default* for a parameter named `self` to be runtime.
This is only a partial fix; more work is required to support `fn
F(generic self)` with an omitted type.
Change the `cursor` parameter in `Core.Iterate.Next` to be a `ref`
parameter
instead of a pointer parameter.
Assisted-by: Gemini via Antigravity
---------
Co-authored-by: Geoff Romer <gromer@google.com>
If the verifier fails, we tried to dump the SemIR::File, but
accidentally dumped a pointer to it instead. However, the raw file dump
is huge and unwieldy, and I don't think it's really helpful, so instead
of including it here, we now just omit the pointer.
Support for `fn` expressions with the same feature set as `fn`
declarations. In particular, there is no support for captures,
positional parameters, `=>` bodies, or `auto` return types.
Note that this results in two parse nodes for each `=>` token. That's a
bug, but it matches the behavior of function declarations, and will be
fixed in a follow-up change.
Assisted-by: Gemini via Antigravity
As described in proposal #4682, make `array(T, N)` be syntactic sugar
for the type `Core.Array` defined in the standard library. This provides
a home for impls that are specific to `Core.Array`.
Assisted-by: Gemini and Claude via Antigravity
No support for `each` yet, but this should be enough that we can try out
an implementation approach supporting expanding a statement across
elements of a tuple without building "real" variadics.
Assisted-by: Claude via Antigravity
Update tree-sitter grammar to match toolchain, plus a couple of features
the toolchain doesn't yet implement: `like`, `each`, `expand`, `friend`.
Assisted-by: Claude via Antigravity
If a test would still fail post-autoupdate due to either mismatched
`fail_` prefix or due to being NOAUTOUPDATE, the test is reported as `?`
instead of `.` in the incremental progress output, and a message is
listed after the progress output describing the problem. For example:
```console
Problems that require manual fixes:
- `toolchain/check/testdata/basics/zz_prefix_probe.carbon` split `bad_missing_prefix.carbon` failed; if failure is expected, add the `fail_` prefix.
- `toolchain/driver/testdata/fail_config.carbon` succeeded; if success is expected, remove the `fail_` prefix.
```
This change caused the `NOAUTOUPDATE` test
`toolchain/driver/testdata/fail_config.carbon` to fail, because when run
under autoupdate (or with `bazel run`), the install manifest would be
present, and the test is checking that it's not. Fixed by using the VFS
to look for the manifest, which then required fixing
toolchain/driver:driver_test to set up a real filesystem VFS so that it
*could* find the manifest.
With this change, there should no longer be a reason to manually run
file_test in addition to running autoupdate.
Assisted-by: Claude via Antigravity
Extend the design's keyword list with the keywords from all the approved
proposals to date. Update our four syntax highlighters to have matching
keyword lists.
There are a few other stale things in some of the syntax highlighters;
those are not fixed here.
Assisted-by: Claude via Antigravity
Implements `typeof(expr)`, as described in #7697. `expr` is treated as
an unevaluated operand, and is kept in an `ExprRegion` separate from the
enclosing scope.
Assisted-by: Claude via Antigravity
Check that every use of a non-constant instruction is dominated by a
definition of that instruction. Remove the fake (instruction creation
order based) dominance checks in convert; these start spuriously failing
during template instantiation of initializers.
Assisted-by: Claude Opus 5 via Antigravity
We've made quite a few changes to how associated constants are
implemented since this was written; update the doc to match.
Assisted-by: Claude via Antigravity
Reconstruct the call syntax from the callee's explicit parameter
patterns, the callee specific, and the call arguments.
Assisted-by: Claude via Antigravity.
If a line or column is unknown, we internally represent it as line or
column -1. When mapped from our 1-based numbering to LSP's 0-based
numbering, it comes out as -2, which is out of range since LSP requires
lines and columns to be >= 0.
Detect this case and produce a fallback.
Assisted-by: Claude via Antigravity
Add support for deferring initialization as a template action, and
performing the deferred initialization during template instantiation.
This is substantially more complex than other conversion actions, for
two primary reasons:
* The initializer in the generic may have storage arguments as inputs.
We model an initializing expression as having a "slot" where
initialization writes the location that should be initialized by that
initializing expression, and that needs to be an output of the
initialization action.
* Initialization from a tuple or struct literal needs to recurse into
that literal, and the literal will have been spelled in the generic,
meaning we don't have an `InstId` that can be used to name the specific
version of the initializer as input for nested conversions.
These issues are addressed by introducing two new features to the action
machinery:
In addition to `InstAction`, we now have `MultiInstAction`, which is an
action that produces a tuple of instruction values instead of a single
instruction value. Initialization actions produce one instruction for
the final result, which is spliced at the point of initialization, plus
one instruction for each storage argument, which are spliced into the
storage argument slots in the original generic. During initialization,
if we find one of those splices in the storage argument of an
initializing expression, we return the new storage argument back to the
initialization action to be included in the specific, instead of
overwriting the storage argument in the generic.
Actions whose `PerforrmAction` takes a `SpecificId` as input no longer
perform automatic refinement of their operands to specific instructions.
Instead, the action is given control over when and where it performs
that refinement. In `InitializeAction`, we use this freedom to form a
`SpecificInst` for the initializer in the primary output block, and form
a `SpecificInst` for the target in the target block. When detecting
whether we are initializing from a tuple or struct literal, we step over
the `SpecificInst` and track its `SpecificId`, and if necessary create a
new `SpecificInst` wrapping the sub-initializer when we recurse into the
nested element conversion.
Assisted-by: Claude and Gemini via Antigravity
Add hover cards and jump to declaration / definition / reference for the
formatted SemIR that appears in check tests. This is done by adding a
heuristic "parser" for SemIR to the language server. The
cross-references are strictly best-effort, since this is just a tool for
Carbon developers, not a user-facing facility.
A couple of other changes made along the way:
* file_test tests with an AUTOUPDATE-SPLIT no longer look for CHECK:
lines outside that split. This was motivated by the tests for this new
facility including CHECK: lines as part of the test input.
* An agent skill for working on the language server, tracking some
things that cost Claude time when working on this.
Assisted-by: Claude via Antigravity
When a generic impl uses a default or final fn, it picks the specific
function value out of the interface to put in the witness table.
However, because this is done by modifying an existing instruction
block, the generics machinery has no hook to convert the function
constant into an attached constant, and because it was found in a
specific for a different generic, the constant inst will be unattached.
Fix this by manually mapping to an attached constant inst in the current
generic when building the witness table.
In the `EntityName` for a binding, preserve the `TypeInstId` describing
how the type was written. When a diagnostic refers to that type via
`TypeOfInstId`, use the type-as-written in the diagnostic rather than
the canonical type.
Assisted-by: Claude Opus via Antigravity
Remove claim from script that it takes minutes; this caused Claude to
decide to not run it. It only takes a few seconds these days. Add note
in toolchain development skill that lints won't be accurate if the
compilation database is outdated.
Assisted-by: Claude Opus 5 via Antigravity
This action was created to wrap any `MetaInstId` operand of an action
instruction. This served two purposes:
1) It had a special hook in `OperandIsDependent` to allow it to be
performed while it had a dependent operand (the reference to the
instruction in the generic).
2) It created a `specific_inst` so that the downstream action saw an
instruction in the specific instead of one in the generic.
These are both replaced: the special case in `OperandIsDependent` for
`RefineInstAction` is replaced by a special case for `MetaInstId`s in
general, and the `SpecificInst` is now created as part of performing the
downstream action, rather than as a separate step carried out
beforehand.
This simplifies the produced SemIR and reduces the number of splices
significantly. It also prepares us to handle actions like
initialization, where we don't actually want to create `SpecificInst`s
immediately in the location where the action is performed, because they
actually belong somewhere else in the IR.
Assisted-by: Claude Opus 5 and Gemini via Antigravity
Replaces several quadratic-time steps in the algorithm with linear or n
log n implementations. Worst-case runtime before hitting the complexity
bailout drops from 25ms to about 12ms, and typical runtime is
single-digit ms on my development machine.
Assisted-by: Claude Code
Instead of always printing types as canonical, attempt to find a sugared
type where possible, and include that type in the diagnostic. We can
only do this when given the instruction whose type is being printed
(`TypeOfInstId`) rather than the canonical type ID.
Initial support here is intentionally minimal: just looking through
calls to the callee's declared return type, and looking through pointer
dereferences and corresponding pointer types, to build out the initial
infrastructure. More cases can be added later; this degrades gracefully
to using the canonical type if a better type can't be found.
Assisted-by: Claude Opus 5 via Antigravity
We use the same conversion codepath to handle both qualification
conversions and derived-to-base conversions, because we allow both to be
performed at once. However, we were previously modeling the
qualification conversion as happening *first*, and producing a result
whose type is the target type of the overall conversion (that is, the
base class type). That led to bogus SemIR, where a `Derived` -> `const
Base` conversion would first have a "compatible" conversion from
`Derived` to `const Base`, *then* an access of the base subobject (of
type `const Base`, within an object of type `const Base`).
We now reverse the order: first we do a derived-to-base conversion,
which already has logic to preserve qualifiers, and then we do any
necessary qualification conversions on the result to reach the overall
target type.
In passing, we now skip forming the `as_compatible` instruction at all
for a pure derived-to-base conversion that has no qualification
conversion, simplifying the SemIR by one instruction in the common case.
Instead of running on the "current" `@` (the state at which the previous
`jj` command happened to be run), intentionally trigger a working copy
snapshot from `jj_prek`.
Assisted-by: Claude Opus 5 via Antigravity
A check for `args.verbose` was missing for one line of autoupdate's
verbose output. Add a wrapper function to print verbose output to fix
this and to make it easier to get verbose output right in future.
This prevents different template instantiations from getting
over-eagerly merged. Unfortunately we don't have a good middle-ground
yet, and this effectively disables all merging for templates. We may be
able to find some smart way to fingerprint spliced instructions so that
we can still merge template instantiations, but for now this change is
just fixing the wrong-code bug.
Test passing a template argument or a symbolic argument to a template.
Fix a bug in the template case where we'd crash when instantiating a
dependent discarded expression, because conversion produced an
`InstId::None` which the actions machinery did not expect and crashed
on.
Add basic support for lowering templates: we can now lower `SpliceInst`
in the case where the generic and specific are from the same file (and
we don't support importing templates from other files yet in general).
In order for this to work, lowering needs to be able to query the
expression category, and to handle instructions that appear to be
(template) constants in the generic but turn out to be non-constant in
the specific, so support for that is added.
Switch `type_of_inst` from being added as an action inst to being added
as a normal inst, since it's not an action and the old approach led to a
crash in lowering.
Replace `refine_type_action` with `refine_inst_action`, and generate a
`specific_inst` instead of an `as_compatible` to represent the specific
version of an instruction that's used as an input to a template action.
This gives us a place to handle other properties of the instruction that
might vary from generic to specific beyond its type, such as its
constant value and its expression category.
For now, we provide a non-template-dependent constant value to the
`specific_inst` in addition to the non-template-dependent type we have
traditionally provided. This doesn't seem to matter for any current
actions, but sets us up to better handle future actions. The
`specific_inst` representation also allows downstream consumers of the
instruction to track which specific they should be requesting
information from. Providing a correct expression category for
`specific_inst` will be handled in a future PR.
Don't wait until we reach the end of the eval block to set the value
block on the specific. This is a prerequisite for allowing template
actions to read from the specific.
Also add a default for `EqWith.NotEqual`.
Switch advent examples to use these named constraints, and also go
through all the other TODOs in the advent examples and fix the ones that
are trivially fixable now.
Per https://arxiv.org/pdf/2602.11988 (section 4.3), repository overviews
have no effect on the time it takes agents to find files, but do
increase the cost of operations and number of required steps to complete
tasks.
Two somewhat related fixes. The first is call-specific for now (because
it's the first action to take a `MetaInstId`), and the second is general
across all actions, but it seems like calls are the easiest place to hit
it.
1) Add support for refining inst blocks as action operands. Refine all
the insts in the block, using the appropriate InstId-derived type.
2) When an action operand is a `MetaInstId` referring to an unattached
constant, form a corresponding attached constant. This comes up when
forming (for example) an implicit `AddWith(%T)` call, where the `%T`
operand is an unattached constant.
This causes us to form correct specifics in more cases, where previously
we formed specifics that contained values that were still
template-dependent.
This unfortunately causes some existing template tests to produce more
errors, but those errors reflect cases where we were previously silently
doing the wrong thing.
`GetCallee` is sometimes called on a spliced instruction, and is
checking its exact inst operand to see if it's a `BoundMethod`. This
fails if the `BoundMethod` is wrapped in another instruction, such as a
splice. Normally our approach for such a situation would be to
constant-evaluate the operand, but that doesn't work here because the
`BoundMethod` will be non-constant if its bound `self` is. So instead we
now step through splice instructions manually when looking for the
`BoundMethod`.
Use a version of Brent's algorithm, suitably adapted to work for our
worklist-based graph traversal, to very cheaply detect if instruction
fingerprinting fell into a cycle and terminate cleanly with a dump of
the cycle.
The algorithm does not immediately catch when we enter a cycle, but is
guaranteed to catch it eventually (generally after running through the
cycle no more than twice).
With this set, autoupdate will print messages indicating what it's
doing, such as the command that it's invoking to execute bazel. For
example:
```console
$ ./toolchain/autoupdate_testdata.py -v toolchain/check/testdata/basics/empty.carbon
Detected --compilation_mode: fastbuild
/home/zygoloid/carbon-lang/scripts/run_bazel.py run -c fastbuild --experimental_convenience_symlinks=ignore --ui_event_filters=-info,-stdout,-stderr,-finish //toolchain/testing:file_test -- --autoupdate --print_slowest_tests 0 --file_tests=toolchain/check/testdata/basics/empty.carbon
[... normal output ...]
```
Follow the pattern used by eval_inst.h's `EvalConstantInst` to generate
declarations of an overload set that handles some but not all typed inst
classes. The pattern is:
* A template computes the signature to use for a particular overload,
producing a fallback `() -> void` signature for overloads that should
not exist.
* The `.def` file is used to generate a declaration per instruction
kind, whose signature is generated by the template.
* The `() -> void` signature that all the "should not exist" cases
generate is explicitly deleted.
This avoids the redundancy of manually declaring all the overloads, as
we did for `PerformAction`, and is less error-prone as it both catches
signature errors and definitions of overloads that are dead code and
should not exist, as it did for the `FacetAccessType` overload of
`LowerInst`.
We track a "next" index into each bucket when we insert instructions.
The insert loop effectively shifts each element in the "next" vector
left by one place, so if we instead start the bucket counts shifted one
place to the right, we can use the same vector for "next" and for the
bucket start indexes.
Assisted-by: Claude Code
---------
Co-authored-by: Geoff Romer <gromer@google.com>
We previously passed in the start of file token, classified as
BracketTokenKind::Other, which allowed the bracket fixer to consider
corrections where it inserted tokens (such as a `{`) *before* the
start-of-file token.
Fixes#7672.
When a template action is created, any (non-meta) instruction operand
will refer to instructions in the corresponding generic, or possibly to
a constant. This means that when the action is eventually executed when
forming a specific, it would see the generic value for that operand
rather than the intended specific value.
Fix this by refining `InstId` operands to refer to a corresponding value
in the specific, much like we would when rebuilding a constant in the
eval block.
Generalize ConvertToValue template action to handle other kinds of
conversion target that don't perform initialization. Initializing
conversions will need more work since they also need to use a splice to
form the storage block.
Fix a bunch of cases where we use the same external name to mean
multiple different things in the same test. We've historically gotten
away with this, but under `--share-cpp-ast`, it becomes an error, at
least if the entity is either defined in, or used from, C++ code.
Assisted-by: Gemini via Antigravity (original change) and Claude Code
(suggested edits in review)
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This allows various templated constructs to get further through
checking, but typically we hit another unsupported action such as a
conversion or call, so it's not enough to make much work.
Adds an algorithm to compute where to insert brackets to repair
bracketing mismatches during lexing. This takes indentation, as well as
a number of other cues, into account to predict where the brackets
should have gone. Detects when there is ambiguity between solutions and
makes no suggestion in that case. Reduces the problem by splitting on
properly bracketed top-level constructs, then uses a beam search to find
good candidate solutions quickly.
This includes both a fuzzer and an eval tool that can be used to
determine how well the algorithm fares against a given corpus of valid
Carbon code, by damaging it in various ways and seeing whether the
algorithm can correctly fix it. On all the eval modes, this algorithm
can correctly infer the positions for over 80% of lost brackets (and can
correctly restore 95+% of brackets in some modes), with low rates of
incorrect suggestions.
See added documentation for full details.
Assisted-by: Gemini via Antigravity, Claude via Claude Code
It's not enough for field types of Carbon classes to be complete in
SemIR. If the field is exported to Clang, we also need the type to be
complete in Clang's AST, since Clang assumes it has a definition
available for the types of all fields of a complete class.
Add support for "jump to declaration", "find references", type
information on hover. This support is strictly single-file for now; only
references and declarations within the same file are found. We could go
a bit beyond that, but to properly handle cross-file references we'll
need to build an index and a compilation database, which is beyond the
scope of this change.
On hover, we provide the type information for the instruction under the
cursor as-is. This is frequently not very useful, as the type of a
function F is simply "<type of F>", but is a starting point for richer
information.
Assisted-by: Claude Code
For certain kinds of error, clang's parser will succeed but produce an
expression marked as "contains error". Clang's constant evaluator
asserts if given one of those, so return early if we encounter one.
We currently provide UTF-8 positions, since our source representation is
UTF-8 and we use byte offsets as column numbers, but the LSP protocol
default is UTF-16 column positions. Negotiate UTF-8 positions where
possible; this is supported by essentially every LSP client other than
VS Code.
In the case where we select a UTF-16 position, we continue to not do any
actual conversions, so our column offsets in such cases will continue to
be wrong on non-ASCII source files in VS Code.
Assisted by: Claude Code
This suppresses warnings for unhandled messages where we currently have
nothing to do. No functionality change, except for less spam in the VS
Code output tab.
Assisted-by: Claude Code
When computing the token range of a declaration, we were using the first
and last parse nodes to determine the first and last tokens. That's not
correct -- the parse tree nodes can be in a different order from the
tokens, so the first parse node need not be at the start of the
declaration. For a malformed declaration such as `fn f` (with no
terminator), the first and last parse nodes were both associated with
the `fn` token for error recovery, meaning that the function name wasn't
even within the symbol we handed back to the LSP client, in violation of
the LSP requirements. This led to VS Code producing an error and
discarding all symbols in the file.
Assisted-by: Claude Code
If we find an overload set containing mulitple methods, discard any
non-const methods and try again. This allows libc++'s `std::vector` to
be iterated with range-based for.
We can't rely on a `clang::CodeGenerator` existing when compiling C++
code; we don't build one unless we're actually emitting code for the
current file any more.
Add `Core.CppCompat.[U]Long64` to represent a 64-bit long that is not
`i64`. Treat it as being "just slightly smaller than" `i64`, like we
treat `Core.CppCompat.LongLong64` as being "just slightly larger than"
`i64`, so that we get implicit conversions `Cpp.long` -> `i64` ->
`Cpp.long_long` on all targets.
This follows the direction of proposal #5448, and seems like the obvious
extension of the `[U]Long32` and `[U]LongLong64` types added in #6275
for targets of this "shape".
Assisted-by: Gemini via Antigravity
Clang treats "file not found" as a fatal error and stops emitting more
diagnostics after reaching it, so these tests don't work in
`--share-cpp-ast` mode if they are all in the same file. So split them
into distinct test files.
Make multiple imports of the same header only parse it once per C++
domain. Reuse of the same header in `--share-cpp-ast` mode now reuses
the representation.
Importing a Carbon file with C++ dependencies now makes those transitive
C++ dependencies in the same C++ domain visible too.
Assisted-by: Gemini via Antigravity
When a split file in a test unexpectedly succeeds or fails, include the
test filename in the error as well as the name of the split. This should
make it a bit easier to figure out which test failed from a failing test
log.
Move `--output-last-file-only` and output filename synthesis logic out
of the general-purpose compile driver and into the `carbon compile`
subcommand, which is the only thing that should be using them. Track on
CompilationUnit whether it is being lowered, or whether it exists only
to be imported into other units.
`carbon compile` now never lowers inputs that it discovered for itself,
only inputs that were specified on the command line. In particular, it
doesn't lower (and throw away the result of lowering) the prelude any
more. This makes the toolchain tests about 10% faster in my crude
measurements.
Also, we now do not create a clang `CodeGenerator` for input files that
we are not lowering, similarly saving compilation time for units that
exist only to be imported, not lowered.
One minor change: we use the same mechanism to determine whether an
input is being lowered and to determine what the output filename is.
This means that `--phase=lower` and `--phase=optimize`, which lower but
don't produce an output file, still need an output filename to be
specified now in some cases. Given those are just debugging tools, I
think that's fine.
Assisted-by: Gemini via Antigravity
Instead of creating a CodeGenerator per CppDomain, and then crashing in
lowering when we try to consume the same llvm Module multiple times,
create a CodeGenerator for each CppFile within the domain.
For now, we mulitplex all of Clang's ASTConsumer output to all code
generators, which means that any strong external definitions within a
Carbon file (for example, in an inline `Cpp` fragment) will be emitted
to all output files in the same `CppDomain`, resulting in link errors
due to symbol redefinitions. This will be addressed later. But this
should be sufficient for Carbon compilations in which such symbols are
not defined.
We also don't yet attempt to classify which compilations will need C++
code generation, and instead create a clang `CodeGenerator` for every
Carbon file that has C++ imports. For `carbom compile`, only one Carbon
file will need code generation, and yet we still build multiple
`CodeGenerator` objects in general. Fixing this requires more plumbing
from the driver, and this will also be handled in a follow-up.
Assisted-by: Gemini via Antigravity
This isolates the C++ imports in different Carbon files from each other
in `--share-cpp-ast` mode, so that a Carbon file can only see the
portions of the shared Clang `ASTContext` that it actually imported.
Assisted-by: Gemini via Antigravity
Inject the name of a macro rather than its contents when computing its
expansion. If the macro refers to itself, it will not expand within its
own body, rather than expanding once.
Switching from `EnterTokenStream` to `EnterToken` exposed that our Clang
preprocessing environment was a little broken -- we reached the end of
the primary source file and starting tearing stuff down before we
actually finished parsing, which we were mostly getting away with before
but aren't any more. Enabled Clang's incremental processing mode to fix
this. This causes Clang to remain in the main source file when it
reaches EOF instead of popping it. This also causes the diagnostics for
invalid `module;` declarations to change, but in a way that seems not
really any worse than before.
Also slightly changes the diagnostics produced from macro expansion
failures. The new diagnostics are a bit more precise -- they now capture
the outermost level of macro expansion -- but we don't do a good job of
rendering the Clang snippet attached to the "in macro expansion" context
note yet, so the context looks a bit weird: we get two different
snippets attached to the same diagnostic.
Start tracking the domain within `CppContext`s instead of having them
duplicate its fields. This allows us to remove the shared ownership of
the clang parser.
---------
Co-authored-by: Christopher Di Bella <cjdb.ns@gmail.com>
Since we started bootstrapping, our stage-1 toolchain binaries have been
inputs to stage-2 compilations, and to the compilation database logic,
they're indistinguishable from generated sources. This caused us to get
compilation command lines for them, which clangd would try to index by
parsing the binary, and would either run incredibly slowly or crash.
Filter out non-source files from the compilation database.
In passing, also emit the JSON dump without whitespace, which makes the
database a bit smaller and faster for clangd to parse.
Assisted-by: Gemini via Antigravity
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Don't CHECK-fail if the file contains mismatched braces and generates an
unbalanced parse tree.
Properly balance the start and end of symbols. This previously caused
errors to be reported in clients such as VS Code.
Compute a correct range for definitions. Don't assume we can find the
matching `}` for an opening symbol, since some of our definitions are
delimited by a `;` instead.
Assisted-by: Gemini via Antigravity
Debian packages github.com/libunwind/libunwind as `libunwind-dev`, and
packages LLVM libunwind as `libunwind-N-dev`, with no meta-package to
install the latest version of LLVM libunwind. The libunwind-dev is not
built as PIC, so doesn't work in our build setup. So a specific version
of LLVM libunwind must be installed.
Instead of building one Clang `ASTContext` per compilation, the
`--share-cpp-ast` flag causes us to build a single `ASTContext` and
share it across all contexts. One new abstraction is added: `CppDomain`
represents the Carbon-side view of a Clang AST that might be shared
across multiple `SemIR::File`s. This object owns the Clang instance and
the AST.
For now, we have no isolation between the C++ state exposed to different
Carbon compilations, and we have no multiplexing of generated LLVM IR
from C++ into different Carbon compilations, so the mode is not usable
yet. The plan is to keep it behind a flag until it's ready.
Assisted-by: Gemini via Antigravity
Split the diagnostic emitter into a separate emitter (regietered with
Clang) and listener (registered with the emitter). The purpose of this
split is to make the Clang emitter not depend on the `Check::Context`,
so that we can use it, and hence the same Clang instance, with multiple
`Check::Context`s. A fallback listener is registered to collect and emit
any diagnostics produced while we don't have a `Check::Context`
registered with the emitter.
Assisted-by: Gemini via Antigravity
Richard Smith
