Refactors the link driver to automatically compile and cache the carbon
prelude for use in linking.
Implements a `carbon_library` rule for compiling the Core library
dependencies in the examples.
We can't use a `CallExpr` to call a constructor; use a
`CXXConstructExpr` instead. While this fixes the crash and gets us past
the initial constant evaluation, we still can't map the constant value
back into Carbon, so this doesn't actually make constexpr constructors
work yet. But it does stop Clang from crashing.
Fixes#7498.
Instead of tracking the cleanup scope depth on entry to each scope,
track an "ambient" cleanup scope depth that's *after* the destructors of
local variables in that scope. This gets increased to include the
destructors of local variables when we create a name-binding
declaration. Then, when we reach a point where temporaries should be
destroyed, run cleanups that are after the ambient cleanup scope depth
on the stack. This happens:
* At the `;` of a statement expression.
* At the `)` of an `if` or `while` statement.
* After performing the implied `HasValue()` call in a `for` statement.
Per informal agreement with leads, this means we lifetime-extend all
temporaries created in the initializer of a name-binding declaration to
the full scope of that declaration, but that temporaries created in an
expression statement are destroyed at the `;`.
Allow (don't diagnose) impls that have the same type structure if they
are associated with the same match_first block. Similarly, allow final
impls (made final by their enclosing match_first block) that overlap in
type structure when they are associated with the same match_first block.
If the type structures are the same, choose the first impl from the
match_first block.same.
If the type structures overlap but are not the same, and share a
match_first block, then we should choose the first overlapping impl from
the match_first block. This is true both for final and non-final impls.
See
https://github.com/carbon-language/carbon-lang/blob/de8b03faa3178ae683d8e7124fbcba81eb88e00c/proposals/p005337-interface-extension-and-final-impl-update.md#impl-selection-algorithm
But in the non-final case if there is an overlapping impl outside the
match_first, it can win if it's more specific.
When a Carbon virtual function overrides a C++ virtual function, we need
to export it with the C++ signature in order for it to work as an
override. Instead of mapping the C++ signature into Carbon and then back
again, use the original C++ signature from the base class as the
signature exported to C++.
Also add documentation explaining how we use thunks in C++ interop,
including in this new virtual function handling logic.
Replace the llvm::map_range based anonymous ranges returned by values()
and enumerate() with custom range and iterator types. This avoids
exposing complex template return types and improves encapsulation. But
most importantly, it is *much* cheaper to compile.
This change:
- Defines ValueStoreIterator, ValueStoreEnumerateIterator, and
ValueStoreRange templates in the Internal namespace.
- Exposes them via aliases in ValueStore: Iterator, ConstIterator,
Range, MutableRange, and EnumerateRange.
- Supports safe implicit conversion from Iterator to ConstIterator.
- Supports operator-> even when the value type is returned by value
(e.g., llvm::StringRef) by conditionally const-qualifying the pointer
type.
- Ensures C++20 comparison consistency with custom operator<=> and
operator== definitions.
- Restricts construction of these iterators and ranges to ValueStore
methods by using private constructors and friend declarations.
Assisted-by: Antigravity with Gemini
Previously the last decl had to be the definition. Now we allow a
declaration after a definition, so that the user can write a match_first
block last, and put (re-)declarations of impls in it, after the
definitions have already been written elsewhere.
We track the location of the decl that was associated with a match_first
block so that we can correctly point to it in diagnostics when an impl
is written twice in match_first blocks. Since impls may not be
redeclared across an import boundary, we will never have a `SemIR::Impl`
with a match_first from a different file in a redeclaration, so we don't
need to import the location of a previous decl that was in a match_first
for diagnostics. As such we just store a LocId on the `SemIR::Impl`
struct.
The `//@include-in-dumps` and `//@dump-sem-ir-begin`/`-end` tooling
directives were consumed for their side effects without a comment
record, so the tokens and comments together no longer reconstructed the
source: tooling that re-emits a file from them, such as `carbon format`,
silently dropped the directive lines. Now each recognized directive line
is also recorded as an ordinary full-line comment alongside its side
effect.
Adjacent full-line comments coalesce into one comment record only within
a category, determined by the byte after the `//` introducer: ordinary
comments (whitespace, or the end of the line or file), `//@...`
directives, and invalid introducers. A transition between categories
starts a new record, so a directive next to a comment block is its own
comment, while all the invalid spellings lump together to keep the
diagnostic noise at one per run.
The category boundary also fixes a lost directive: the invalid-comment
bulk skip compared only the `//` prefix, so `//!x` directly above
`//@dump-sem-ir-begin` absorbed the directive line and its side effect
was never recorded. Invalid comment runs now skip line by line (they
start from a diagnosed error, so they are not hot) and stop at a
whitespace or `@` introducer; the SIMD bulk skip handles only ordinary
comment blocks, whose prefix comparison already includes the whitespace
byte.
Nothing outside the lexer and the formatter reads comment records, and
lex dumps do not include comments, so no other behavior changes.
Assisted-by: Claude Code
Remove toolchain/parse/typed_nodes.h from transitive imports of tree.h
and node_stack.h to reduce compile-time overhead in toolchain/check.
Add an explicit include of typed_nodes.h to tree_and_subtrees.cpp where
it is needed for instantiating templates.
Assisted-by: Antigravity with Gemini
Remove clang/Frontend/CompilerInvocation.h from check_unit.h and forward
declare clang::CompilerInvocation instead to reduce header exposure.
Add an explicit include of CompilerInvocation.h to check_unit.cpp.
Assisted-by: Antigravity with Gemini
Remove clang/CodeGen/ModuleBuilder.h from toolchain/lower/file_context.h
to reduce transitive dependencies. Forward declare clang::CodeGenerator
and clang::FunctionDecl instead.
Add an explicit include of ModuleBuilder.h to context.cpp where the
complete types are required.
Assisted-by: Antigravity with Gemini
Because we merge cleanups across scopes in various cases, and want to
use linear indexes into the complete stack, the ArrayStack abstraction
is getting in the way more than it's helping. Switch to just a
SmallVector.
This loses the unit testing of the MergeIntoGrandparent logic. This is
covered indirectly by check tests still, but direct testing of it is a
bit tricky given that ScopeStack isn't set up for use without a Context.
Add extern template declarations and explicit template instantiations
for ValueStore<InstId, Inst> and ValueStore<EntityNameId, EntityName> to
prevent redundant template instantiations.
Place them in their respective domain files (inst.h/cpp and
entity_name.h/cpp) to maintain modularity.
Assisted-by: Antigravity with Gemini
The type of `.Self` introduced by `where` may contain a `.Self` inside
it. Freeze the type so that we have a consistent view of `.Self` inside
the facet type, where they are all frozen.
The type of `.Self` only has extend constraints from the LHS of the
`where`. So we need to copy any non-extend constraints into the
`where_stack` so they are available as early-impls and can be used for
impl lookups on the RHS of the where. We need to freeze any `.Self`
references in these just as we do for rewrite constraints.
Destroy local variables and temporaries at each `}`, and when branching
with `break` and `continue`. In `for` statements, destroy loop variables
along with anything created within the loop at the end of each loop
iteration, and destroy the cursor and range object when the loop
terminates.
Assisted-by: Gemini via Antigravity
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Fix a few API issues. There's also a newly-added file in compiler-rt
that is not supposed to be built by default but is not being excluded
properly by a glob. Added a patch to exclude that and sent
https://github.com/llvm/llvm-project/pull/208861 upstream.
Instead of manually creating a map from symbolic types to concrete
types, create a Specific and look up parameter types via that Specific.
This allows C++ to call a Carbon function like `fn F[T: type](unused t:
T*) {}`. See generic_pointer.carbon.
Document the phase of associated constants, which the discussions of
contextual phase defaults never covered: an associated constant is
always a checked generic binding. This is deliberately not presented as
a contextual default, because no other phase is possible for the
construct; correspondingly, no phase keyword (including `template`) is
allowed on one. Also rework the associated constants section of the
design README where the migration left a non-sequitur ("set to
compile-time values ... and so are defined using a `let` declaration"):
describe the `let` syntax and the binding's contextual phase separately.
Use consistent terminology for bindings versus constants. Bindings are
"checked" or "template" (generic) bindings, replacing the "symbolic
binding" and bare "generic binding" terms, so "symbolic" now only
describes constants and values. The "Symbolic facet bindings" section of
the generics details becomes "Checked facet bindings". The
expression-phase terms "symbolic constant" and "template constant" are
unchanged, and the binding-pattern definitions now name the constant
each kind binds. Template bindings are additionally described as
dependent and late checked, with each instantiation providing the
binding's value.
Also normalize the remaining "regular parameter" mentions to "runtime
parameter" to match the binding terminology, and describe the
compile-time "let template" as introducing a template generic binding
"T: C" whose uses are template constants.
Assisted-by: Claude Code
When we check an `impl` decl, find the containing `match_first` block,
if any, and store a connection to it in the `SemIR::Impl` structure,
along with the impl's position in that `match_first` block so that we
can sort/prioritize the `SemIR::Impl`s later.
Also, update the `is_final` flag if the `match_first` block is modified
as `final`. But ensure we diagnose trying to put a `final impl`, or a
redeclaration of one, in a `match_first` block. Also diagnose if an impl
is attached to a `match_first` block more than once - either in two
different blocks or in the same block at different positions.
Putting the `match_first` connection on the `SemIR::Impl` structure
means we have to import it, so implement import and add a smoke test for
that, which ensures nothing explodes.
Drop the `scope_stack` entry for the `match_first` block, as it was not
needed. Once we started tracking the `match_first` size on the `Context`
class, it became more straightforward to just store the `match_first`
decl `InstId` in the same place. The `match_first` block is not supposed
to act like a different scope for the purpose of redecls anyhow, so it's
a bit simpler this way.
It must be in a namespace, function, or class. In particular, it can't
be in another match_first, doing so immediately associates any impl
inside with two different match_first scopes, but we only want them
associated with one for prioritization.
Push a scope_stack entry for match_first blocks, and handle impls being
inside those scope entries. An impl should not use the `match_first`
block as its "enclosing scope" for the purpose of deciding if the impl
is a redecl of another impl. We should look through it to the class or
namespace the impl (and match_first) are located inside.
We don't yet store the relationship between the impl and its match_first
block, nor then can we use it in impl lookup for prioritization.
Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
When formatting an impl definition, include in the textual output if the
impl is final. Since we (mostly) write them as `final impl` in the code,
use the same notation in the textual semir.
The implicit/explicit `.Self` concept is a heuristic at best, so we
should avoid relying on it. The ImplicitOnly option is no longer used,
as it was used to remove/disambiguate `.Self` in nested facet types, but
we have banned nesting `where` on the RHS of a `where`. So we no longer
have to worry about ambiguous `.Self`.
We can remove all the designator tracking heuristics in `.Self`
substitution as well now, as they were used for the now-removed
ExplicitOnly (removed in
https://github.com/carbon-language/carbon-lang/pull/7460) and
ImplicitOnly options.
The `style = "fixed"` is partially implied by `indent = 4`, but not
fully in some specific cases. There were also bugs in fully applying it
that are now fixed, and so we can specify it explicitly to get rumdl to
canonicalize nested list indentation more thoroughly.
Assisted-by: Antigravity with Gemini
The self and interface given to TryFindMatchingWitnessFromImplLookup
come from the output of SubstPeriodSelf, so there is no need to do the
substitution again. And the self came from a LookupImplWitness
instruction so it is already fully canonicalized.
The implicit/explicit `.Self` concept is a heuristic at best, so we
should avoid relying on it. We now only have one value of `.Self` in a
facet type (we have banned nested `where` on the RHS of a `where`). So
we don't need to preserve any `.Self` for the purpose of disambiguation,
and we can subst all `.Self` on the the RHS of a rewrite constraint.
`LookupImplWitness` instructions inside the impl declaration can't use
the impl they are apart of. Previously we had an heuristic in eval which
would try to prevent finding the impl for a lookup from inside that
impl. But it breaks when the `.Self` is replaced in a generic impl with
a symbolic, and then that symbolic is replaced in a specific. The
specific's decl block contains that `LookupImplWitness` instruction and
it tries to use the impl it came from. This causes the same specific to
be formed again, but now it exists, so it's used as-is but it has no
decl block yet, and so we crash.
Now we ban an impl while we resolve its specific, both deduction of its
arguments and from any other substitution. The prevents instructions
from inside the impl (which are evaluated when resolving the specific)
from finding their own impl. We do so by adding the ImplId to a stack on
the Context, and then skipping such impls when looking for candidates
during eval.
This fixes a crash, which was demonstrated by the new test being added.
It also makes another todo test pass.
There's a whole lot of other semir churn, which seems to be mostly
reordering of constants. There are some fingerprint changes in
constants, but it appears they are the same canonical instructions, so
they don't represent a behaviour change. For example in
`toolchain/check/testdata/for/actual.carbon` the `%N.patt` constant has
been given its fingerprint suffix now as `%N.patt.aa5`. But they are
both this instruction, so it is just a formatting change:
```
inst6100001A: {kind: SymbolicBindingPattern, arg0: entity_name61000002, type: type(inst61000018)}
- name: `N`
- type: type(inst61000018): <pattern for Core.IntLiteral>; {kind: PatternType, arg0: inst(IntLiteralType), type: type(TypeType)} (concrete)
- value: symbolic_constant61000001
```
I attempted the TODO as stated but we can't remove the `.Self` from the
LHS of a rewrite right now, without causing evaluation to run and
potentially find a concrete value to replace the access with, which then
breaks the association with the associated constant. There's a separate
TODO about that in SubstPeriodSelfInFacetType.
`.Self` will only be replaced in a facet type, as the facet type
constrains a facet. If it's part of a (non-facet) type, then the object
of that type is not a facet, and we can never replace that `.Self`.
`match_first` is a declaration followed by a curly-brace block of
declarations.
The check phase will ensure only impl decls (or defns) are inside the
block.
Clarify that the functions that search for a witness in a facet type no
longer take inputs from the query directly, but now take them from an
identified facet type constructed from the query. That means the self
type is already canonicalized.
We used to have to batch these at the end of pattern traversal in order
to avoid accidentally adding them to a block that was speculatively
pushed for an expression within a pattern, but that's no longer a
concern with the more precise handling of those speculative blocks in
#7445.
In #7436 we stopped substituting `.Self` when collecting witnesses out
of a facet type. While this was correct, it did not capture all the
cases that need to avoid substituting `.Self`. And it poisoned the
`IdentifiedFacetType` cache by not replacing `.Self` but storing the
result in the cache. This led to incoherent behaviour, where the result
of an impl lookup would change depending on which ones had been done
previously.
Now we use a flag to track for each `.Self` if we're currently
type-checking inside the scope where it was introduced in a facet type.
While inside that scope, identify should not replace the `.Self`. Any
use of it should remain as-is since we don't yet know what value will
replace it. We call this state "frozen" since it should not be modified
by identify. This requires a substitution step when we leave the scope
that introduced the `.Self`, to remove the flag. The flag is set in the
`EntityName` of the `SymbolicBinding`, and is part of the canonical
value, since `.Self` can become part of types, which are constants, and
the flag needs to follow it for correct behaviour.
We also have to ensure the flag is the same when doing comparison with
constants from inside a facet type and constants from outside. For
instance in `(Z where .Z1 = ()) where .Z2 = .Z1`, when we arrive at the
second `.Z1` its `.Self` will be frozen, while the `.Z1 = ()` contains a
non-frozen `.Self`. So we add the frozen flag to the first when storing
it in `where_stack` in order to compare the constant values of the two
`.Z1`.
The `WhereExpr` requirement inst kinds now have an `InstConstantKind` of
`AlwaysUnique` instead of `Never`. This allows us to add them to the
usual InstBlocks, and in an `eval fn` body they have a constant value,
so eval does not fail when trying to call that function. We have to be
careful to not consider `AlwaysUnique` as being actually concrete
though, since their constant value erases `.Self`-dependence. This
allows us to stop special casing them when thawing the requirements
block in a `WhereExpr`, and we can just thaw each `InstId` in the block
in a straightforward manner.
We add the new flag to the instruction's fingerprint and name in
formatted semir.
This adds SemIR structs and implements building `observe` lists, as well
as naming, formatting, and importing `observe` declarations.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
This allows C++ to call Carbon functions with generic type parameters,
with some conditions. Example:
```carbon
interface I {
fn Doit(self);
}
class A {
impl as I { fn Doit(unused self) {} }
}
class B {
impl as I { fn Doit(unused self) {} }
}
fn F[T:! I](t: T) {
t.Doit();
}
inline Cpp '''
void G() {
Carbon::A a;
Carbon::B b;
Carbon::F(a);
Carbon::F(b);
}
''';
```
The initial support is limited; only explicit parameters are handled
currently.
`CarbonExternalASTSource::GetOrExportFunctionToCpp` now generates a
`clang::FunctionTemplateDecl` for generic Carbon functions. If C++ code
attempts to call that templated function,
`CarbonExternalASTSource::LoadExternalSpecializations` will be called
with the template argument types of that call site. Then we can generate
a specialized thunk for those argument types for C++ to call.
This prevents an issue when trying to check in changes top
bazel-generated template files,
which aren't actually valid C++, making clangd_tidy issue errors and
block the commit.
This follows the pattern of our main compile benchmarks, but instead of
running the compile through a library API, it does so by running a
separate process. This lets us see the performance that we would expect
from `make` or another build system invoking the compiler, as opposed to
a minimal view from the library API.
Assisted-by: Claude Code
In some cases the pattern block can depend on the initializer, so it
must be sequenced after it. See #7469 for a more detailed explanation of
why this is necessary.
Each file dump now starts with a `; ---` comment and ends with a blank
line. This makes it easier to visually scan the dump for a file of
interest. The comment format is somewhat arbitrary; I chose `---` to
align with the `--- filename.carbon` separator in SemIR dumps, but
without the filename, because that appears on each of the next two lines
already.
In `SourceGenTest.IdentifierByteSumStableAcrossSeeds`, the `first`
variable was storing `llvm::StringRef`s pointing to memory allocated by
a temporary `SourceGen` instance. This memory was freed at the end of
the loop iteration, leaving `first` with dangling references that were
accessed in subsequent iterations.
Fix this by storing `std::string` copies of the identifiers in `first`
to own the memory, and use `llvm::equal` to compare them.
Assisted-by: Antigravity with Gemini
Lucile Rose Nihlen
Richard Smith
Dana Jansens
Chandler Carruth
Nicholas Bishop
Özgür T. ÖnsoyandDana Jansens