The rules in early impls are stored with frozen `.Self`. However when
forming a specific, we can perform an impl lookup query against a facet
type written outside of the current facet type, which means any `.Self`
in that facet type will _not_ be frozen. In order to compare a witness
from early impls with the non-frozen requirement, we need to thaw what
we get out of the early impls.
However there are also cases where we are doing a lookup against
requirements that are part of the current facet type, so the
requirements are frozen.
Rather than picking one, and having to thaw all impl lookup queries, for
example, we store the frozen and thawed copy of the early impls rule so
that we can satisfy lookup queries that involve either.
We allow using `InstId` as a `LocId` for diagnostics, which works great
in `handle_*` functions, where we are working with non-canonical insts.
And convert and pattern matching often are given non-canonical insts,
which they can use, or pull apart to find some other inst, and use those
as a location for diagnostics. However these operations can also be
called with canonical insts, which have no location. In that case if we
need to diagnose an error, it ends up not being attached to any code at
all. Since we always have a `LocId` somewhere up the call stack, pass it
along and in these places that handle canonical instructions too, use
`GetInstLocationOrFallback()` to get a valid location for diagnostics.
This is also similar to what eval does with its fallback `LocId`.
The result is that many diagnostics now have a location where they had
none before. It also means some insts don't get `<elided>` in formatted
semir for not having a location at all and thus not being in range.
Diagnostic rendering was part of the data model: `Loc::FormatLocation`
and
`Loc::FormatSnippet` wrote plain text to a stream, so no capability,
color, or
terminal width could reach them.
A diagnostic is now one message with labels, contexts, and location
information
attached, and `Diagnostics::Renderer` draws it into a
`Terminal::Buffer`. The
design is in `toolchain/docs/diagnostics_rendering.md` (#7668),
authoring in
`toolchain/docs/diagnostics.md`.
Before:
```
example.carbon:6:3: error: 1 argument passed to function expecting 0 arguments
Run0(1);
^~~~~~~
example.carbon:3:1: note: calling function declared here
fn Run0() {}
^~~~~~~~~~~
```
After:
```
error: 1 argument passed to function expecting 0 arguments
╭─┤ example.carbon:6:3
┆
3 │ fn Run0() {}
· ─────┬─────
· ╰──┤ calling function declared here
┆
-> 6 │ Run0(1);
· ───────
╰────────────
```
`--no-diagnostic-snippets` gives a compact form, `example.carbon:6:3-9`.
File
tests use it; the rendering tests under `toolchain/diagnostics/testdata`
ask for
snippets.
C++ interop diagnostics are data now rather than text Clang drew:
`SemIR::ClangSourceLocId` names a `clang::SourceRange`, and a Clang
location is
marked as the point it names.
`StreamConsumer` moves to a `:renderer` target that `:emitter` does not
depend
on, keeping `common/terminal` out of `consumer.h`. Capabilities come
from the
file behind the error stream, so tests get plain ASCII.
`renderer_fuzzer` draws diagnostics built from fuzzer bytes with every
capability combination, since a shape the renderer doesn't expect must
never be
why the compiler dies while reporting a problem.
TODOs mark the follow-ups: line weights and crossing lines, wrapping
that keeps
a quoted type whole, and ordering diagnostics that share a byte offset.
Also: `IncorrectExtension`'s hint moves into its message, since its note
had no
location of its own, and `strip_noise` in `check_diagnostics.py` blanks
multi-line `#define`s instead of deleting them.
Assisted-by: Claude Code
When unqualified lookup finds a name from an enclosing generic, create a
`specific_constant` if it's a constant, or a `specific_inst` otherwise,
to refer to that name from the inner generic. This allows consumers of
generic instructions to always use a specific of the current scope's
generic when accessing the constant value or type of a dependent
instruction.
Fixes crashes in dominance and in `GetExprCategory` caused by using
specifics for the wrong generic.
Assisted-by: Gemini via Antigravity
The return type is deduced from the function's `return` statements
(currently required to all have the same type), or from its first
`returned var`. Calls to generic functions with deduced return types are
not yet supported.
When a function has a deduced return type, we defer the return type
portion of the callee pattern match until deduction, and during
deduction we extend the decl block and the pattern block with the
return-related instructions.
Conversions for `return expr;` statements are deferred until the return
type has been deduced. The design requires that we take the common type
of the returned types as the deduced return type, so we can't in general
perform any conversions for returned expressions until we reach the end
of the function. This is accomplished by inserting a `splice_block`
where the conversions will go, and later updating that instruction to
contain the conversion.
Assisted-by: Claude Opus 5.5 via Antigravity
Normally ImplSelfWitness would be replaced in an impl decl by the time
we are constructing the witness table. If it was part of an access to an
associated constant, rewrite constraint resolution would have replaced
that access with the value assigned to the associated constant already.
But we can use `.Self` in places where it converts and forms a
FacetValue, which preserves the ImplSelfWitness in the witness block of
the FacetValue. In that case, we want to replace it with the impl's
ImplWitness before we put it into the witness table.
This is a followup fix for
https://github.com/carbon-language/carbon-lang/pull/7930 which
introduces the CARBON_FATAL incorrectly, now that we know how to run
into it.
Add some tests that cover these situations, including one that tries to
access using the FacetValue and its witness, which fails, as a TODO
test.
Rewrites in `extend requires` are used to set the witness table values
of the impl. Rewrites in non-extend `requires` are treated as same-type
constraints and must be true at the point of the impl definition.
When identifying a facet type, defer evaluation of LookupImplWitnesses
by redirecting them to evaluate into IdentifiedWitness. Then the caller
of identify will need to substitute the IdentifiedWitnesses, either with
witnesses of their own, or with LookupImplWitness to cause a lookup
operation.
This allows:
- An `impl as` on a named constraint can subst the witnesses in
designators from the named constraint with a witness to the impl itself.
This prevents them from looking for and finding some other overlapping
impl. (This PR)
- Impl lookup to collect rewrite/equality requirements from all named
constraints and then use the impls it has already found to evaluate any
designators in those requirements instead of performing impl lookup
again (creating a cycle). (Done in
https://github.com/carbon-language/carbon-lang/pull/7944)
- Similarly, an `impl as` can collect impls and equality requirements
from all named constraints and verify them correctly by replacing
witnesses that should refer to the impl with the correct witness instead
of performing a lookup and finding an overlapping impl. (Done in
https://github.com/carbon-language/carbon-lang/pull/7937)
- ImplWitnessAccess to be able to find rewrites from a named constraint,
not just from the initial DeclaredFacetType. (TBD)
To do this we need to introduce the concept of "identified" specifics.
In order to identify a facet type, we must construct a specific for each
require decl in a named constraint. These specifics will have
IdentifiedWitnesses inside them, but their args would otherwise match a
specific for the require decl constructed in another context. To avoid
polluting the canonical specific namespace, we introduce a second
namespace of specifics that are "identified" since they are created by
identifying.
As we have introduced a new global state that modifies eval behaviour,
we must disable that state when resolving a specific (as per
https://github.com/carbon-language/carbon-lang/pull/7923). But inside an
"identified" specific, we preserve the behaviour so that the witnesses
modified by identify (through SubstPeriodSelf or through evaluating a
require specific) have their evaluation deferred.
We set the `declaring_impl_decls` state to modify the evaluation of
LookupImplWitness instructions formed inside an `impl as` decl's
constraint. However while checking that constraint, we may form other
specifics through calls to generic classes, etc. Specifics are canonical
and are keyed by their arguments and the generic ID. This means every
time we see the same arguments for a specific, we must produce the same
instructions. The `declaring_impl_decls` state can break that, causing
instructions inside the specific to be formed in a way they wouldn't at
another time.
To fix this, remove the effect of `declaring_impl_decls` while resolving
a specific. This keeps the specific's evaluation consistent so that the
key truly represents the canonical specific for its arguments.
This doesn't break anything. Existing LookupImplWitnesses (such as in a
class decl) are not guaranteed to point to an `impl as` when they are
modified by a specific. It is designators in the `impl as` constraint
that are guaranteed as such. Designators in an `extend require impls`
statement in a named constraint will be pointed to the `impl as` through
another mechanism in a followup, which will make the
`impl_as_named_constraint.carbon` test pass again.
We perform the same canonicalization as in LookupImplWitness, which
makes the output of eval more consistent across witness types, and
enables more straightforward constant comparisons between them when
needed.
As described in proposal #4682, make `array(T, N)` be syntactic sugar
for the type `Core.Array` defined in the standard library. This provides
a home for impls that are specific to `Core.Array`.
Assisted-by: Gemini and Claude via Antigravity
Base classes will be destroyed in a dedicated change, so we can
trivially confirm that the base is being destroyed.
This is a partial implementation of #7362.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This allows us to collect rewrites from named constraints and use them
to initialize the witness table for an `impl as` statement.
Only rewrites from extend constraints are tracked, as other constraints
should turn into equality constraints, as they don't modify the witness
table.
This change partially implements [PR #7362], which revises how objects
are destroyed. It is a partial implementation for two reasons:
1. This change moves `Destroy.Op`'s current behaviour into
`Destroy.SubobjectDestroy`, but it doesn't add support for objects with
non-trivial destruction.
2. `Destroy.SubobjectDestroy` is a workaround for `require impls
SubobjectDestroy`. We aren't able to use the latter until the dependents
add their requirements' implementations to their own witness tables.
[PR #7362]: https://github.com/carbon-language/carbon-lang/pulls/7362
The type `type` is now a `FacetType` inst with no constraints. This
brings the model implemented in the toolchain into better alignment with
the language design. The `SemIR::TypeType` struct remains as a scope for
holding the `TypeInstId`, `ConstantId`, and `TypeId` constants, but is
not an `InstKind` anymore.
The `TypeType` inst looks a lot like singletons, but there are many
`FacetType` insts so it doesn't quite fit that model. So we put it
alongside singletons with a fixed inst id but refer to it as a more
general "builtin" inst that is not a singleton.
`Namespace::PackageInstId` is similar, and we group it with `TypeType`
conceptually as another builtin instruction with a fixed id.
No conversion is needed anymore to use a `type` as a facet, since types
also have a `FacetType` type. This simplifies and removes a number of
helpers and branches throughout the code.
The `TypeType` inst is now part of the constant store, so we end up
printing it in the constants block in every test. But it's also named
`type` rather than `%type` to preserve the majority of existing
formatting behaviour, though this does look different from other
constants.
Assisted-by: Opus 5 was used to generate a first draft and validate the
refactoring. Though nearly everything non-trivial the tool wrote has
been modified or rewritten.
When evaluating a deferred member access action, the scope stack cannot
be relied on, so `LookupUnqualifiedName` cannot be used in
`GetHighestAllowedAccess` to get the `Self` type.
Instead, store the `Self` type in the `Context` when evaluating a
method, and use that in `GetHighestAllowedAccess`.
Instead of always printing types as canonical, attempt to find a sugared
type where possible, and include that type in the diagnostic. We can
only do this when given the instruction whose type is being printed
(`TypeOfInstId`) rather than the canonical type ID.
Initial support here is intentionally minimal: just looking through
calls to the callee's declared return type, and looking through pointer
dereferences and corresponding pointer types, to build out the initial
infrastructure. More cases can be added later; this degrades gracefully
to using the canonical type if a better type can't be found.
Assisted-by: Claude Opus 5 via Antigravity
We use the same conversion codepath to handle both qualification
conversions and derived-to-base conversions, because we allow both to be
performed at once. However, we were previously modeling the
qualification conversion as happening *first*, and producing a result
whose type is the target type of the overall conversion (that is, the
base class type). That led to bogus SemIR, where a `Derived` -> `const
Base` conversion would first have a "compatible" conversion from
`Derived` to `const Base`, *then* an access of the base subobject (of
type `const Base`, within an object of type `const Base`).
We now reverse the order: first we do a derived-to-base conversion,
which already has logic to preserve qualifiers, and then we do any
necessary qualification conversions on the result to reach the overall
target type.
In passing, we now skip forming the `as_compatible` instruction at all
for a pure derived-to-base conversion that has no qualification
conversion, simplifying the SemIR by one instruction in the common case.
Use a single `SemIR::Function` per `Core` interface method, whether it's
generated locally or imported. This prevents generating duplicate
functions, which lead to different types when the witness appears in a
`FacetValue` as part of a specific for a class.
We use a `CanonicalValueStore` of `GeneratedFunction` objects that allow
finding an existing FunctionId for a `Generated` special function before
(re-)generating it. Mangling for `Generated` functions is also moved to
use the values from the `GeneratedFunction`'s canonicalization key, so
that we have a consistent source of truth for the unique ID of a
`Generated` function across all files.
New tests are in
`toolchain/check/testdata/impl/custom_witness/destroy.carbon`.
Don't wait until we reach the end of the eval block to set the value
block on the specific. This is a prerequisite for allowing template
actions to read from the specific.
This adds most support for default and final methods. Missing components
include rejecting definitions for non-default/final methods, and
permitting out-of-line definitions.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Implementing interface modifiers causes an infinite loop when generating
fingerprints because the witness value generates a fingerprint that's
dependent on something dependent on the witness value. We've debugged
this to the witness table's `elements_id` field.
This hack is a workaround for creating a new block type whose value is
not codependent with its identity.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The toolchain crash is easily diagnosed, but looking at the log doesn't
offer immediate insight into why the program crashes. This provides us
with a graceful exit.
When implementing a named constraint, like `impl as N`, any accesses
through `Self` in the named constraint need to get the value of the
associated constant from the impl's witness table. This isn't possible
immediately, since the impl does not even exist until the declaration is
complete. We use the same model as for accesses found directly in the
impl declaration, but applied to the point where accesses in the named
constraint are substituted during identify to point at the impl's self
type. To get there, we need LookupImplWitness instructions in the named
constraint, when re-evaluated during construction of their enclosing
specific, to evaluate to ImplSelfWitness when they are a reference to
the type and interface being implemented.
In Convert, we require the source value's type to be compete so that we
can look for `base` classes and `adapt` relationships. However these can
only be present in a `ClassType`, so we only need `ClassType`s to be
complete.
Reduce the requirement in Convert to not complete types that are not a
`ClassType`, and which can not contain a `ClassType` as part of their
class.
Ideally we would only _only_ require the `ClassType` itself to be
complete, and only if we're looking for a base or adapt. However lower
depends on us completing all Convert source types that contain a class.
This seems to suggest we're lacking checks for complete types somewhere
else and Convert is making up for it. A TODO has been added. The
`toolchain/driver/testdata/compile/optimize/optimize_debug.carbon` test
is an example that CHECKs due to failing to verify the LLVM module if we
do not compute the complete type of all class-containing types in
Convert.
The critical step this PR is doing is to stop trying to complete a
`FacetType` when converting from a facet. This avoids trying to complete
a named constraint when converting `Self` inside that named constraint.
Doing so causes a cycle when the conversion of `Self` is performed in
eval of an `extend require` decl, since requiring the named constraint
to be complete re-evaluates the `extend require` decl again. A test is
added that crashed in an infinite loop before this change.
It also depends on #7584, which was intended to be an optimization but
is now load bearing. Because converting `Self` leaves an impl lookup
inst behind, and if that inst is re-evaluated inside impl lookup (by
forming a specific of a `require` decl through identify) then we have a
similar cycle.
During impl lookup, for each (generic) impl candidate, we form a
specific for that impl by deducing its generic arguments. Then we
compare the query interface against the impl's specific interface. That
comparison needs the deduced arguments applied to the impl's specific
interface. Previously we were doing this by getting the impl's
constraint facet type with the impl's specific applied (via
`GetConstantValueInSpecific()`) and then identifying that facet type
with the impl's deduced self.
Identify is a fairly expensive operation. It runs subst, trying to
replace `.Self` references. It walks named constraints. It collects
require declarations. We're looking at making it do _more_ in the future
too, including rewrite constraint resolution and collecting rewrite and
same-type constraints. For this reason we have a cache to make it cheap
on the second run, but it's still a very heavyweight operation to
involve in impl lookup, when all we want is to apply the impl's specific
to its target interface.
We almost have all the information we need to avoid the identification
step. We have the impl's specific after deduction. And we have the
SpecificInterface that the impl is targeting in the `Impl` struct. When
we form the specific for the impl itself, we resolve the declaration
block and form new constant values for all instructions in there, but
that does not cover the SpecificInterface that we're storing in the
`Impl` struct. So we add a new instruction to the impl's eval block,
which will be symbolic when the impl is generic and the target interface
depends on a generic parameter. And we store the `InstId` in the `Impl`
struct. This allows us to gets its constant value later with the impl's
specific applied. From that constant value we can then pull out the
SpecificInterface that the impl is targeting.
Point symbolic witnesses into `.Self` written inside an impl decl at the
impl that is being declared. This is tricky because the impl does not
yet exist. So we use a new instruction `ImplSelfWitness` which _will_ be
replaced by the `ImplWitness` once it becomes available. The
`ImplSelfWitness` acts like a symbolic witness, except it does not
perform lookup, since we know which impl we will get a witness from.
This prevents us from finding other impls when performing lookups into
`.Self` in an impl decl, which produces incorrect/incoherent results.
`DiagnoseOrphanImpl` used `definition_id`, but #7140 defines the anchor
as the first owning declaration, so a class declared but not defined was
rejected, which is why three cases in `orphan.carbon` were marked
`fail_todo`, and they now pass.
`fail_use_extern_class` no longer errors, `handle_class.cpp` never
passes the `extern library` name into the class entity, so `C` is
treated as locally owned and counts as an anchor. The expected error is
replaced with a TODO in the test, but it should come back once `extern
library` is implemented for classes.
We checked that requirements inside the impl-as target interface were
satisfied. But we also need to check that requirements coming from the
constraint facet type, or named constraints that it targets, are
satisfied.
This supports impl lookup choosing an impl that targets a generic
interface through a named constraint, without crashing.
Instead of assuming the impl's target is a facet type containing an
interface, we use the identified facet type to find the specific
interface it targets.
Refactors the link driver to automatically compile and cache the carbon
prelude for use in linking.
Implements a `carbon_library` rule for compiling the Core library
dependencies in the examples.
Instead of tracking the cleanup scope depth on entry to each scope,
track an "ambient" cleanup scope depth that's *after* the destructors of
local variables in that scope. This gets increased to include the
destructors of local variables when we create a name-binding
declaration. Then, when we reach a point where temporaries should be
destroyed, run cleanups that are after the ambient cleanup scope depth
on the stack. This happens:
* At the `;` of a statement expression.
* At the `)` of an `if` or `while` statement.
* After performing the implied `HasValue()` call in a `for` statement.
Per informal agreement with leads, this means we lifetime-extend all
temporaries created in the initializer of a name-binding declaration to
the full scope of that declaration, but that temporaries created in an
expression statement are destroyed at the `;`.
Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
When formatting an impl definition, include in the textual output if the
impl is final. Since we (mostly) write them as `final impl` in the code,
use the same notation in the textual semir.
`LookupImplWitness` instructions inside the impl declaration can't use
the impl they are apart of. Previously we had an heuristic in eval which
would try to prevent finding the impl for a lookup from inside that
impl. But it breaks when the `.Self` is replaced in a generic impl with
a symbolic, and then that symbolic is replaced in a specific. The
specific's decl block contains that `LookupImplWitness` instruction and
it tries to use the impl it came from. This causes the same specific to
be formed again, but now it exists, so it's used as-is but it has no
decl block yet, and so we crash.
Now we ban an impl while we resolve its specific, both deduction of its
arguments and from any other substitution. The prevents instructions
from inside the impl (which are evaluated when resolving the specific)
from finding their own impl. We do so by adding the ImplId to a stack on
the Context, and then skipping such impls when looking for candidates
during eval.
This fixes a crash, which was demonstrated by the new test being added.
It also makes another todo test pass.
There's a whole lot of other semir churn, which seems to be mostly
reordering of constants. There are some fingerprint changes in
constants, but it appears they are the same canonical instructions, so
they don't represent a behaviour change. For example in
`toolchain/check/testdata/for/actual.carbon` the `%N.patt` constant has
been given its fingerprint suffix now as `%N.patt.aa5`. But they are
both this instruction, so it is just a formatting change:
```
inst6100001A: {kind: SymbolicBindingPattern, arg0: entity_name61000002, type: type(inst61000018)}
- name: `N`
- type: type(inst61000018): <pattern for Core.IntLiteral>; {kind: PatternType, arg0: inst(IntLiteralType), type: type(TypeType)} (concrete)
- value: symbolic_constant61000001
```
In #7436 we stopped substituting `.Self` when collecting witnesses out
of a facet type. While this was correct, it did not capture all the
cases that need to avoid substituting `.Self`. And it poisoned the
`IdentifiedFacetType` cache by not replacing `.Self` but storing the
result in the cache. This led to incoherent behaviour, where the result
of an impl lookup would change depending on which ones had been done
previously.
Now we use a flag to track for each `.Self` if we're currently
type-checking inside the scope where it was introduced in a facet type.
While inside that scope, identify should not replace the `.Self`. Any
use of it should remain as-is since we don't yet know what value will
replace it. We call this state "frozen" since it should not be modified
by identify. This requires a substitution step when we leave the scope
that introduced the `.Self`, to remove the flag. The flag is set in the
`EntityName` of the `SymbolicBinding`, and is part of the canonical
value, since `.Self` can become part of types, which are constants, and
the flag needs to follow it for correct behaviour.
We also have to ensure the flag is the same when doing comparison with
constants from inside a facet type and constants from outside. For
instance in `(Z where .Z1 = ()) where .Z2 = .Z1`, when we arrive at the
second `.Z1` its `.Self` will be frozen, while the `.Z1 = ()` contains a
non-frozen `.Self`. So we add the frozen flag to the first when storing
it in `where_stack` in order to compare the constant values of the two
`.Z1`.
The `WhereExpr` requirement inst kinds now have an `InstConstantKind` of
`AlwaysUnique` instead of `Never`. This allows us to add them to the
usual InstBlocks, and in an `eval fn` body they have a constant value,
so eval does not fail when trying to call that function. We have to be
careful to not consider `AlwaysUnique` as being actually concrete
though, since their constant value erases `.Self`-dependence. This
allows us to stop special casing them when thawing the requirements
block in a `WhereExpr`, and we can just thaw each `InstId` in the block
in a straightforward manner.
We add the new flag to the instruction's fingerprint and name in
formatted semir.
This adds SemIR structs and implements building `observe` lists, as well
as naming, formatting, and importing `observe` declarations.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Dana Jansens
Chandler Carruth
Richard Smith
Geoff Romer
Christopher Di BellaandRichard Smith
Nicholas Bishop
Lucile Rose Nihlen
arhwx
Özgür T. ÖnsoyandDana Jansens