When impl lookup results in an `ImplWitness`, either directly or later
through `LookupImplWitness` evaluating to an `ImplWitness` inside some
specific, it represents a dependency on the eval blocks of the
`ImplDecl` that the `ImplWitness` comes from. The `ImplWitness` contains
a `SpecificId` of the `ImplDecl`, and the constants in the witness table
must be viewed through that `SpecificId`. The constants may depend on
the declaration or the definition block of the `ImplDecl`'s specific, so
any use of the witness table from `ImplWitness` requires the specific
from the `ImplWitness` to be fully resolved: both the decl and
definition blocks.
The specific declaration block is always resolved immediately when
forming a specific id, so there's nothing to do there. And things are
simple when the `ImplWitness` returned from impl lookup is concrete. We
would simply resolve the specific definition directly in the evaluation
of `LookupImplWitness`.
But when the `ImplWitness` is symbolic, meaning there's a dependency on
generic parameters in its referenced specific, then we have a dependency
on resolving the specific definition in the `ImplWitness` for every
specific that gets applied to the `ImplWitness`, eventually resulting in
a concrete specific.
Previously we attempted to do this by representing a dependency edge
from the eval block containing the `ImplWitness` instruction to the
definition block of the specific inside the `ImplWitness`. We
represented this dependency edge with the `RequireSpecificDefinition`
instruction.
The challenge with the `RequireSpecificDefinition` instruction was
ensuring it always landed in the _same eval block_ as the one holding
the symbolic `ImplWitness`. Ultimately, we've found a way to do this
without the `RequireSpecificDefinition`. The witness table is accessed
through `ImplWitnessAccess`, so we resolve the specific definition
before each access to the witness table. If the definition is already
resolved, this just costs a lookup of the Specific from the value store.
We document the other things we tried, which don't work or were too
complex, below.
In PR #6236, we added the `RequireSpecificDefinition` instruction in the
eval of `LookupImplWitness`, but this is too late. If impl lookup
resolved to a symbolic `LookupImplWitness`, the `LookupImplWitness` is
added some eval block. Later, we form specifics and re-evaluate that
eval block, and the `LookupImplWitness` can evaluate to an
`ImplWitness`. Adding the `RequireSpecificDefinition` instruction at
this point inserts the instruction into whatever generic's eval block
happens to be on the top of the generic stack, not the eval block of the
generic we are building a specific for. And indeed we can't add
instructions to a generic's eval block later like this, as it would be
missing from specifics that were already resolved for the generic. This
incorrect insertion shows up as a crash in eval at the CHECK introduced
but commented out in [eval.cpp in
#7981](https://github.com/carbon-language/carbon-lang/blob/56d9bbc40add1e9263f2f9b2555c3a27292d1725/toolchain/check/eval.cpp#L3507-L3512).
This is covered by the
`require_specific_impl_definition_through_specific_instantiation.carbon`
test.
We tried moving the insertion of `RequireSpecificDefinition` to be
earlier - at the point where the `LookupImplWitness` is constructed, in
the `LookupImplWitness()` function, instead of inside evaluation. This
seemed like a straightforwardly correct answer, as the eval block that
contains the `LookupImplWitness` will then also contain the
`RequireSpecificDefinition`. However, this can cause us to lose the
`RequireSpecificDefinition` entirely. Because it's a separate
instruction, not an operand of `LookupImplWitness`, it does not get
treated as a dependency of the witness. When we do an impl lookup inside
an `alias`, we build a generic for the alias and then discard it. This
discards the eval block containing the `LookupImplWitness`. Then, in an
enclosing generic, we refer to the constructed `AliasDecl` instruction,
which pulls all its dependent operands into the enclosing eval block.
This pulls in the `LookupImplWitness` into the enclosing eval block, but
it does not preserve the `RequireSpecificDefinition`. This is covered by
the
`require_specific_impl_definition_from_lookup_impl_witness_in_class_definition.carbon`
test.
We tried having `ImplWitness` evaluation resolve the specific definition
of the specific `ImplDecl` it refers to. This seems like the simplest
approach by far as we should always be working with the correct eval
block and nothing can get lost. However it does not work because of the
recursive nature of `ImplWitness`. The `ImplWitness` instruction appears
in the declaration of an `ImplDecl`, and the `ImplWitness` holds a
`SpecificId` for that same `ImplDecl`. When building a specific
`ImplDecl`, we evaluate the `ImplWitness` in the `ImplDecl`'s
declaration block which then immediately resolves the specific
definition before the declaration is finished. This leads to missing
instructions in the specific definition. In short, this puts the
dependency edge to the `ImplDecl`'s specific inside the `ImplDecl`
itself, which is problematic.
We tried wrapping `ImplWitness` in another instruction,
`ImplWitnessUse`. Inside the `ImplDecl` declaration there is still
`ImplWitness`, but then in the definition there is `ImplWitnessUse` and
that is what is returned from the evaluation of `LookupImplWitness`. The
`ImplWitnessUse` is an `ImplWitness` plus it ensures the specific
referred to by the witness has its definition resolved. However this
breaks constant comparisons, as we get different witnesses inside an
impl versus outside.
The first iteration of this PR's solution had us insert the
`RequireSpecificDefinition` into the eval block when we sniffed a
`LookupImplWitness` instruction, however that missed cases where impl
lookup immediately resolved to a symbolic `ImplWitness`, which is why we
now sniff for the `ImplWitness` in the eval block. This is covered by
the
`require_specific_impl_definition_from_symbolic_impl_in_class_definition.carbon`
test.
We also originally made a mistake of not inserting
`RequireSpecificDefinition` when inside the impl definition that the
`ImplWitness` refers to. But we can form specifics ImplWitness values
through function signatures, and these must add the dependency edge on
that specific ImplDecl definition. This is covered by the
`require_specific_impl_definition_from_inside_impl_definition.carbon`
test.
Finally, we did have a working solution. It was to sniff instructions
added to the eval block of a generic, and when an `ImplWitness` is added
to the eval block (it is implied that it is symbolic), also add a
`RequireSpecificDefinition` to that eval block immediately. We also
resolved the definition of a specific referred to by `ImplWitness` when
`LookupImplWitness` evaluates to `ImplWitness` to cover the concrete
witness case. This complicated the construction of the eval block
immensely, however. It caused a situation where adding one inst to the
eval block actually adds two, which we needed to avoid when importing.
And it required inst-kind specific behaviour in the very general
construction of an eval block, completely outside of eval itself.
The values in `IdentifiedFacetType` may contain `IdentifiedWitness`
instructions, and those insts need to be either converted to
`LookupImplWitness` and evaluated as such, or substituted by the user of
the `IdentifiedFacetType` with witnesses of its own.
This changes the API so that the user must go through a method doing one
of those two things to get to the underlying values in the
`IdentifiedFacetType`, unless they explcitly request to bypass them.
Then we split extend impls requirements out into their own set so that
the user can determine what is being extended _after_ evaluation or
substitution of `IdentifiedWitness` insts. This is critical because we
can not dedupe SpecificInterfaces until after that takes places, as
demonstrated by the test fixed by this PR.
`IdentifiedFacetType` is a concept that is only used by check. It does
not appear in the semir output, unlike `DeclaredFacetType`s.
`IdentifiedFacetType` is used in check to collapse named constraints
together, building a single structure of requirements from a
`DeclaredFacetType` and any referenced named constraints. This is used
for operations like impl lookup, and building `FacetValue`s.
The outputs of `IdentifiedFacetType` are now in an intermediate state,
where they contain `IdentifiedWitness`es, and need to be processed
before they are used. We'd like to build that into the API of
`IdentifiedFacetType` so that their use is more robust. But the
`IdentifiedWitnesses` need to be replaced by subst, which is a part of
check, so we need to move `IdentifiedFacetType` to check in order to
build the type on top of subst.
If we perform impl lookup during identify by resolving a specific, we
can run into IdentifiedWitnesses in a FacetValue. If the witness is the
only one in the FacetValue we may use it as a final witness, which is
not correct. IdentifiedWitness is not a final witness.
To test this, we add a test that places an IdentifiedWitness into a
FacetValue and then uses that facet value to get an associated constant.
If treated as a final witness, we try to do a lookup where the query
self contains the IdentifiedWitness and we crash coming across it in the
TypeIterator. We should never come across an IdentifiedWitness in an
impl lookup query.
The test also shows a case that should work but does not because of
`impl_lookup_no_symbolic_final_lookups`. We set
`impl_lookup_no_symbolic_final_lookups` while collecting witnesses to
avoid cycles in impl lookup. This test fails because of that. It
prevents us from forming a correct specific for the class `C` by
creating a specific where `C.C1` is a symbolic value. Later we try to
form the specific again, where we would see a concrete value for `C.C1`,
but the specific already exists, so we can't get to the concrete value.
The rules in early impls are stored with frozen `.Self`. However when
forming a specific, we can perform an impl lookup query against a facet
type written outside of the current facet type, which means any `.Self`
in that facet type will _not_ be frozen. In order to compare a witness
from early impls with the non-frozen requirement, we need to thaw what
we get out of the early impls.
However there are also cases where we are doing a lookup against
requirements that are part of the current facet type, so the
requirements are frozen.
Rather than picking one, and having to thaw all impl lookup queries, for
example, we store the frozen and thawed copy of the early impls rule so
that we can satisfy lookup queries that involve either.
Diagnostic rendering was part of the data model: `Loc::FormatLocation`
and
`Loc::FormatSnippet` wrote plain text to a stream, so no capability,
color, or
terminal width could reach them.
A diagnostic is now one message with labels, contexts, and location
information
attached, and `Diagnostics::Renderer` draws it into a
`Terminal::Buffer`. The
design is in `toolchain/docs/diagnostics_rendering.md` (#7668),
authoring in
`toolchain/docs/diagnostics.md`.
Before:
```
example.carbon:6:3: error: 1 argument passed to function expecting 0 arguments
Run0(1);
^~~~~~~
example.carbon:3:1: note: calling function declared here
fn Run0() {}
^~~~~~~~~~~
```
After:
```
error: 1 argument passed to function expecting 0 arguments
╭─┤ example.carbon:6:3
┆
3 │ fn Run0() {}
· ─────┬─────
· ╰──┤ calling function declared here
┆
-> 6 │ Run0(1);
· ───────
╰────────────
```
`--no-diagnostic-snippets` gives a compact form, `example.carbon:6:3-9`.
File
tests use it; the rendering tests under `toolchain/diagnostics/testdata`
ask for
snippets.
C++ interop diagnostics are data now rather than text Clang drew:
`SemIR::ClangSourceLocId` names a `clang::SourceRange`, and a Clang
location is
marked as the point it names.
`StreamConsumer` moves to a `:renderer` target that `:emitter` does not
depend
on, keeping `common/terminal` out of `consumer.h`. Capabilities come
from the
file behind the error stream, so tests get plain ASCII.
`renderer_fuzzer` draws diagnostics built from fuzzer bytes with every
capability combination, since a shape the renderer doesn't expect must
never be
why the compiler dies while reporting a problem.
TODOs mark the follow-ups: line weights and crossing lines, wrapping
that keeps
a quoted type whole, and ordering diagnostics that share a byte offset.
Also: `IncorrectExtension`'s hint moves into its message, since its note
had no
location of its own, and `strip_noise` in `check_diagnostics.py` blanks
multi-line `#define`s instead of deleting them.
Assisted-by: Claude Code
Instead of only getting rewrite/equality requirements from the declared
facet type, impl lookup now uses the full IdentifiedFacetType of the
query's constraint to get rewrite/equality requirements from named
constraints as well.
When identifying a facet type, defer evaluation of LookupImplWitnesses
by redirecting them to evaluate into IdentifiedWitness. Then the caller
of identify will need to substitute the IdentifiedWitnesses, either with
witnesses of their own, or with LookupImplWitness to cause a lookup
operation.
This allows:
- An `impl as` on a named constraint can subst the witnesses in
designators from the named constraint with a witness to the impl itself.
This prevents them from looking for and finding some other overlapping
impl. (This PR)
- Impl lookup to collect rewrite/equality requirements from all named
constraints and then use the impls it has already found to evaluate any
designators in those requirements instead of performing impl lookup
again (creating a cycle). (Done in
https://github.com/carbon-language/carbon-lang/pull/7944)
- Similarly, an `impl as` can collect impls and equality requirements
from all named constraints and verify them correctly by replacing
witnesses that should refer to the impl with the correct witness instead
of performing a lookup and finding an overlapping impl. (Done in
https://github.com/carbon-language/carbon-lang/pull/7937)
- ImplWitnessAccess to be able to find rewrites from a named constraint,
not just from the initial DeclaredFacetType. (TBD)
To do this we need to introduce the concept of "identified" specifics.
In order to identify a facet type, we must construct a specific for each
require decl in a named constraint. These specifics will have
IdentifiedWitnesses inside them, but their args would otherwise match a
specific for the require decl constructed in another context. To avoid
polluting the canonical specific namespace, we introduce a second
namespace of specifics that are "identified" since they are created by
identifying.
As we have introduced a new global state that modifies eval behaviour,
we must disable that state when resolving a specific (as per
https://github.com/carbon-language/carbon-lang/pull/7923). But inside an
"identified" specific, we preserve the behaviour so that the witnesses
modified by identify (through SubstPeriodSelf or through evaluating a
require specific) have their evaluation deferred.
We plumb through the ID so that we can later see if a requirement was in
the IdentifiedFacetType, and potentially grab a final witness from the
witness source if it's a FacetValue.
Instead of plumbing through the IdentifiedFacetTypeId, collect all the
requirements up front, and the final witnesses if they exist.
We want to do this because in future work we will need to subst/evaluate
the outputs of IdentifiedFacetType to use it since it will contain
IdentifiedWitnesses, and we'd like to do that once up front instead of
multiple places. That is more efficient and ensures less time/code with
access to the intermediate state of the IdentifiedFacetTypeId.
This statically ensures that identify always works with a constant
value, in the most restrictive way possible (ensuring it's a type not
just a constant).
This allows us to collect rewrites from named constraints and use them
to initialize the witness table for an `impl as` statement.
Only rewrites from extend constraints are tracked, as other constraints
should turn into equality constraints, as they don't modify the witness
table.
The type `type` is now a `FacetType` inst with no constraints. This
brings the model implemented in the toolchain into better alignment with
the language design. The `SemIR::TypeType` struct remains as a scope for
holding the `TypeInstId`, `ConstantId`, and `TypeId` constants, but is
not an `InstKind` anymore.
The `TypeType` inst looks a lot like singletons, but there are many
`FacetType` insts so it doesn't quite fit that model. So we put it
alongside singletons with a fixed inst id but refer to it as a more
general "builtin" inst that is not a singleton.
`Namespace::PackageInstId` is similar, and we group it with `TypeType`
conceptually as another builtin instruction with a fixed id.
No conversion is needed anymore to use a `type` as a facet, since types
also have a `FacetType` type. This simplifies and removes a number of
helpers and branches throughout the code.
The `TypeType` inst is now part of the constant store, so we end up
printing it in the constants block in every test. But it's also named
`type` rather than `%type` to preserve the majority of existing
formatting behaviour, though this does look different from other
constants.
Assisted-by: Opus 5 was used to generate a first draft and validate the
refactoring. Though nearly everything non-trivial the tool wrote has
been modified or rewritten.
We were passing a SpecificInterfaceId which just makes code have to do a
lookup to get the actual SpecificInterface. The caller already has the
SpecificInterface, so plumb that around.
SpecificInterfaceId really only exists when we need to stick a
SpecificInterface into an instruction as an operand.
When implementing a named constraint, like `impl as N`, any accesses
through `Self` in the named constraint need to get the value of the
associated constant from the impl's witness table. This isn't possible
immediately, since the impl does not even exist until the declaration is
complete. We use the same model as for accesses found directly in the
impl declaration, but applied to the point where accesses in the named
constraint are substituted during identify to point at the impl's self
type. To get there, we need LookupImplWitness instructions in the named
constraint, when re-evaluated during construction of their enclosing
specific, to evaluate to ImplSelfWitness when they are a reference to
the type and interface being implemented.
During impl lookup, for each (generic) impl candidate, we form a
specific for that impl by deducing its generic arguments. Then we
compare the query interface against the impl's specific interface. That
comparison needs the deduced arguments applied to the impl's specific
interface. Previously we were doing this by getting the impl's
constraint facet type with the impl's specific applied (via
`GetConstantValueInSpecific()`) and then identifying that facet type
with the impl's deduced self.
Identify is a fairly expensive operation. It runs subst, trying to
replace `.Self` references. It walks named constraints. It collects
require declarations. We're looking at making it do _more_ in the future
too, including rewrite constraint resolution and collecting rewrite and
same-type constraints. For this reason we have a cache to make it cheap
on the second run, but it's still a very heavyweight operation to
involve in impl lookup, when all we want is to apply the impl's specific
to its target interface.
We almost have all the information we need to avoid the identification
step. We have the impl's specific after deduction. And we have the
SpecificInterface that the impl is targeting in the `Impl` struct. When
we form the specific for the impl itself, we resolve the declaration
block and form new constant values for all instructions in there, but
that does not cover the SpecificInterface that we're storing in the
`Impl` struct. So we add a new instruction to the impl's eval block,
which will be symbolic when the impl is generic and the target interface
depends on a generic parameter. And we store the `InstId` in the `Impl`
struct. This allows us to gets its constant value later with the impl's
specific applied. From that constant value we can then pull out the
SpecificInterface that the impl is targeting.
Point symbolic witnesses into `.Self` written inside an impl decl at the
impl that is being declared. This is tricky because the impl does not
yet exist. So we use a new instruction `ImplSelfWitness` which _will_ be
replaced by the `ImplWitness` once it becomes available. The
`ImplSelfWitness` acts like a symbolic witness, except it does not
perform lookup, since we know which impl we will get a witness from.
This prevents us from finding other impls when performing lookups into
`.Self` in an impl decl, which produces incorrect/incoherent results.
I wanted to move it to check but there'd still be an
IdentifiedFacetTypeId and we keep all Ids in sem_ir. So leaving it in
sem_ir, but moving it to its own file. This more clearly separates
DeclaredFacetTypes and IdentifiedFacetTypes.
This supports impl lookup choosing an impl that targets a generic
interface through a named constraint, without crashing.
Instead of assuming the impl's target is a facet type containing an
interface, we use the identified facet type to find the specific
interface it targets.
Since the introduction of `other_requirements`, we now have a dedicated
step in impl lookup for checking that the requirements of the query
facet type are satisfied. That is the place where we will be checking
same-type constraints, which `other_requirements` signals the presence
of.
Consolidate all checking of `other_requirements` to that step, which
reduces our use of `FacetTypeInfo` (as opposed to the
`IdentifiedFacetType`) and removes interest in same-type constraints
from code that is not related to them.
Allow (don't diagnose) impls that have the same type structure if they
are associated with the same match_first block. Similarly, allow final
impls (made final by their enclosing match_first block) that overlap in
type structure when they are associated with the same match_first block.
If the type structures are the same, choose the first impl from the
match_first block.same.
If the type structures overlap but are not the same, and share a
match_first block, then we should choose the first overlapping impl from
the match_first block. This is true both for final and non-final impls.
See
https://github.com/carbon-language/carbon-lang/blob/de8b03faa3178ae683d8e7124fbcba81eb88e00c/proposals/p005337-interface-extension-and-final-impl-update.md#impl-selection-algorithm
But in the non-final case if there is an overlapping impl outside the
match_first, it can win if it's more specific.
The implicit/explicit `.Self` concept is a heuristic at best, so we
should avoid relying on it. The ImplicitOnly option is no longer used,
as it was used to remove/disambiguate `.Self` in nested facet types, but
we have banned nesting `where` on the RHS of a `where`. So we no longer
have to worry about ambiguous `.Self`.
We can remove all the designator tracking heuristics in `.Self`
substitution as well now, as they were used for the now-removed
ExplicitOnly (removed in
https://github.com/carbon-language/carbon-lang/pull/7460) and
ImplicitOnly options.
The self and interface given to TryFindMatchingWitnessFromImplLookup
come from the output of SubstPeriodSelf, so there is no need to do the
substitution again. And the self came from a LookupImplWitness
instruction so it is already fully canonicalized.
`LookupImplWitness` instructions inside the impl declaration can't use
the impl they are apart of. Previously we had an heuristic in eval which
would try to prevent finding the impl for a lookup from inside that
impl. But it breaks when the `.Self` is replaced in a generic impl with
a symbolic, and then that symbolic is replaced in a specific. The
specific's decl block contains that `LookupImplWitness` instruction and
it tries to use the impl it came from. This causes the same specific to
be formed again, but now it exists, so it's used as-is but it has no
decl block yet, and so we crash.
Now we ban an impl while we resolve its specific, both deduction of its
arguments and from any other substitution. The prevents instructions
from inside the impl (which are evaluated when resolving the specific)
from finding their own impl. We do so by adding the ImplId to a stack on
the Context, and then skipping such impls when looking for candidates
during eval.
This fixes a crash, which was demonstrated by the new test being added.
It also makes another todo test pass.
There's a whole lot of other semir churn, which seems to be mostly
reordering of constants. There are some fingerprint changes in
constants, but it appears they are the same canonical instructions, so
they don't represent a behaviour change. For example in
`toolchain/check/testdata/for/actual.carbon` the `%N.patt` constant has
been given its fingerprint suffix now as `%N.patt.aa5`. But they are
both this instruction, so it is just a formatting change:
```
inst6100001A: {kind: SymbolicBindingPattern, arg0: entity_name61000002, type: type(inst61000018)}
- name: `N`
- type: type(inst61000018): <pattern for Core.IntLiteral>; {kind: PatternType, arg0: inst(IntLiteralType), type: type(TypeType)} (concrete)
- value: symbolic_constant61000001
```
Clarify that the functions that search for a witness in a facet type no
longer take inputs from the query directly, but now take them from an
identified facet type constructed from the query. That means the self
type is already canonicalized.
In #7436 we stopped substituting `.Self` when collecting witnesses out
of a facet type. While this was correct, it did not capture all the
cases that need to avoid substituting `.Self`. And it poisoned the
`IdentifiedFacetType` cache by not replacing `.Self` but storing the
result in the cache. This led to incoherent behaviour, where the result
of an impl lookup would change depending on which ones had been done
previously.
Now we use a flag to track for each `.Self` if we're currently
type-checking inside the scope where it was introduced in a facet type.
While inside that scope, identify should not replace the `.Self`. Any
use of it should remain as-is since we don't yet know what value will
replace it. We call this state "frozen" since it should not be modified
by identify. This requires a substitution step when we leave the scope
that introduced the `.Self`, to remove the flag. The flag is set in the
`EntityName` of the `SymbolicBinding`, and is part of the canonical
value, since `.Self` can become part of types, which are constants, and
the flag needs to follow it for correct behaviour.
We also have to ensure the flag is the same when doing comparison with
constants from inside a facet type and constants from outside. For
instance in `(Z where .Z1 = ()) where .Z2 = .Z1`, when we arrive at the
second `.Z1` its `.Self` will be frozen, while the `.Z1 = ()` contains a
non-frozen `.Self`. So we add the frozen flag to the first when storing
it in `where_stack` in order to compare the constant values of the two
`.Z1`.
The `WhereExpr` requirement inst kinds now have an `InstConstantKind` of
`AlwaysUnique` instead of `Never`. This allows us to add them to the
usual InstBlocks, and in an `eval fn` body they have a constant value,
so eval does not fail when trying to call that function. We have to be
careful to not consider `AlwaysUnique` as being actually concrete
though, since their constant value erases `.Self`-dependence. This
allows us to stop special casing them when thawing the requirements
block in a `WhereExpr`, and we can just thaw each `InstId` in the block
in a straightforward manner.
We add the new flag to the instruction's fingerprint and name in
formatted semir.
This performs `.Self` substitution in a single step, for the whole facet
type, instead of doing it individually for each constraint visited in
the top-level facet type. Then we don't need to track state to avoid
subst in constraints that come from other named constraints.
The semir changes are because we now generate a whole other FacetType
from the substitution.
When we find a named constraint during identity, we recurse into it. The
specific args of the named constraint may contain references to `.Self`
which can then make `.Self` appear inside the named constraint, which
was making us replace `.Self` at multiple levels and incorrectly. The
first specific argument replaces `Self` in the named constraint, and we
pass in the self-type of the identify operation. This may contain
`.Self` and we should _not_ be replacing the `.Self` references with the
self-type that they are contained within. This led to infinite cycles.
In the meantime, we have made the toolchain reject any ambiguous `.Self`
from being constructed. So we know there is only one value of `.Self`
around in a facet type.
So now we replace `.Self` only in the top level facet type during
identity. That means replacing `.Self` in the specifics of the named
constraints that we recurse into. But we do _not_ replace `.Self`
anymore inside those named constraints. This resolves the infinite loop.
At the same time, when we are identifying an `impls` constraint from
earlier in the same facet type, like `C impls Z(.Self)` we are
identifying with a self-type of `C`. We want the output to use `C` as
the self-type since we should get back an identified facet type that
says `C impls Z(.Self)`. But we do _not_ want to replace the `.Self`
there since we're inside a facet type and the `.Self` does not refer to
`C`. So we parameterize `TryToIdentifyFacetType` to not replace `.Self`
when identifying an `impls` constraint from the `where_stack()`. This
resolves a large number of `fail_todo_` tests.
Impl lookup identifies the types of facets in the query, replacing
`.Self` in each type with the facet. This allows references using the
facet from outside the facet type to match similar structures inside the
facet type.
However replacing `.Self` with the facet can re-evaluate symbolic impl
lookups inside the facet type. These can perform deduction in generic
`final impl`s, which can attempt to convert the facet. Convert does an
impl lookup with that facet in the query, which causes us to form an
infinite recursion cycle.
Add tests that caused such a cycle, to demonstrate we no longer crash.
Some of these tests fail impl lookups using concrete values in the query
that match values from a `final impl`, with TODOs to address them.
We allow impl lookup to use `Self` which is partially identified, but we
were only allowing this for the `Self` facet appearing in the impl
lookup query self. We should also allow it when `Self` appears in the
impl lookup query target facet type.
But demonstrate that `Self` appearing in the interface of an
ImplWitnessAccess (a compound member access) is not sufficient at this
time, as it does not put `Self` into the type structure.
The `.Self` needs to be canonicalized when recording the witness, since
it will be a FacetAccessType in the `impls` clause.
Todo tests are also added for `C impls Y(.Self)`. The `.Self` in the
interface specific breaks everything as we end up replacing the `.Self`
with `C` in later `C as Y(.Self)` lookups, which then does not find the
witness.
Pull all the witnesses from `impls` constraints so that we capture
witnesses coming from named constraints on different self types, as `C
impls N(.Self)` may give witnesses for `C` or for other self types.
If a facet type contains `.X impls Y` then later references to `.X`
should know that it impls `Y`. This is done through a stack on the
context like for rewrites, where impl lookup can find constraints from
the current `where` expression being checked.
Similarly, if a facet type contains `.X impls (Y where .Z = T)`, then
`.X.(Y.Z) = T` should be available to later constraints in the facet
type for early application of rewrite rules. We add these rewrites to
the rewrite stack when handling the `impls` constraint
A nested designator like `.(X.X1).(Y.Y1)` results in nested
ImplWitnessAccess instructions, which can produce cycles in the
toolchain easily when replacing `.Self`.
First, when constructing a facet type like `V:! Z where .Z1 impls (Y
where .Y1 = U)` we substitute replace `.Self` in the nested facet type,
and in this case we replace `.Self` with `.Z1` which contains a `.Self`
of its own. This was coming from us being lazy about replacing `.Self`
in an `impl as` declaration, such as `impl C as Z where .Z1 = .Self`.
The self type is known there, so we can more eagerly replace `.Self` as
we do in a `require impls` declaration. Then the replacement for `.Self`
never comes with a `.Self` that needs to also be replaced. Any resulting
`.Self` would always be the top-level one.
Second, when evaluating ImplWitnessAccess, we were replacing .Self in
the LHS of rewrite constraints, but the `.Self` may itself have a type
that contains rewrite constraints. If one of those rewrite constraints
has nested ImplWitnessAccess instructions, we evaluate the new
ImplWitnessAccess, which again finds rewrite constraints to replace
`.Self` in, and we repeat forever. For this one we just stop replacing
.Self in the LHS of rewrite constraints. Since they are always against
.Self, we can always look in the access facet's type for a value.
While fixing ImplWitness access, also correct the lookup to search
through the types of nested ImplWitnessAccess instructions to find a
rewrite value, since it may find it at any level up to the eventual
`.Self`.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
We were treating ImplWitnessAccess as a concrete type, but that is
incorrect if its accessing a symbolic type value. This results in
concrete impl lookup queries failing to match a generic impl that is
built with a symbolic ImplWitnessAccess in its type structure, when the
query does not have the equivalent ImplWitnessAccess in its own type
structure.
We need to look in the top level facet being accessed through
ImplWitnessAccess for witnesses, such as in `T:! Z where .Z1 impls Y`
where `T` provides the witness for `T.Z1 as Y`. But we also need to look
in the facet type of the ImplWitnessAccess for witnesses, such as in
`T:! Z` for `interface Z { let Z1:! Y }`, where `T.Z1` provides the
witness for `T.Z1 as Y`.
To support that we give TypeIterator an iteration step for
ImplWitnessAccess before recursing into it, like we do for FacetValue.
While doing this, we make TypeIterator more recursive, by making less
special casing around the step from one inst into the next. Instead of
eagerly finding a SymbolicType, we consistently recurse back into the
big switch statement and have it decide the next iteration step. This
allows it to recurse into instructions like ImplWitnessAccess and
FacetValue in a consistent manner.
This enables searching facets like `T:! Z where .Z1 impls Y` for queries
like `T.Z1 as Y`, etc.
Any facet in the query self or the query target type may provide a
witness for the requirements of the impl lookup, so search through them
all.
We accept partially identified facet types in the query self, since
`Self` can be used inside a named constraint before it's fully
identified. But any use of `Self` being converted would put it in the
query self, not the query target, which would be some generic parameter
facet type that a type involving `Self` is being converted to.
TypeIterator is expanded to support this use case, by giving it the
ability to walk through FacetType's extend/impls constraints. We return
the facet values that we find in the iterator instead of a type id. And
we also include FacetValue instructions as an iterator step for clients
that want them, while also recursing through them.
A `where` expression nested inside a `T impls X` constraint makes
`.Self` ambiguous on the right-hand side of the `where` if `T` is
anything other than `.Self`. After the `where`, the value of a `.Self`
could be `T` or could be the value of `.Self` before the `impls`
constraint: the so-called top-level value of `.Self`.
Implicit use of `.Self` in designators is always allowed, and they are
bound (and replaced by a reference) to the inner-most possible value of
`.Self`. On the right-hand side of the nested `where` above, they have
the value `T as X`.
`.Self impls ...` is also always allowed, since it acts more as a
keyword here, and it always refers to the inner-most possible value of
`.Self`.
Any other explicit use of `.Self` is diagnosed when ambiguous, in any
kind of constraint. This is done in the handling of `WhereExpr` since it
has enough context to allow `.Self impls` (which is an explicit use)
while disallowing other explicit uses. And because it has non-canonical
instructions to work with, so it is able to diagnose errors with precise
locations.
Since `.Self` is no longer going to be marked with depth modifiers, the
eval of `WhereExpr` does not need an input facet value instruction
representing `.Self` to compare with, as they are now going to all be
equivalent. So revert it back to just looking for the `PeriodSelf` name
id, through a shared helper being introduced as `IsPeriodSelf`. And drop
the period self InstId from the `WhereExpr` instruction. This causes
most of the formatted SemIR changes.
Move helpers for working with and replacing `.Self` to their own file,
out of the `facet_type.h` header/cpp files. These are working with
`.Self` facet values more than facet types, though `.Self` is a name
that only exists inside the scope of a facet type.
This allows `T impls X` constraints to function, since they must contain
some reference to `.Self` in order to be valid. This should be
sufficient to support the interfaces we need for for loops over C++
range-for-compatible types.
We replace `.Self` in the following places:
- In a require decl, as we have a specific self facet to replace it with
from the declaration, either a user-specified facet or the symbolic
`Self`.
- When identifying a facet type, as we have a specific self that we are
identifying the facet type with. That self gets used for all `.Self`
references.
- Implicit `.Self` references on the RHS of an `impls` constraint when
building a facet type. The `.Self` references there no longer refer to
the top level self facet, so replace them with the facet that we now
know they refer to, which is found on the LHS of the `where` before the
`impls`.
- Rewrite constraints in impl lookup when validating them and comparing
them with constants from witnesses, which come from identifying a facet
type.
- Rewrite constraints in ImplWitnessAccess eval when comparing them with
constants from witnesses, which come from identifying a facet type.
Substitution is done through `SubstPeriodSelf`. It handles replacing
`.Self` and `.Self as type`, for a replacement facet that is either of
type FacetType or TypeType.
Eval currently diagnoses some ambiguous `.Self` references when doing
substitution of `.Self` but this is the incorrect place to do it, so
there are TODOs about moving this to name lookup. To support these
diagnostics there's some additional complexity in `SubstPeriodSelf` that
can go away once the TODOs are addressed, such as asking the caller if
they want to replace each `.Self`, in order for it to report a
diagnostic.
There are a number of follow-up work items here:
- Some TODO tests.
- Remove `SymbolicBindingType` since its intention was to support
`.Self` but we don't need it with this approach.
- Replace `.Self` in rewrite constraints of require decls.
- Replace `.Self` in rewrite constraints of impl as when constructing
the witness table.
- Reject explicit `.Self` in name lookup when it would be ambiguous.
- Officially disallow `.Self.A = B` in rewrite constraints in the design
docs, so that we don't have the case where `.A` is allowed but `.Self.A`
is not due to ambiguity.
The existing `GetCoreInterface` has linear-time complexity and adds more
than the search criteria to the `CoreInterfaceCache`. Adding a new field
to `Interface` changes this operation when building packages other than
`Core`, as this should only be set for the core library.
Add `InstIs`, `GetInstAs`, and `TryGetInstAs` which act on the
underlying constant instruction in a constant value, to save an explicit
call to `GetInstId`.
```carbon
context.insts().GetAs<InstT>(context.constant_values().GetInstId(const_id))
```
can now be written as simply
```carbon
context.constant_values().GetInstAs<InstT>(const_id)
```
For future work, we might provide `GetInst()` so that
`context.insts().Get(context.constant_values().GetInstId(const_id)` can
be shortened also.
Performing a lookup against `Self` inside the definition of the named
constraint leads to cycles, as described in the document [Self
contradictions in Named
Constraints](https://docs.google.com/document/d/17rn2XmME8o2MM4OJqatSVuMa1iYZ1PAgcNrf0PXR9Q4/edit?tab=t.0).
To prevent those cycles, this change introduces a large refactoring of
impl lookup.
The impl lookup done inside eval is reduced to only performing
monomorphization. That is it:
- Only looks for an provides final witnesses.
- Is not allowed to identify the facet type of the query self.
- Returns either a final witness or None (or an error)
The paths for finding non-final witnesses are now done outside of eval,
directly in the initial `LookupImplWitness()` function. If no final
witness it found through eval, the resulting non-final
`LookupImplWitness` instruction witness is returned. It does not produce
cycles to identify the facet type of query self outside of eval, since
that does not result in repeating the identification when resolving
specifics of the named constraint or require decl.
Move the ArrayStack for Context::require_impls_stack into a new class
which tracks a NamedConstraintId (or InterfaceId) for each frame of
RequireImplsIds, so that in type completion we always can find the
correct frame for a given named constraint which is still being defined,
in order to find the RequireImplsIds in the in-progress definition.
This commit creates an instance for any associated types in an interface
with a custom witness table. This unlocks interfaces designed for C++
interop that rely on arbitrary return types. For example,
`CppUnsafeDeref` becomes usable as of this commit.
This commit may have also implemented support for non-type associated
constants, but since we're lacking a practical test case, they're still
marked as TODO for the time being.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The self value can be a facet-value or a facet-value-as-type. The self
value used in `require` decls is the former. The the self value used for
identifying the facet type is the latter, we end up with two different
required interfaces in the identified facet type: one for each self
value.
Always canonicalize the self value to a facet value in identification.
Then dedupe the list of extend interfaces when constructing the
`IdentifiedFacetType` before counting them. And then impl lookup needs
to canonicalize its query self for comparing with the result from the
`IdentifiedFacetType`.
The type of the query self is looked into for a witness, but that type
may be unable to be identified. For example when the query is against
`Self` inside the declaration of a named constraint. Before this PR, we
would crash when identification failed. Now we produce a diagnostic.
This makes `RequireIdentifiedFacetType` take a `ContextScope` callback
(like it used to with an `AnnotationScope` callback) since all callers
now expect to handle diagnostics, and can provide useful context.
This is a followup to #6761.
Introduces `Context` and `SoftContext` messages, which can be introduced
through a `ContextBuilder`:
- The `Context` messages come before the diagnostic in the output.
- The first `Context` message steals the diagnostic level from the main
diagnostic, and turns the main diagnostic into a Note attached to the
context.
- A `SoftContext` message works similarly, but if it's preceeded by a
`Context` or `SoftContext` message, then it is dropped. This can be used
as a default/backup scope when nothing more interesting is provided up
the stack, such as in `TryEvalBlockForSpecific`.
The `ContextBuilder` is provided to a callback through
`Diagnostics::ContextScope`, an RAII type `AnnotationScope` but for
context messages.
This allows a high level operation to provide a context message like
"failed to identify facet type {0}" which will then be used as the error
if a diagnostic is produced during identification, with the latter
diagnostic attached as a note to explain why the contextual operation
failed.
In particular, this allows monomorphization errors (such as an array
bound being negative) to be attached to a higher lever operation instead
of being top-level diagnostics themselves, with the monomorphization
site being a note. This inverts the source code locations that appear in
the diagnostic, so that the top-level diagnostic points to the "user
code" which causes the monomorphization.
This is presented as an alternative strategy to #6753, which plumbed
diagnoser callbacks around to achieve the same goals.
We replace the diagnoser callbacks in type completion and operators with
ContextScope callbacks instead, which now provide better diagnostics for
monomorphization errors. Other callers to MakeSpecific do not yet have
ContextScopes introduced in order to turn monomorphization errors into
more interesting diagnostics.
This is related to #6727, but is generally a necessary fix even without
that issue. I'm not adding a specific test of #6727 because it should
also be covered by the tests in #6726.
Assisted-by: Google Antigravity with Gemini 3 Flash
Dana Jansens
Chandler Carruth
Geoff Romer
Özgür T. Önsoy
Christopher Di Bella
Jon Ross-Perkins