Currently each interface has a `Self` facet internally that becomes a
binding to every entity inside the interface: associated constants,
functions, and require decls. Each of these has to be independently
generic as a result. This makes is challenging in extended name lookup
to move into an extended scope of an interface, as we have a specific
for the interface, but the names within require a different specific
that includes a `Self` facet value.
We generalize this relationship by adding a second generic to Interface,
called `generic_with_self`. When we want to work with entities inside
the interface, we move from the interface-without-specific to the
interface-with-self specific by adding a Self to the specific. This is
done independently of any particular entity inside the Interface, as
those entities are now all members of the interface-with-self generic.
Associated constants no longer need a generic of their own, as they do
not have separate generic bindings. Functions retain a generic, but if
the function has no generic arguments, it will have no bindings of its
own now.
Require decls retain a generic so that their specific can be
instantiated separately from the interface. Requiring the interface to
be complete does not require the types in a require decl to be complete
unless it is modified by `extend`. So we allow them to be completed
later by keeping them in a separate generic.
Named constraints look like interfaces and gain the additional inner
generic-with-self, with the same relationship to require decls.
This removes the need for name lookup to perform Substitution of a Self
facet into the extended scope instruction. Instead, the
`SpecificConstant` instruction inserted by a `require` decl is part of
the interface-with-self generic. When looking through a FacetType for
extended scopes, for each interface, we push the scope with the specific
for the interface-with-self. Then the constant value of the
`SpecificConstant` is correctly modified by the provided self
automatically through applying that specific.
Undo changes that were meant to prevent use of a reference into
`ValueStore` after being invalidated. After #5576, the `ValueStore`
makes such references stable, so there's no need to worry about
invalidation.
Deduction can do conversion, and conversion can import impls from the
Core package. If you have the right number of impls in your ImplStore at
that moment, it will reallocate and any pointer into context.impls()
will be invalidated.
In particular, in impl lookup, we currentl loop over context.impls() and
do deduction on each impl. So this can break the for loop. Additionally,
we pass around a reference to the currently-being-looked-at Impl, which
becomes invalidated.
This is very challenging to test in any reliable way as you need a
specific number of impls in your ImplStore. I hit it when making changes
to a test in the middle of a bunch of file splits. Putting the same test
in its own file did not trigger the issue. It was caught by ASAN, which
showed:
- The memory was allocated by SmallVector in handle_impl when making the
Impl.
- The memory was freed by SmallVector reallocating in import_ref.cpp
- The memory was accessed when reading through the `impl` reference in
FindWitnessInImpls(). I was able to reproduce by printing the
`impl.interface.interface_id` after the call to GetWitnessIdForImpl()
which does the deduction.
I didn't save the ASAN stack and now I can't find the exact permutation
of the test file that caused it to occur in order to reproduce. :(
To avoid the UAF we stop passing around the Impl reference, and pass
around either the ImplId, or values from the Impl. To avoid copying the
entirety of the impl ids in context.impls() into a separate container in
order to iterate safely, we move the early outs from
GetWitnessIdForImpl() up to the caller where it can use them to reduce
the number impl ids that we iterate over. Type structures will be able
to further reduce the size of this set.
If the query facet type has more than one interface, we must find an
impl that provides that interface for the query type for each interface.
This just looks like a for loop over the interfaces and ensuring we
found one impl witness for every one.
However the impl matching must change since it can't look at the
constant value of the entire query facet type for comparison with the
impl, as that query facet type may be for multiple interfaces and we are
looking to match an impl of a single interface.
To do this we break the query facet type up into each interface and make
sure the interface ids match. Then ensure that the impl was able to
deduce any generic parameters using the specific of the single query
interface.
There are some TODOs left here:
1. If the facet type for the query or the impl constraint has
"other_requirements" then we can't verify that they match since they are
lost. We fall back to comparing the constant id of the query to the
impl's constraint (after deducing generics in the impl). This correctly
eliminates mismatches but eagerly eliminates impls that could match the
query interface as well when there's more than one interface in the
query.
2. We don't return a witness for every interface in the query facet
type. Since we can't demonstrate any use of the witness there yet, for
cases that can have more than one interface in the query facet type,
this doesn't break anything that was previously working. The return
value is currently treated as a bool for cases with multiple interfaces
in the facet type (as a test for "can this be converted") but the
converted-to facet value's witnesses are unused.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
For an expression such as `(Type as Interface).AssocFn()`, track the
`Self` type `Type` in the result of the member access so that it's
available when checking the function call.
This introduces a new kind of type, `ImplFunctionType`, that represents
the type of a function that is expected within an impl, modeled as the
type of the function within the interface plus a value to use as `Self`.
Calls to values of this type behave like calls to the underlying
function except that the `Self` parameter is pre-bound to the self type
from the facet.
In order to support this, fix an issue where the imported list of
generic bindings lost their association with their enclosing generic.
This adds a little complexity to `import_ref`, including a new recursive
cycle that I intend to address in a follow-up PR.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
High level, replacing `Id::Invalid` with `Id::None` and `Id::is_valid`
with `Id::has_value` for clarity, as discussed
[here](https://discord.com/channels/655572317891461132/655578254970716160/1331664574545395794).
The `IntId` refactoring is needed together with `AnyIdBase` because it's
also used with `ValueStore`.
Note, trying to be careful not to rewrite `EnumBase::InvalidIndex`, or
`is_valid` in general (e.g., `IdKind::is_valid`).
I've tried to sequence commits here:
1. Automatic replacements:
- `((?:Id|Index)(?: |::|\(|Base(?:\(|::)))Invalid((?:Index)?\W)` ->
`$1None$2`
- `<invalid>` -> `<none>`
- `InvalidNodeId` -> `NoneNodeId`
- `/\*invalid\*/` -> `/*none*/`
- `id((?:_|\(\))(?:\.|->))is_valid` -> `id$1has_value`
2. Manual edits:
- In `int.h` and `int_test.cpp`
- `IntT` has `is_value`, which I'm renaming to `is_embedded_value`.
- Manual edits to comments in this file.
- `AnyIdBase` and `IdBase`
- Declaration of `is_valid` -> `has_value`, `InvalidIndex` ->
`NoneIndex`.
- In `ids.h` and `ids.cpp`
- `is_valid` -> `has_value`
- `// An explicitly invalid ID.` -> `// An ID with no value.`; similar
for index
- Various math on `InvalidIndex` -> `NoneIndex`
- Various mentions of "valid" in comments
- In `value_store.h`, for `IdT::Invalid`, plus one comment
- In `impl.h` and `tokenized_buffer.h`, we had different initialization
of `::None` values (versus `ids.h` syntax) that I fixed manually.
- Spot checks to compile
- Particularly where `is_valid` replacements didn't catch spots due to
different naming.
3. Autoupdate tests
4. verbose.carbon (NOAUTOUPDATE)
5. Comment spot checks
Note there are probably other mentions of "Invalid" that should be swept
up, but I'd like to argue for merging and separating out remaining
cleanup since this is so sweeping (and likely to hit merge conflicts
from churn). We'll probably have lingering mentions of "invalid" for a
bit regardless, just because there are uses of "invalid" in non-Id APIs.
Refactor the current function call deduction logic to make it reusable.
Call into it from `impl` deduction. Also build a generic region for the
definition portion of a generic `impl` and substitute into it before
accessing the witness in a specific `impl`.
This is enough to get simple uses of generic `impl`s to work. The main
blocker for more complex cases is that we have very little support for
non-trivial deduction, so while we can deduce `forall [T:! type] T as
I`, we can't deduce `forall [T:! type] C as I(T)` yet.
Add these interfaces to the core library. For now, they're two separate
interfaces because we don't yet support one interface extending another.
This collapses a lot of the layering in check: for example, the call
building logic depends on implicit conversions, conversions now depend
on the overloaded operator machinery, and that machinery depends on
building calls.
In passing, improve the diagnostics for failing to find a name required
from the prelude. Also convert all the transitively-called code from
`NodeId` to `LocId` given the latter is what the conversion machinery
has available.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Dana Jansens
Geoff Romer
Richard SmithandJon Ross-Perkins