Files
carbon-lang/toolchain/check/subst.cpp
T
Dana JansensandJon Ross-Perkins 53c98a8619 Support specialization in impl lookup with a symbolic query/impl. (#5169)
Add a new instruction called ImplSymbolicWitness which represents a
search for an impl declaration given a self type and an interface to
find implemented for the self type. The self type is stored as a
constant instruction id, rather than as a ConstantId, as instructions
don't currently support holding ConstantId. The interface is stored as a
SpecificInterface but we can't fit all of it directly into the
instruction. So we add a new id to refer to the SpecificInterface as
follows.

Add a new SpecificInterfaceId which indexes into a canonical value store
on SemIR::File. This tracks all `SpecificInterface`s stored in an
instruction - specifically the ImplSymbolicWitness instruction.

The SpecificInterface on Impl is still stored there as a value, not as
an id, and no id is eagerly constructed for it. We wait until an id is
needed to make one. Since they are canonical, a new id is only create
when a new SpecificInterface value is seen.

When doing impl lookup, and the query is not concrete, and the impl is
not effectively final, the query needs to consider future impls that may
specialize either the self type or the constaint to make a more precise
match and replace the found impl declaration. Instead of returning the
ImplWitness instruction from the found impl, we generate a
ImplSymbolicWitness instruction, storing the query so that it can be
replayed later. This instruction is added to the generic eval block and
thus will be re-evaluated later with a SpecificId that may make the
query more concrete. When evaluating the instruction and replaying the
query, the lookup has the same conditions and if it does not decide to
use the found impl concretely, then the same instruction is returned
from eval, leaving it as symbolic.

--- Impl lookup changes ---

Impl lookup gets a little more interesting now. It continues to look in
the facet value for a witness if the self type is a facet value. Then
falls back to looking for an impl declaration. This step is no longer
done directly. Instead, we construct a ImplSymbolicWitness instruction
and evaluate it immediately for each interface that are in the query
facet type.

The ImplSymbolicWitness instruction, when evaluated, calls back to the
impl lookup code, with a query specific interface. There we resume back
into the same code path as from before, finding a witness in an impl
declaration. But we may return "found a non-final impl" instead of a
concrete witness. If eval receives this back, it evaluates to the
current ImplSymbolicWitness instruction as the resulting constant value.

To pass lookup failures back through eval, a result of InstId::None from
the second step of impl lookup will result in a non-constant value,
which is used as a signal back up the stack to the original impl lookup
function that the lookup failed. Using a non-constant value here would
break evaluation of the generic eval block if impl lookup could fail
there, however we know it will not since we only leave behind an
ImplSymbolicWitness instruction in the eval block if we found at least
one matching impl already, and we just want to look for a better match
with a more specific query.

We must take care to not store a reference into any value store across
computation in impl lookup, since impl lookup can recurse into itself
invalidate those stores. That includes the SpecificInterface obtained
from a SpecificInterfaceId, which impl lookup also inserts into the
store.

--- The long tail ---

Adding a new instruction and a new id type requires a myriad of changes
to support them:

We add Dump() support for SpecificInterfaceId. And fix a crash in Dump
for SpecificId::None. We also add MakeSpecificInterfaceId() for dumping
arbitrary ids.

The type of ImplSymbolicWitness is a new singleton builtin type
instruction called WitnessSymbolicType (like WitnessType is the type for
an ImplWitness).

Both ImplSymbolicWitness and WitnessSymbolicType are given `Value` as
their expression category as they are builtin constant values. And
BuildInfo() in TypeCompleter is taught about them both, returning a
`ValueRepr::Copy`.

WitnessSymbolicType is added to the set of SingletonInstKinds, so that
it can have a singleton instrution id as a static member.

Lower's BuildTypeForInst() is taught to make an empty struct for
WitnessSymbolicType, similar to WitnessType.

Instruction formatter (FormatterImpl) grows support for printing a
SpecificInterfaceId so that it can print both arguments of
ImplSymbolicWitness on the RHS when printing the SemIR instruction. To
print a SpecificInterfaceId, it prints both the interface id and the
specific id (if there is one). For example, for a query on a generic
interface `Z` with one parameter, the RHS includes the query, interface,
and specific:
```
%Z.impl_symbolic_witness: <symbolic witness> = impl_symbolic_witness %U, @Z, @Z(%U.as_type) [symbolic]
```

IdKind is extended to include SpecificInterfaceId.

InstFingerprinter is taught to look through SpecificInterfaceId and use
the interface and specific ids in the fingerprint.

InstNamer is taught about SpecificInterfaceId, counting the interfaces
when building an index. It is also tought about ImplSymbolicWitness,
using the name of the interface within and the `.impl_symbolic_witness`
suffix. For example, here the LHS is named after the interface in the
query:
```
%Z.impl_symbolic_witness: <symbolic witness> = impl_symbolic_witness %U, @Z, @Z(%U.as_type) [symbolic]
```

StringifyTypeExpr is taught about WitnessSymbolicType, which uses its IR
name since it's a singleton. And about ImplSymbolicWitness which uses
its constant value. The handling of ImplWitnessAccess also needed to be
adjusted, since it assumed that ImplWitnessAccess::witness_id would
always be a FacetAccessWitness, but it can now also be an
ImplSymbolicWitness. (It seems that the witness_id is also assigned
ImplWitness instructions, but those ImplWitnessAccess instructions don't
ever seem to get stringified in a diagnostic at this time.) At the
moment the ImplWitnessAccess with a symbolic witness is just stringified
as "<symbolic>", such as in:
```
x.carbon:1:2: error: cannot implicitly convert value of type `()` to `<symbolic>` [ConversionFailure]
  let a: C(D).(Z.X) = ();
                      ^~
```

There is a TODO left behind to include more information there.

The TypeStructure builder is made to handle WitnessSymbolicType and
WitnessType. These come up now in deduce where a generic impl will have
a ImplSymbolicWitness in a FacetValue for a generic self type. The query
may have a concrete ImplWitness in the same position. Since deduce tries
to deduce through the FacetValue, it tries to convert ImplWitness to
ImplSymbolicWitness, tries to do an impl lookup for `impl ImplWitness as
ImplicitAs(ImplSymbolicWitness)` and causes us to build type structures
with each of these.

Subst is updated to handle pushing and popping SpecificInterfaceId.
Without this, when finishing a generic's eval block, we would walk into
the ImplSymbolicWitness instruction, and its arguments, and fail to
recurse down into the SpecificInterfaceId. Then any specifics inside
would be left as "orphaned" without any generic id attached to them, and
we would never update the instructions in the SpecificInterface's
instructions (inside its own SpecificId) with new constant values when
evaluating the generic eval block against a specific. To do this we push
the specific_id inside the SpecificInterface, and when popping we pop
the specific_id then construct a new canonical SpecificInterface with it
and return that id.

We add support for importing ImplSymbolicWitness by importing its self
constant instruction and specific interface id. However we also had to
add import support for SpecificImplFunction, which can now appear in the
generic eval block for a generic impl declaration, and thus must be
imported with the declaration. This is done very similarly to
SpecificFunction, except the `type_id` is a singleton value.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2025-03-26 15:10:23 +00:00

424 lines
16 KiB
C++

// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
// Exceptions. See /LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
#include "toolchain/check/subst.h"
#include "toolchain/check/eval.h"
#include "toolchain/check/generic.h"
#include "toolchain/sem_ir/copy_on_write_block.h"
#include "toolchain/sem_ir/ids.h"
#include "toolchain/sem_ir/inst.h"
namespace Carbon::Check {
namespace {
// Information about an instruction that we are substituting into.
struct WorklistItem {
// The instruction that we are substituting into.
SemIR::InstId inst_id;
// Whether the operands of this instruction have been added to the worklist.
bool is_expanded : 1;
// The index of the worklist item to process after we finish updating this
// one. For the final child of an instruction, this is the parent. For any
// other child, this is the index of the next child of the parent. For the
// root, this is -1.
int next_index : 31;
};
// A list of instructions that we're currently in the process of substituting
// into. For details of the algorithm used here, see `SubstConstant`.
class Worklist {
public:
explicit Worklist(SemIR::InstId root_id) {
worklist_.push_back(
{.inst_id = root_id, .is_expanded = false, .next_index = -1});
}
auto operator[](int index) -> WorklistItem& { return worklist_[index]; }
auto size() -> int { return worklist_.size(); }
auto back() -> WorklistItem& { return worklist_.back(); }
auto Push(SemIR::InstId inst_id) -> void {
CARBON_CHECK(inst_id.has_value());
worklist_.push_back({.inst_id = inst_id,
.is_expanded = false,
.next_index = static_cast<int>(worklist_.size() + 1)});
CARBON_CHECK(worklist_.back().next_index > 0, "Constant too large.");
}
auto Pop() -> SemIR::InstId { return worklist_.pop_back_val().inst_id; }
private:
// Constants can get pretty large, so use a large worklist. This should be
// about 4KiB, which should be small enough to comfortably fit on the stack,
// but large enough that it's unlikely that we'll need a heap allocation.
llvm::SmallVector<WorklistItem, 512> worklist_;
};
} // namespace
// Pushes the specified operand onto the worklist.
static auto PushOperand(Context& context, Worklist& worklist,
SemIR::IdKind kind, int32_t arg) -> void {
auto push_block = [&](SemIR::InstBlockId block_id) {
for (auto inst_id :
context.inst_blocks().Get(SemIR::InstBlockId(block_id))) {
worklist.Push(inst_id);
}
};
auto push_specific = [&](SemIR::SpecificId specific_id) {
if (specific_id.has_value()) {
push_block(context.specifics().Get(specific_id).args_id);
}
};
switch (kind) {
case SemIR::IdKind::For<SemIR::InstId>:
case SemIR::IdKind::For<SemIR::MetaInstId>:
if (SemIR::InstId inst_id(arg); inst_id.has_value()) {
worklist.Push(inst_id);
}
break;
case SemIR::IdKind::For<SemIR::TypeId>:
if (SemIR::TypeId type_id(arg); type_id.has_value()) {
worklist.Push(context.types().GetInstId(type_id));
}
break;
case SemIR::IdKind::For<SemIR::InstBlockId>:
push_block(SemIR::InstBlockId(arg));
break;
case SemIR::IdKind::For<SemIR::StructTypeFieldsId>: {
for (auto field :
context.struct_type_fields().Get(SemIR::StructTypeFieldsId(arg))) {
worklist.Push(context.types().GetInstId(field.type_id));
}
break;
}
case SemIR::IdKind::For<SemIR::TypeBlockId>:
for (auto type_id : context.type_blocks().Get(SemIR::TypeBlockId(arg))) {
worklist.Push(context.types().GetInstId(type_id));
}
break;
case SemIR::IdKind::For<SemIR::SpecificId>:
push_specific(SemIR::SpecificId(arg));
break;
case SemIR::IdKind::For<SemIR::SpecificInterfaceId>: {
auto interface =
context.specific_interfaces().Get(SemIR::SpecificInterfaceId(arg));
push_specific(interface.specific_id);
break;
}
case SemIR::IdKind::For<SemIR::FacetTypeId>: {
const auto& facet_type_info =
context.facet_types().Get(SemIR::FacetTypeId(arg));
for (auto interface : facet_type_info.impls_constraints) {
push_specific(interface.specific_id);
}
for (auto rewrite : facet_type_info.rewrite_constraints) {
auto lhs_inst_id =
context.constant_values().GetInstId(rewrite.lhs_const_id);
auto rhs_inst_id =
context.constant_values().GetInstId(rewrite.rhs_const_id);
worklist.Push(lhs_inst_id);
worklist.Push(rhs_inst_id);
}
// TODO: Process other requirements as well.
break;
}
default:
break;
}
}
// Converts the operands of this instruction into `InstId`s and pushes them onto
// the worklist.
static auto ExpandOperands(Context& context, Worklist& worklist,
SemIR::InstId inst_id) -> void {
auto inst = context.insts().Get(inst_id);
auto kinds = inst.ArgKinds();
PushOperand(context, worklist, SemIR::IdKind::For<SemIR::TypeId>,
inst.type_id().index);
PushOperand(context, worklist, kinds.first, inst.arg0());
PushOperand(context, worklist, kinds.second, inst.arg1());
}
// Pops the specified operand from the worklist and returns it.
static auto PopOperand(Context& context, Worklist& worklist, SemIR::IdKind kind,
int32_t arg) -> int32_t {
auto pop_block_id = [&](SemIR::InstBlockId old_inst_block_id) {
auto size = context.inst_blocks().Get(old_inst_block_id).size();
SemIR::CopyOnWriteInstBlock new_inst_block(context.sem_ir(),
old_inst_block_id);
for (auto i : llvm::reverse(llvm::seq(size))) {
new_inst_block.Set(i, worklist.Pop());
}
return new_inst_block.GetCanonical();
};
auto pop_specific = [&](SemIR::SpecificId specific_id) {
if (!specific_id.has_value()) {
return specific_id;
}
auto& specific = context.specifics().Get(specific_id);
auto args_id = pop_block_id(specific.args_id);
return context.specifics().GetOrAdd(specific.generic_id, args_id);
};
switch (kind) {
case SemIR::IdKind::For<SemIR::InstId>:
case SemIR::IdKind::For<SemIR::MetaInstId>: {
SemIR::InstId inst_id(arg);
if (!inst_id.has_value()) {
return arg;
}
return worklist.Pop().index;
}
case SemIR::IdKind::For<SemIR::TypeId>: {
SemIR::TypeId type_id(arg);
if (!type_id.has_value()) {
return arg;
}
return context.types().GetTypeIdForTypeInstId(worklist.Pop()).index;
}
case SemIR::IdKind::For<SemIR::InstBlockId>: {
return pop_block_id(SemIR::InstBlockId(arg)).index;
}
case SemIR::IdKind::For<SemIR::StructTypeFieldsId>: {
SemIR::StructTypeFieldsId old_fields_id(arg);
auto old_fields = context.struct_type_fields().Get(old_fields_id);
SemIR::CopyOnWriteStructTypeFieldsBlock new_fields(context.sem_ir(),
old_fields_id);
for (auto i : llvm::reverse(llvm::seq(old_fields.size()))) {
new_fields.Set(i, {.name_id = old_fields[i].name_id,
.type_id = context.types().GetTypeIdForTypeInstId(
worklist.Pop())});
}
return new_fields.GetCanonical().index;
}
case SemIR::IdKind::For<SemIR::TypeBlockId>: {
SemIR::TypeBlockId old_type_block_id(arg);
auto size = context.type_blocks().Get(old_type_block_id).size();
SemIR::CopyOnWriteTypeBlock new_type_block(context.sem_ir(),
old_type_block_id);
for (auto i : llvm::reverse(llvm::seq(size))) {
new_type_block.Set(
i, context.types().GetTypeIdForTypeInstId(worklist.Pop()));
}
return new_type_block.GetCanonical().index;
}
case SemIR::IdKind::For<SemIR::SpecificId>: {
return pop_specific(SemIR::SpecificId(arg)).index;
}
case SemIR::IdKind::For<SemIR::SpecificInterfaceId>: {
auto interface =
context.specific_interfaces().Get(SemIR::SpecificInterfaceId(arg));
auto specific_id = pop_specific(interface.specific_id);
return context.specific_interfaces()
.Add({
.interface_id = interface.interface_id,
.specific_id = specific_id,
})
.index;
}
case SemIR::IdKind::For<SemIR::FacetTypeId>: {
const auto& old_facet_type_info =
context.facet_types().Get(SemIR::FacetTypeId(arg));
SemIR::FacetTypeInfo new_facet_type_info;
// Since these were added to a stack, we get them back in reverse order.
new_facet_type_info.rewrite_constraints.resize(
old_facet_type_info.rewrite_constraints.size(),
SemIR::FacetTypeInfo::RewriteConstraint::None);
for (auto i : llvm::reverse(
llvm::seq(old_facet_type_info.rewrite_constraints.size()))) {
auto rhs_id = context.constant_values().Get(worklist.Pop());
auto lhs_id = context.constant_values().Get(worklist.Pop());
new_facet_type_info.rewrite_constraints[i] = {.lhs_const_id = lhs_id,
.rhs_const_id = rhs_id};
}
new_facet_type_info.impls_constraints.resize(
old_facet_type_info.impls_constraints.size(),
SemIR::SpecificInterface::None);
for (auto i : llvm::reverse(
llvm::seq(old_facet_type_info.impls_constraints.size()))) {
const auto& old = old_facet_type_info.impls_constraints[i];
new_facet_type_info.impls_constraints[i] = {
.interface_id = old.interface_id,
.specific_id = pop_specific(old.specific_id)};
}
new_facet_type_info.other_requirements =
old_facet_type_info.other_requirements;
new_facet_type_info.Canonicalize();
return context.facet_types().Add(new_facet_type_info).index;
}
default:
return arg;
}
}
// Pops the operands of the specified instruction off the worklist and rebuilds
// the instruction with the updated operands if it has changed.
static auto Rebuild(Context& context, Worklist& worklist, SemIR::InstId inst_id,
const SubstInstCallbacks& callbacks) -> SemIR::InstId {
auto inst = context.insts().Get(inst_id);
auto kinds = inst.ArgKinds();
// Note that we pop in reverse order because we pushed them in forwards order.
int32_t arg1 = PopOperand(context, worklist, kinds.second, inst.arg1());
int32_t arg0 = PopOperand(context, worklist, kinds.first, inst.arg0());
int32_t type_id =
PopOperand(context, worklist, SemIR::IdKind::For<SemIR::TypeId>,
inst.type_id().index);
if (type_id == inst.type_id().index && arg0 == inst.arg0() &&
arg1 == inst.arg1()) {
return callbacks.ReuseUnchanged(inst_id);
}
// TODO: Do we need to require this type to be complete?
inst.SetType(SemIR::TypeId(type_id));
inst.SetArgs(arg0, arg1);
return callbacks.Rebuild(inst_id, inst);
}
auto SubstInst(Context& context, SemIR::InstId inst_id,
const SubstInstCallbacks& callbacks) -> SemIR::InstId {
Worklist worklist(inst_id);
// For each instruction that forms part of the constant, we will visit it
// twice:
//
// - First, we visit it with `is_expanded == false`, we add all of its
// operands onto the worklist, and process them by following this same
// process.
// - Then, once all operands are processed, we visit the instruction with
// `is_expanded == true`, pop the operands back off the worklist, and if any
// of them changed, rebuild this instruction.
//
// The second step is skipped if we can detect in the first step that the
// instruction will not need to be rebuilt.
int index = 0;
while (index != -1) {
auto& item = worklist[index];
if (item.is_expanded) {
// Rebuild this item if necessary. Note that this might pop items from the
// worklist but does not reallocate, so does not invalidate `item`.
item.inst_id = Rebuild(context, worklist, item.inst_id, callbacks);
index = item.next_index;
continue;
}
if (callbacks.Subst(item.inst_id)) {
index = item.next_index;
continue;
}
// Extract the operands of this item into the worklist. Note that this
// modifies the worklist, so it's not safe to use `item` after
// `ExpandOperands` returns.
item.is_expanded = true;
int first_operand = worklist.size();
int next_index = item.next_index;
ExpandOperands(context, worklist, item.inst_id);
// If there are any operands, go and update them before rebuilding this
// item.
if (worklist.size() > first_operand) {
worklist.back().next_index = index;
index = first_operand;
} else {
// No need to rebuild this instruction: its operands can't be changed by
// substitution because it has none.
index = next_index;
}
}
CARBON_CHECK(worklist.size() == 1,
"Unexpected data left behind in work list");
return worklist.back().inst_id;
}
namespace {
// Callbacks for performing substitution of a set of Substitutions into a
// symbolic constant.
class SubstConstantCallbacks final : public SubstInstCallbacks {
public:
SubstConstantCallbacks(Context& context, Substitutions substitutions)
: context_(context), substitutions_(substitutions) {}
// Applies the given Substitutions to an instruction, in order to replace
// BindSymbolicName instructions with the value of the binding.
auto Subst(SemIR::InstId& inst_id) const -> bool override {
if (context_.constant_values().Get(inst_id).is_concrete()) {
// This instruction is a concrete constant, so can't contain any
// bindings that need to be substituted.
return true;
}
auto entity_name_id = SemIR::EntityNameId::None;
if (auto bind =
context_.insts().TryGetAs<SemIR::BindSymbolicName>(inst_id)) {
entity_name_id = bind->entity_name_id;
} else if (auto bind =
context_.insts().TryGetAs<SemIR::SymbolicBindingPattern>(
inst_id)) {
entity_name_id = bind->entity_name_id;
} else {
return false;
}
// This is a symbolic binding. Check if we're substituting it.
// TODO: Consider building a hash map for substitutions. We might have a
// lot of them.
for (auto [bind_index, replacement_id] : substitutions_) {
if (context_.entity_names().Get(entity_name_id).bind_index() ==
bind_index) {
// This is the binding we're replacing. Perform substitution.
inst_id = context_.constant_values().GetInstId(replacement_id);
return true;
}
}
// If it's not being substituted, don't look through it. Its constant
// value doesn't depend on its operand.
return true;
}
// Rebuilds an instruction by building a new constant.
auto Rebuild(SemIR::InstId /*old_inst_id*/, SemIR::Inst new_inst) const
-> SemIR::InstId override {
auto result_id = TryEvalInst(context_, SemIR::InstId::None, new_inst);
CARBON_CHECK(result_id.is_constant(),
"Substitution into constant produced non-constant");
return context_.constant_values().GetInstId(result_id);
}
private:
Context& context_;
Substitutions substitutions_;
};
} // namespace
auto SubstConstant(Context& context, SemIR::ConstantId const_id,
Substitutions substitutions) -> SemIR::ConstantId {
CARBON_CHECK(const_id.is_constant(), "Substituting into non-constant");
if (substitutions.empty()) {
// Nothing to substitute.
return const_id;
}
if (!const_id.is_symbolic()) {
// A concrete constant can't contain a reference to a symbolic binding.
return const_id;
}
auto subst_inst_id =
SubstInst(context, context.constant_values().GetInstId(const_id),
SubstConstantCallbacks(context, substitutions));
return context.constant_values().Get(subst_inst_id);
}
} // namespace Carbon::Check