Based on review feedback on
https://github.com/carbon-language/carbon-lang/pull/6215#discussion_r2430177644
There are some intermediate commits with alternatives, finding other
ways (non-templates) to address the layering boundaries between
`ValueStore` construction and `CheckIRId` tagging. But, yeah, template
seems like the way to go - certainly in terms of terseness and probably
in terms of extensibility to other Id tagging as/when needed.
This would save space for every `EntityName` that is not an imported C++
global variable.
C++ global variables include static data members.
Created `CppGlobalVarId`, `CppGlobalVarKey` and `CppGlobalVar` to allow
having `CanonicalValueStore` that maps `EntityNameId` (which is in
`CppGlobalVarKey` and `CppGlobalVar`) to `ClangDeclId` (which is also in
`CppGlobalVar`).
This is similar to `ClangDeclId`, `ClangDeclKey` and `ClangDecl` .
This change makes dumping and debugging work again with InstIds that are
now tagged with the CheckIRId. The textual representation of an InstId
is changed from `irN.instM` back to `instM` but the `M` is now a hex
value with the tag as part of it, which is the same number that is
physically in the `InstId::index` field. This prevents any cases where
we would potentially print incorrect values for large InstIds.
We teach the `dump` command in lldb to parse hex values for InstId so
that we can paste these numbers back into the debugger.
Use the `CheckIRId` as a unique identifier for the scope of an `InstId`
- if an `InstId` is created within the scope of one `CheckIRId` it must
not be used in the scope of a different `CheckIRId`.
This is achieved without extra storage, but with false negatives for
large inputs.
When an `InstId` is created, the original index of the `Inst` is XORed
with a tag derived from the `CheckIRId` to produce the final `InstId`.
When the `InstId` is used, the expected tag is XORed with the `InstId`
to get back to the original index - if the tags don't match, the
resulting index will be corrupted, likely too large - resulting in an
out of bounds index CHECK-failure.
(the tag value is derived as such:
* take the CheckIRId
* left shift one bit (padding zero)
* left shift another bit (padding 1 - used to signify that the resulting
`InstId` has a tag combined into it)
* reverse the bits
In this way, the tag is unlikely to overlap with the index for small
test cases - making it possible to separate out the `CheckIRId` from the
index in these cases to provide more meaningful debugging/CHECK
messages, and more informative `SemIR` textual dumping that can now
include the `CheckIRId` along with the `Inst`'s index in the name of an
`inst`)
The test churn here is improved printing as tagged `InstId`s can now,
with best effort (more likely for small test cases where the `CheckIRId`
and the `Inst` index aren't at risk of overlapping from the high and low
bits), render the `CheckIRId` as part of the inst's name. Going from
`instNN` to `irMM.instNN`.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The general strategy here is to force use of a thunk when we want to use
default arguments, and have Clang generate uses of the default arguments
on its side of the thunk.
To support this, change the key type used in `clang_decls` from being
just a `Decl*` to being a pair of `Decl*` and number of parameters in
the case of function decls. Import distinct `SemIR::Function`s for each
number of parameters that's used, and corresponding distinct thunks.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This changes ConstantValueStore to use ValueStore so that we get the
allocation flow that we're leaning towards there. This adds
`ConstantId::SymbolicId` to represent what was previously an int32
"symbolic_index" (although called an index, it's consistent with ID
usages).
I'm also changing Chunk::at/push to be consistent with the wrapping
Get/Add naming; the naming difference sticks out more with
UninitializedFill being added.
This avoids reallocating the backing buffer in ValueStore so that
references into the ValueStore are never invalidated when adding new
values. This works especially well since we never delete values from a
ValueStore.
The strategy used is to allocate chunks of a fixed size, and inserting
into each chunk until it is full before allocating the next. The
ValueStore starts with an initial allocated chunk in all cases, so that
there is only a single indirection for adding and accessing values from
this chunk. After it's full, additional chunks are allocated in a
vector, so two indirections are required to add or access values in
these chunks.
This obviates the need for
https://github.com/carbon-language/carbon-lang/pull/5529 as we no longer
need to worry about holding pointers into a ValueStore.
We introduce a Flatten operation for ranges. It flattens a "range over
ranges over Ts" down to a "range over Ts". This allows us to make an
range over the values in the ValueStore from a range over the chunks in
the ValueStore. See
https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.flatten
for inspiration for this name choice. Flatten is used in one other case
where we were writing two levels of for loops to do the same thing.
The `array_ref()` accessor is changed to `values()` and its now a range
(typed as a `ValueStoreRange`) over all values as references (like
ArrayRef was, but without random access).
As pointers to a ValueStore can no longer be invalidated, we remove the
ASAN poisoning feature and support from ValueStore.
This may cause a regression in our compile benchmark of up to 5%, though
that is close to or within the noise of the benchmark. We can look at
ways to optimize things further in the future. Perhaps by tuning the
chunk size further, or by making later chunks larger than earlier
chunks, or other strategies.
We frequently want to operate on singletons. Per discussion, drop
`Singleton` to make the code shorter.
This started off as wanting to write `inst_id.is_error()`, but the
dependency relationship between ids.h and singleton_insts.h would
require some kind of delayed evaluation to allow the implementation to
remain in headers (which I suspect is helpful to have for inlining). I
could have added something like `IsErrorInst`, forward declared in ids.h
and defined in singleton_insts.h (which would always be included by
typed_insts.h), but the template approach felt like a decent balance
between (a) removing the boilerplate `::SingletonInstId`, (b)
understandability, (c) still visually mirroring if we immediately return
a singleton, and (d) flexibility for more than just `ErrorInst`. But TBH
I'd probably still have written `is_error()` if it didn't require
addressing the cross-header cycle.
Then I tried `SemIR::InstId::Is<SemIR::ErrorInst>`, which generally
worked with types but generated the complaint that it didn't shorten
*all* singleton uses. So pulling back on `::Is`, and instead just
dropping `Singleton`.
In preparation for shifting from `TypeId`s potentially representing
attached types to always representing unattached types, using
[terminology suggested on
Discord](https://discord.com/channels/655572317891461132/963846118964350976/1359286326779973712).
This change causes us to track slightly more type spelling information
through SemIR.
One change that has significant impact on the SemIR output is that we
now build a `struct_type` instruction in each class representing the
types of the fields, including the spelling used for those types. This
is now no longer always identical to the corresponding canonical
`struct_type` for the object representation, so it's built separately
and owned by the class.
Also remove `TypeBlock` support entirely, as its only use was
representing `TupleType`s, which now use an `InstBlock`.
Fixes#5186
With @zygoloid's kind assistance, this generalizes the existing
non-const lowering of ClassInit, that had previously only handled
InitializeFrom, to find other cases - such as a nested ClassInit used to
initialize a class member.
This refactors the `FindReturnSlotArgForInitializer` from
`check/convert.cpp` into `sem_ir/file.{h,cpp}` for use from lowering
(since lower doesn't depend on check, which I assume is an intentional
layering constraint - so figured it made sense to move it to sem_ir, and
found one or two similar-ish utility functions in `sem_ir/file.{h,cpp}`,
so figured that was a good spot)
What this does:
- Adds tracking where storage is allocated.
- Determines if that storage supports destruction and, if so, records
the `destroy` function for it.
- Calls any found `destroy` functions when going out-of-scope.
What this does not do:
- Precise scope tracking of temporaries. We currently don't define
temporary scopes, which would probably be the solution.
- Destruction for anything but a `class` with `fn destroy`, in an
implicit return. That excludes:
- Classes with members that need destruction, particularly in the
absence of `fn destroy`.
- Structs, tuples, and arrays.
- Explicit returns, break, continue, nested scopes.
Noting the exclusions in particular, I think those will need work to
support, but this should set the right framework.
The cleanup block concept stems from clang and trying to share code
across cleanups, from discussion with chandlerc. Note in this
implementation I try to find `destroy` functions early on: that's so
that, when destruction is present on multiple paths, particularly
non-shared paths, we only bind the `destroy` method once.
Implementation-wise, I'll note this adds a `has_cleanup` flag to
`TemporaryStorage` and `VarStorage`. There are several related options,
but this felt similar to other information we're trying to track on
instructions. My goal with this is to mitigate the chance of accidental
calls where the storage may not be tracked for destruction. Alternatives
I considered were to not add the flag (I was worried about heightened
risk of errors), or to just add a concept for the relevant `requires`
(which just felt inconsistent).
Cleanup logic ends up in control_flow in this change because I thought
it was a reasonably consistent place for the cleanup block concept and
its pretty direct control flow interactions.
The instruction does act somewhat like a witness, saying that an impl
does exist for a lookup, but the instruction more concretely represents
an impl lookup - since that is done when it is evaluated.
Add a new instruction called ImplSymbolicWitness which represents a
search for an impl declaration given a self type and an interface to
find implemented for the self type. The self type is stored as a
constant instruction id, rather than as a ConstantId, as instructions
don't currently support holding ConstantId. The interface is stored as a
SpecificInterface but we can't fit all of it directly into the
instruction. So we add a new id to refer to the SpecificInterface as
follows.
Add a new SpecificInterfaceId which indexes into a canonical value store
on SemIR::File. This tracks all `SpecificInterface`s stored in an
instruction - specifically the ImplSymbolicWitness instruction.
The SpecificInterface on Impl is still stored there as a value, not as
an id, and no id is eagerly constructed for it. We wait until an id is
needed to make one. Since they are canonical, a new id is only create
when a new SpecificInterface value is seen.
When doing impl lookup, and the query is not concrete, and the impl is
not effectively final, the query needs to consider future impls that may
specialize either the self type or the constaint to make a more precise
match and replace the found impl declaration. Instead of returning the
ImplWitness instruction from the found impl, we generate a
ImplSymbolicWitness instruction, storing the query so that it can be
replayed later. This instruction is added to the generic eval block and
thus will be re-evaluated later with a SpecificId that may make the
query more concrete. When evaluating the instruction and replaying the
query, the lookup has the same conditions and if it does not decide to
use the found impl concretely, then the same instruction is returned
from eval, leaving it as symbolic.
--- Impl lookup changes ---
Impl lookup gets a little more interesting now. It continues to look in
the facet value for a witness if the self type is a facet value. Then
falls back to looking for an impl declaration. This step is no longer
done directly. Instead, we construct a ImplSymbolicWitness instruction
and evaluate it immediately for each interface that are in the query
facet type.
The ImplSymbolicWitness instruction, when evaluated, calls back to the
impl lookup code, with a query specific interface. There we resume back
into the same code path as from before, finding a witness in an impl
declaration. But we may return "found a non-final impl" instead of a
concrete witness. If eval receives this back, it evaluates to the
current ImplSymbolicWitness instruction as the resulting constant value.
To pass lookup failures back through eval, a result of InstId::None from
the second step of impl lookup will result in a non-constant value,
which is used as a signal back up the stack to the original impl lookup
function that the lookup failed. Using a non-constant value here would
break evaluation of the generic eval block if impl lookup could fail
there, however we know it will not since we only leave behind an
ImplSymbolicWitness instruction in the eval block if we found at least
one matching impl already, and we just want to look for a better match
with a more specific query.
We must take care to not store a reference into any value store across
computation in impl lookup, since impl lookup can recurse into itself
invalidate those stores. That includes the SpecificInterface obtained
from a SpecificInterfaceId, which impl lookup also inserts into the
store.
--- The long tail ---
Adding a new instruction and a new id type requires a myriad of changes
to support them:
We add Dump() support for SpecificInterfaceId. And fix a crash in Dump
for SpecificId::None. We also add MakeSpecificInterfaceId() for dumping
arbitrary ids.
The type of ImplSymbolicWitness is a new singleton builtin type
instruction called WitnessSymbolicType (like WitnessType is the type for
an ImplWitness).
Both ImplSymbolicWitness and WitnessSymbolicType are given `Value` as
their expression category as they are builtin constant values. And
BuildInfo() in TypeCompleter is taught about them both, returning a
`ValueRepr::Copy`.
WitnessSymbolicType is added to the set of SingletonInstKinds, so that
it can have a singleton instrution id as a static member.
Lower's BuildTypeForInst() is taught to make an empty struct for
WitnessSymbolicType, similar to WitnessType.
Instruction formatter (FormatterImpl) grows support for printing a
SpecificInterfaceId so that it can print both arguments of
ImplSymbolicWitness on the RHS when printing the SemIR instruction. To
print a SpecificInterfaceId, it prints both the interface id and the
specific id (if there is one). For example, for a query on a generic
interface `Z` with one parameter, the RHS includes the query, interface,
and specific:
```
%Z.impl_symbolic_witness: <symbolic witness> = impl_symbolic_witness %U, @Z, @Z(%U.as_type) [symbolic]
```
IdKind is extended to include SpecificInterfaceId.
InstFingerprinter is taught to look through SpecificInterfaceId and use
the interface and specific ids in the fingerprint.
InstNamer is taught about SpecificInterfaceId, counting the interfaces
when building an index. It is also tought about ImplSymbolicWitness,
using the name of the interface within and the `.impl_symbolic_witness`
suffix. For example, here the LHS is named after the interface in the
query:
```
%Z.impl_symbolic_witness: <symbolic witness> = impl_symbolic_witness %U, @Z, @Z(%U.as_type) [symbolic]
```
StringifyTypeExpr is taught about WitnessSymbolicType, which uses its IR
name since it's a singleton. And about ImplSymbolicWitness which uses
its constant value. The handling of ImplWitnessAccess also needed to be
adjusted, since it assumed that ImplWitnessAccess::witness_id would
always be a FacetAccessWitness, but it can now also be an
ImplSymbolicWitness. (It seems that the witness_id is also assigned
ImplWitness instructions, but those ImplWitnessAccess instructions don't
ever seem to get stringified in a diagnostic at this time.) At the
moment the ImplWitnessAccess with a symbolic witness is just stringified
as "<symbolic>", such as in:
```
x.carbon:1:2: error: cannot implicitly convert value of type `()` to `<symbolic>` [ConversionFailure]
let a: C(D).(Z.X) = ();
^~
```
There is a TODO left behind to include more information there.
The TypeStructure builder is made to handle WitnessSymbolicType and
WitnessType. These come up now in deduce where a generic impl will have
a ImplSymbolicWitness in a FacetValue for a generic self type. The query
may have a concrete ImplWitness in the same position. Since deduce tries
to deduce through the FacetValue, it tries to convert ImplWitness to
ImplSymbolicWitness, tries to do an impl lookup for `impl ImplWitness as
ImplicitAs(ImplSymbolicWitness)` and causes us to build type structures
with each of these.
Subst is updated to handle pushing and popping SpecificInterfaceId.
Without this, when finishing a generic's eval block, we would walk into
the ImplSymbolicWitness instruction, and its arguments, and fail to
recurse down into the SpecificInterfaceId. Then any specifics inside
would be left as "orphaned" without any generic id attached to them, and
we would never update the instructions in the SpecificInterface's
instructions (inside its own SpecificId) with new constant values when
evaluating the generic eval block against a specific. To do this we push
the specific_id inside the SpecificInterface, and when popping we pop
the specific_id then construct a new canonical SpecificInterface with it
and return that id.
We add support for importing ImplSymbolicWitness by importing its self
constant instruction and specific interface id. However we also had to
add import support for SpecificImplFunction, which can now appear in the
generic eval block for a generic impl declaration, and thus must be
imported with the declaration. This is done very similarly to
SpecificFunction, except the `type_id` is a singleton value.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
When performing a call through an impl witness, the callee that we
type-check against is the function in the interface, so we form a
specific for that callee. However, once the impl witness access
resolves, the eventual callee is a different function -- the function in
the impl -- so this would cause us to form a `SpecificFunction` where
the callee is one function but the specific refers to a different
function.
Address this by adding another instruction, `SpecificImplFunction`, that
takes a function in an impl and a specific for the corresponding
function in the interface, and computes and returns a `SpecificFunction`
referring to the corresponding specific function in the impl, or returns
a direct reference to the function in the `impl` if it's not a generic
function.
Expands the CompleteTypeInfo to include information about abstract
classes and computes that information as part of completing the type.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Dana Jansens <danakj@orodu.net>
Parameter lists need substantially different treatment than tuple
patterns in other contexts, so this change splits them into separate
parse node kinds.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This allows iterating on all values in a store along with the Id for
each value, instead of `llvm::enumerate(store.array_ref())` which would
give you the indices.
While the indices are really the same as the Ids, this provides a
typesafe way to enumerate() over a store.
There's no use for this right now, but I thought I needed this, and it
helped me debug, and it was a pain to write correctly without dangling
references.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Change representation of package names from `IdentifierId` to
`PackageNameId`, and add a special value `PackageNameId::Core` for the
Core package. Add a `Core` expression to name the Core package, and
support for parsing the `Core` keyword in `package` and `import`
declarations.
For now, I've made no changes to instruction fingerprinting or name
mangling. This means that fingerprints and mangled names will collide
between names in the `Core` package and names in a `r#Core` package. See
#4908.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
For an expression such as `(Type as Interface).AssocFn()`, track the
`Self` type `Type` in the result of the member access so that it's
available when checking the function call.
This introduces a new kind of type, `ImplFunctionType`, that represents
the type of a function that is expected within an impl, modeled as the
type of the function within the interface plus a value to use as `Self`.
Calls to values of this type behave like calls to the underlying
function except that the `Self` parameter is pre-bound to the self type
from the facet.
In order to support this, fix an issue where the imported list of
generic bindings lost their association with their enclosing generic.
This adds a little complexity to `import_ref`, including a new recursive
cycle that I intend to address in a follow-up PR.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
High level, replacing `Id::Invalid` with `Id::None` and `Id::is_valid`
with `Id::has_value` for clarity, as discussed
[here](https://discord.com/channels/655572317891461132/655578254970716160/1331664574545395794).
The `IntId` refactoring is needed together with `AnyIdBase` because it's
also used with `ValueStore`.
Note, trying to be careful not to rewrite `EnumBase::InvalidIndex`, or
`is_valid` in general (e.g., `IdKind::is_valid`).
I've tried to sequence commits here:
1. Automatic replacements:
- `((?:Id|Index)(?: |::|\(|Base(?:\(|::)))Invalid((?:Index)?\W)` ->
`$1None$2`
- `<invalid>` -> `<none>`
- `InvalidNodeId` -> `NoneNodeId`
- `/\*invalid\*/` -> `/*none*/`
- `id((?:_|\(\))(?:\.|->))is_valid` -> `id$1has_value`
2. Manual edits:
- In `int.h` and `int_test.cpp`
- `IntT` has `is_value`, which I'm renaming to `is_embedded_value`.
- Manual edits to comments in this file.
- `AnyIdBase` and `IdBase`
- Declaration of `is_valid` -> `has_value`, `InvalidIndex` ->
`NoneIndex`.
- In `ids.h` and `ids.cpp`
- `is_valid` -> `has_value`
- `// An explicitly invalid ID.` -> `// An ID with no value.`; similar
for index
- Various math on `InvalidIndex` -> `NoneIndex`
- Various mentions of "valid" in comments
- In `value_store.h`, for `IdT::Invalid`, plus one comment
- In `impl.h` and `tokenized_buffer.h`, we had different initialization
of `::None` values (versus `ids.h` syntax) that I fixed manually.
- Spot checks to compile
- Particularly where `is_valid` replacements didn't catch spots due to
different naming.
3. Autoupdate tests
4. verbose.carbon (NOAUTOUPDATE)
5. Comment spot checks
Note there are probably other mentions of "Invalid" that should be swept
up, but I'd like to argue for merging and separating out remaining
cleanup since this is so sweeping (and likely to hit merge conflicts
from churn). We'll probably have lingering mentions of "invalid" for a
bit regardless, just because there are uses of "invalid" in non-Id APIs.
Add a Vtable typed inst with a type_id (of the type this vtable applies
to) and list of virtual function decls (or import refs to function
object constants).
This doesn't add lowering/emission of the vtable, or usage when
initializing objects of the type.
Some questions in case they're interesting to discuss:
* is it right/worth having the type_id in the vtable? (probably makes it
easier to emit - using the type to get the class name to figure out the
mangled name for the vtable) perhaps it should be a ClassId?
* I'm thinking the logic in CheckCompleteClassType could be the place we
handle diagnostics for mismatched keywords (virtual/abstract for a
function that's already virtual/abstract, maybe checking for non-virtual
functions with the same name in a base class, or derived class functions
without `impl`, etc) - but we could move some of that to the moment we
walk the function decl, and record our findings in the function decl
(record the base function it overrides, or the index of the vtable to
slot to use when building the vtable at the end of the class)
* the Vtable typed inst has `constant_kind = InstConstantKind::Always`
and `is_lowered = false`, I think I added that in to workaround/address
some failures in lowering. And seems correct for this intermediate step
- I'll add lowering in a follow-up patch. But the constant_kind - what
should this be? We can just say all vtables are of VtableType (in which
case the `Always` constant kind sounds right to me) or we could have
them introduce a type with each virtual function as a named member,
even?
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
* Change `InterfaceWitness` -> `ImplWitness`
* Include a `SpecificId` in the `ImplWitness`. This allows the
`InstBlock` it contains to have its own identity, allowing it to be
changed as the impl is processed. Evaluation only updates the specific.
* Create the `ImplWitness` at the start of the impl definition. In the
future, this will be populated with the values of non-function
associated constants. For now, it starts full of invalid instruction
ids.
* Implements the model suggested in #4672 .
Note that the non-SemIR testdata changes are to these file:
* `toolchain/check/testdata/impl/lookup/fail_todo_undefined_impl.carbon`
* `toolchain/check/testdata/struct/import.carbon`
* `toolchain/check/testdata/tuple/import.carbon`
The last two are due to an import of generics bug exposed by this PR,
which will be fixed in a follow-on.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Instead of treating `Core.Int` as the toolchain's builtin `IntType`,
model it as a class that adapts the builtin type. This aligns us better
with the intended language model, gives an associated library for
`impl`s involving `Core.Int` to live within, and opens the door adding
member functions to `Core.Int` if we decide that is desirable.
Remarkably it also seems to make the formatted SemIR a little smaller,
because a call to a generic class generates less IR than a call to a
function.
When a generic requires a symbolic type to be complete, add a new
`require_complete_type` instruction to the generic eval block. During
monomorphization of such an instruction, require that type to be
complete.
David Blaikie
Boaz Brickner
Thomas Köppe
Geoff RomerandRichard Smith
