High level, replacing `Id::Invalid` with `Id::None` and `Id::is_valid`
with `Id::has_value` for clarity, as discussed
[here](https://discord.com/channels/655572317891461132/655578254970716160/1331664574545395794).
The `IntId` refactoring is needed together with `AnyIdBase` because it's
also used with `ValueStore`.
Note, trying to be careful not to rewrite `EnumBase::InvalidIndex`, or
`is_valid` in general (e.g., `IdKind::is_valid`).
I've tried to sequence commits here:
1. Automatic replacements:
- `((?:Id|Index)(?: |::|\(|Base(?:\(|::)))Invalid((?:Index)?\W)` ->
`$1None$2`
- `<invalid>` -> `<none>`
- `InvalidNodeId` -> `NoneNodeId`
- `/\*invalid\*/` -> `/*none*/`
- `id((?:_|\(\))(?:\.|->))is_valid` -> `id$1has_value`
2. Manual edits:
- In `int.h` and `int_test.cpp`
- `IntT` has `is_value`, which I'm renaming to `is_embedded_value`.
- Manual edits to comments in this file.
- `AnyIdBase` and `IdBase`
- Declaration of `is_valid` -> `has_value`, `InvalidIndex` ->
`NoneIndex`.
- In `ids.h` and `ids.cpp`
- `is_valid` -> `has_value`
- `// An explicitly invalid ID.` -> `// An ID with no value.`; similar
for index
- Various math on `InvalidIndex` -> `NoneIndex`
- Various mentions of "valid" in comments
- In `value_store.h`, for `IdT::Invalid`, plus one comment
- In `impl.h` and `tokenized_buffer.h`, we had different initialization
of `::None` values (versus `ids.h` syntax) that I fixed manually.
- Spot checks to compile
- Particularly where `is_valid` replacements didn't catch spots due to
different naming.
3. Autoupdate tests
4. verbose.carbon (NOAUTOUPDATE)
5. Comment spot checks
Note there are probably other mentions of "Invalid" that should be swept
up, but I'd like to argue for merging and separating out remaining
cleanup since this is so sweeping (and likely to hit merge conflicts
from churn). We'll probably have lingering mentions of "invalid" for a
bit regardless, just because there are uses of "invalid" in non-Id APIs.
Add a Vtable typed inst with a type_id (of the type this vtable applies
to) and list of virtual function decls (or import refs to function
object constants).
This doesn't add lowering/emission of the vtable, or usage when
initializing objects of the type.
Some questions in case they're interesting to discuss:
* is it right/worth having the type_id in the vtable? (probably makes it
easier to emit - using the type to get the class name to figure out the
mangled name for the vtable) perhaps it should be a ClassId?
* I'm thinking the logic in CheckCompleteClassType could be the place we
handle diagnostics for mismatched keywords (virtual/abstract for a
function that's already virtual/abstract, maybe checking for non-virtual
functions with the same name in a base class, or derived class functions
without `impl`, etc) - but we could move some of that to the moment we
walk the function decl, and record our findings in the function decl
(record the base function it overrides, or the index of the vtable to
slot to use when building the vtable at the end of the class)
* the Vtable typed inst has `constant_kind = InstConstantKind::Always`
and `is_lowered = false`, I think I added that in to workaround/address
some failures in lowering. And seems correct for this intermediate step
- I'll add lowering in a follow-up patch. But the constant_kind - what
should this be? We can just say all vtables are of VtableType (in which
case the `Always` constant kind sounds right to me) or we could have
them introduce a type with each virtual function as a named member,
even?
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Rename `CheckIsAllowedRedecl` to `DiagnoseIfInvalidRedecl` to try to
better document behavior, and clean up comments.
This extends the no-merge-if-defined behavior to functions. It was
already the case for class/interface, and just added for impl, so if
anything functions were now inconsistent. I was kind of tempted to make
a helper for it, but I didn't think of a great structure/name to get
there: `DiagnoseRedef` isn't always called when it's a redefinition, for
example due to `extern` diagnostics, it's hard to combine.
Cleans up `is_defined` calls to rely more on `has_definition_started`,
removing some code paths that are unused since definitions aren't
merged.
This change splits `NodeKind::IdentifierName` into separate node kinds
depending on whether the identifier is followed by parameters, and
similarly splits `NameQualifier` based on whether the qualifier has
parameters. This enables us to only push a pattern block when it's
actually needed, rather than "defensively" pushing one when it might be
needed.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Change the implementation to use an explicit `is_poisoned` bit instead
of `InstId::PoisonedName` value.
Zero behavior change.
This would allow to more easily change the API to support accessing the
poisoning declaration so we can have better name poisoning diagnosis.
#4622
* Change `InterfaceWitness` -> `ImplWitness`
* Include a `SpecificId` in the `ImplWitness`. This allows the
`InstBlock` it contains to have its own identity, allowing it to be
changed as the impl is processed. Evaluation only updates the specific.
* Create the `ImplWitness` at the start of the impl definition. In the
future, this will be populated with the values of non-function
associated constants. For now, it starts full of invalid instruction
ids.
* Implements the model suggested in #4672 .
Note that the non-SemIR testdata changes are to these file:
* `toolchain/check/testdata/impl/lookup/fail_todo_undefined_impl.carbon`
* `toolchain/check/testdata/struct/import.carbon`
* `toolchain/check/testdata/tuple/import.carbon`
The last two are due to an import of generics bug exposed by this PR,
which will be fixed in a follow-on.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
When a generic requires a symbolic type to be complete, add a new
`require_complete_type` instruction to the generic eval block. During
monomorphization of such an instruction, require that type to be
complete.
This is a preparation change for adding name poisoning support
(https://github.com/carbon-language/carbon-lang/issues/4622), which is
expected to require more elaborate logic around NameScope since a name
can be not defined yet, defined, or poisoned.
The API separates looking up a name from getting the full entry since we
have cases where the entries are invalidated between the time we're
looking for the name and when we access (and sometimes modify) the
entry.
This change has the following benefits:
* `names` and `name_map` are internal to `NameScope` and are guaranteed
to match.
* `extended_scopes` and `import_ir_scopes` can not be manipulated (only
new scopes can be added).
* `inst_id`, `name_id` and `parent_scope_id` are constants.
* `has_error` can only be mutated from false to true.
---------
Co-authored-by: jonmeow <jperkins@google.com>
`ids.h` and `ids.cpp` are the manual edits, everything else is
search-and-replace.
The full list of things moved is:
- `TypeId::TypeType`
- `TypeId::AutoType`
- `TypeId::Error`
- `ConstantId::Error`
This is to unblock removing `InstId::Builtin*`.
Represent the type as an `InstId` rather than as a `TypeId` to preserve
how it was written and better support tracking its value in a generic.
Add accessors to `Class` to get the base and adapted type to reduce code
duplication, and add `TypeStore::GetObjectRepr` to make it easier to map
from a type to its possibly-adapted object representation type. In
passing, also move `GetIntTypeInfo` and `GetUnqualifiedType` into
`TypeStore`.
This fixes specifics of generic adapters to properly look at the
specific adapted type, and also fixes importing of adapters.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
We considered a couple of other options for this:
* https://github.com/carbon-language/carbon-lang/pull/4515 Keep the
`ElementIndex` numbering vptr-ignorant, and do +1 offsets as needed -
seems subtle/easy to miss
* https://github.com/carbon-language/carbon-lang/pull/4517 Always have a
zeroth element in the object representation, make it zero-size in the
case of no-vptr - @zygoloid was concerned this would add overhead
especially to stateless objects used in type-trait-like things.
But currently moving forward with this direction - of initializing field
indexes with an invalid value until the end of the class definition,
then assigning field indexes during construction of the class's object
representation struct type. This direction might reinforce/help avoid
premature access to the object representation before the class is
complete, and give a single place where class layout is done (at class
completion) if we want to add more options there, such as class layout
optimizations, etc.
This patch still has problems with object initialization (that #4515
does not have/does address) but does address normal `obj.member` access
correctly.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This is for more clearly distinct names, and to make it a clearer
transition from `BuiltinInst` for name conflicts. `FloatType` is also an
instruction, and we have `Carbon::Error` (common/error.h). This avoids
affecting tests, although the name is embedded in the builtin test.
In `LegacyFloatType`, `Legacy` because I was having trouble coming up
with a more appropriate name. I'm not clear this is a `FloatLiteralType`
at present, it needs some work to mirror `IntLiteralType`.
In `ErrorInst`, the suffix `Inst` was discussed as good and similar to
`BuiltinInst` (although I'm trying to get rid of that).
Stop rejecting `extend adapt` of non-class types such as struct and
tuple. These don't actually work just yet because name lookup into
struct and tuple types is a special case that doesn't handle adapters,
but this gets us a bit closer.
This also slightly improves error recovery for name lookup into an
invalid scope.
* The `extended_scopes` in a `NameScope` were represented by a
`NameScopeId`. Replace that with an `InstId` of an instruction returning
the type that is extending this name scope.
* `Context::LookupQualifiedName` now can take multiple scopes to look
in.
* `GetAsLookupScope` was moved out of `member_access.cpp` and is now
`Context::AppendLookupScopesForConstant`
This PR also fixes some existing issues that were revealed as part of
writing and testing this PR:
* Additional validation and handling of invalid ids.
* `extend impl` in a class is not properly imported yet, but at least
now it doesn't crash.
The change to use an `InstId` also allowed some diagnostics and
formatting to be improved.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
This converts `StructTypeField` from an instruction to a dedicated type,
with its own store. This had originated from discussing how
`.GetAs<SemIR::StructTypeField>` was more prevalent than for other
instructions, but is probably more interesting for the storage savings
(16 bytes StructTypeField + 4 byte LocId + 4 byte InstId -> 8 byte
StructTypeField).
Due to the different structure, these now have their own stack during
construction, reducing (but not eliminating) `args_type_info_stack_`
use-cases.
The test changes of different InstIds is expected because structs and
classes generate fewer instructions now. Other than that, results should
remain the same.
I'm generally trying to avoid unrelated cleanup here due to the PR size,
though I did scrutinize the `VerifyOnFinish` calls, adding one and
commenting others (putting them in member order because that's how I was
checking what was verified and what wasn't).
- Generate runtime indices as part of pattern matching, rather than as a
separate postprocessing/rewriting step.
- In contexts where runtime parameters aren't permitted, avoid emitting
insts for them to begin with, rather than trying to detect the problem
and rewrite the IR to remove them later on.
A small step to virtual functions - adding vtable pointers to the
layout, but not initializing or otherwise using them at this stage.
A few open design questions I'd love feedback on:
* Is this the right/good enough SemIR representation for now? This patch
adds a `is_dynamic` attribute to `SemIR::Class` and populates/flags it
based on the flag of the base class, or if any virtual function is
declared in the class (or, at least that's my intent). Some other
options include:
* Each `Class` could store a `ClassId` (or `TypeId`?) of the (possibly
indirect, possibly self) base class that is the first one that is
dynamic/has a vtable pointer
* Could make the property narrower, like `has vtable pointer` and have
it `true` only on the type that introduces the vtable - then derived
classes would have to walk their base classes to check if they're the
one that needs to define the vtable pointer or not
* Should the vtable be the first element in the type? If there's a
non-dynamic base type, we could have a layout that's `{<non-dynamic base
type>, vtable ptr, <derived members>}`? Derived types would still be
able to uniquely identify where their vtable pointer is just fine... -
and the vtable pointer is, in a sense, a member of that intermediate
type, so it does seem a bit strange to force it to the front - but I
guess it's probably more efficient in some ways?
Open to any other suggestions/advice/thoughts on the direction, etc.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Also propagate the pattern IR along with the pattern-match IR, and use
it where appropriate.
Strictly speaking, some parts of the pattern-match IR are allocated
eagerly, while traversing the pattern's parse tree, but they still
aren't actually emitted until we traverse the associated pattern insts.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
A good first-pass, at least. (abstract adapters are rejected with this
change, though pending further language design discussion)
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Instead of stringifying types in the caller in some cases, add new types
to represent:
- `InstIdAsType`: an `InstId` diagnostic argument that represents a type
expression that should be included in the diagnostic
- `InstIdAsTypeOfExpr`: an `InstId` diagnostic argument that represents
an expression whose type should be included in the diagnostic
For these cases, we can produce more user-friendly descriptions of a
type than we can with a canonicalized `TypeId`. Add comments to
discourage using `TypeId` diagnostic arguments when one of the above can
be used, and move over existing uses where it's straightforward to do
so.
Move type stringification code to its own files and out of `SemIR::File`
to make `File` smaller and to further discourage the direct use of the
stringification logic.
Also update type printing to include the `` ` `` delimiters surrounding
the type. The intent is that we will eventually want to include other
information when formatting a type, like Clang does when printing a
typedef (`'string' (aka 'std::basic_string<char>')`), and such
formatting requires that the diagnostic machinery produces the `` ` ``s
itself.
There are a couple of cases where we really want to format valid Carbon
type syntax directly into a diagnostic, rather than an `aka` or similar,
because the diagnostic text includes part of the type itself, for
example: ``"consider using `partial {0}`"``. For such cases, a `Raw`
form of the diagnostic argument types is added: `TypeIdAsRawType` and
`InstIdAsRawType`. In principle we could instead use ``"consider using
`partial {0:raw}`"``, but our diagnostic machinery isn't set up for
that.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
According to
https://docs.carbon-lang.dev/docs/design/generics/details.html#adapting-types:
> You can add any declaration that you could add to a class except for
declarations that would change the representation of the type. This
means you can add methods, functions, interface implementations, and
aliases, but not fields, base classes, or virtual functions. The
specific implementations of virtual functions are part of the type
representation, and so no virtual functions may be overridden in an
adapter either.
So, let's check/reject that.
Checking at the end of the class ensures that no matter the order of
methods and adapt statements, the issue will still be correctly
diagnosed.
Introduces the `BindingPattern` and `SymbolicBindingPattern` insts, and
a separate stack of pattern blocks that they are emitted into. The
intent is to generate the corresponding pattern-matching insts (like
`BindName`) from them in a separate pass, but that is deferred to future
PRs.
See
[here](https://docs.google.com/document/d/1U_vQH17V893J9aF1LJXUnFYBNSs2MjKl4bJPaWCB2zo/edit?usp=sharing&resourcekey=0-w0xGYZ0An31Kpz-wvzSXwQ)
for the design this is based on, but note that during review we have
chosen to deviate from that design by putting the patterns in separate
blocks, and omitting the "forward references" from a `BindingPattern` to
its corresponding `BindName`. This in turn necessitates having separate
inst kinds for symbolic and non-symbolic binding patterns.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Doing this to a couple of diagnostics was suggested in review comments
on #4320, so I've applied the suggestions across the whole file.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Add support for initializing types like `GenericClass(i32)` from a
struct literal. A new kind of instruction, `complete_type_witness`, is
added to the class definition to track the object representation type so
that it's visible to the generics machinery. Accesses to the object
representation of a class have all been updated to pass in the class's
`SpecificId` so that the types of the fields of the specific class are
used instead of the types of the fields of the generic class in places
that look at the object representation -- primarily class
initialization.
Instead of the `call` instruction having a block with one argument per
explicit argument, preceded optionally by `self` and followed optionally
by a return slot, change the `call` to store only the *runtime*
arguments. Store an index on the runtime parameters to make it easier to
determine the correspondence between arguments and parameters in a call.
Compile-time parameters, whether implicit or explicit, are no longer
included in the call argument list. Instead, they're tracked only in the
`specific_id` on the callee.
For calls to generic classes and generic interfaces, it no longer makes
sense to form a `call` instruction, given that the entirety of the
result is determined by the `specific_id`, which is now formed when
checking the call. Instead, the `call` instruction now only models
function calls, and not calls to other kinds of parameterized entity
names, and we create a `class_type` or `interface_type` instead of a
`call` instruction to model these kinds of calls. Notionally the model
here is that we're following the #3720 approach for calls, but for now
we inline the `Call.Op` function when forming SemIR.
We now also track the enclosing specific for a generic class or generic
interface that appears within an enclosing generic. This is necessary in
order for deduction of the inner generic parameters to not get confused
by the outer generic parameters being absent.
In order to not regress diagnostics, the template argument deduction
mechanism has been extended to specify the name of the parameter we're
deducing against when possible, and call arity mismatch errors are now
diagnosed before performing deduction rather than afterwards.
This switches `DCHECK` and `FATAL` as well.
The goal is to reduce the code size impact of these assertions so that
we can keep more of them enabled. Currently, the largest cost I see from
`CHECK` is not the actual check or the cold code itself, but actually
the failure to inline trivial functions due to the presence of the cold
code. This means that our goal isn't to reduce apparent code size in the
final binary but the LLVM IR cost assessed for these routines in the
inliner, which closely correlates with code size but is a bit different.
As discussed in #4283, experimentation shows that a single function call
with a minimal number of arguments is the lowest cost model for these.
This is easily achieved with a format-string API that internally uses
`llvm::formatv`. This PR is essentially the `CHECK` version of #4283.
However, the check macros are substantially harder to make work with
both format strings and streaming because they also take a condition.
Also, unexpectedly, I was very successful at devising a regular
expression based automated rewrite from the streaming to the format
string form with only low 10s of manual fixes. This includes compacting
strings broken up across lines, etc. Given how well that went, I've
prepared this PR which just directly switches to the format string API
and migrate everything to use it.
One nice side-effect is that the format string approach ends up greatly
simplifying the implementation here as well.
This is ... *shockingly* effective. Parsing speeds up by more than 3%
with just this change. And checking speeds up by **8%** with this change
alone:
```
BM_CompileAPIFileDenseDecls<Phase::Parse>/256 86.3µs ± 1% 82.9µs ± 1% -3.94% (p=0.000 n=17+19)
BM_CompileAPIFileDenseDecls<Phase::Parse>/1024 431µs ± 1% 415µs ± 1% -3.76% (p=0.000 n=18+19)
BM_CompileAPIFileDenseDecls<Phase::Parse>/4096 1.77ms ± 1% 1.71ms ± 1% -3.18% (p=0.000 n=18+19)
BM_CompileAPIFileDenseDecls<Phase::Parse>/16384 7.44ms ± 1% 7.17ms ± 2% -3.56% (p=0.000 n=18+20)
BM_CompileAPIFileDenseDecls<Phase::Parse>/65536 30.7ms ± 1% 29.7ms ± 1% -3.15% (p=0.000 n=18+20)
BM_CompileAPIFileDenseDecls<Phase::Parse>/262144 131ms ± 1% 127ms ± 1% -2.81% (p=0.000 n=18+18)
BM_CompileAPIFileDenseDecls<Phase::Check>/256 878µs ± 2% 800µs ± 1% -8.91% (p=0.000 n=19+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/1024 1.88ms ± 2% 1.72ms ± 1% -8.56% (p=0.000 n=19+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/4096 5.78ms ± 2% 5.28ms ± 1% -8.70% (p=0.000 n=20+18)
BM_CompileAPIFileDenseDecls<Phase::Check>/16384 21.9ms ± 1% 20.1ms ± 1% -8.02% (p=0.000 n=18+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/65536 90.4ms ± 2% 83.1ms ± 1% -8.04% (p=0.000 n=19+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/262144 381ms ± 2% 352ms ± 1% -7.79% (p=0.000 n=19+19)
```
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
The particular test this focused on is indirect_two_file in
toolchain/check/testdata/function/definition/no_prelude/extern_library.carbon.
This removes `parent_scope_id_for_new_inst` because I think it's
returning unhelpful results. The use was at the root of incorrect
results for the indirect import chain. `name_id_for_new_inst` is
actually wrapping a union, so it's more important.
The merging of `is_extern` and `first_owning_decl_id` in
`handle_function.cpp` feels like it's less correct with the changes
that've been made to `extern`. This ripples in tests, because the error
recovery shifts.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Support for types (particularly classes) is left as a TODO.
There's also an issue I'm observing with a "define in impl" test, but
this is probably an issue with resolving the prior declaration which is
imported indirectly. The PR was already feeling big, so I'm choosing to
cut here.
Note, this does not implement the rule "The owning library's API file
must import the `extern` declaration, and must also contain a
declaration."
This adds fields to `EntityWithParamsBase` to reflect the intention with
`extern library` design. I'm renaming `decl_id` because it shouldn't be
expected to be assigned anymore. import_ref.cpp I'm deliberately keeping
on `first_owning_decl_id` (which will break when importing `extern
library` declarations). Most other cases are for diagnostics, and I'm
using `latest_decl_id` to try and get the closest declaration to the
error. Note I'm partly splitting out this PR to show the test effect,
which apparently we don't test related cases.
It's actually possible to get into all four combinations of having
parameter lists versus being generic:
- An entity nested within a generic, such as a member class, can be
generic even if it has no parameters.
- As a corner case, an entity with an *empty* parameter list has
parameter lists, but isn't a generic because it doesn't have any generic
parameters.
As discussed in toolchain meeting, we want to avoid overloading the
meaning of "instance", and "specific" was the best name we found. It's a
little unorthodox and inventive, but hopefully over time will become as
unsurprising as the term "generic" is.
I'd considered moving DeclParams uses over, but when handling qualified
names, there's a parse node instead of an instruction. I did try to
unify a couple other uses though, including adding MergeDefinition. I
expect `interface` will use a little more once it's more completely
implemented, but maybe I'm wrong about that.
Note this isn't implementing checking through imports. The parse node
there is harder to access through the context, so would require
examining the entity in order to get the import declaration, to get at
the ImportIR. We also don't have a parse tree attached in that case, and
would need to add one to SemIR::File. But I believe we do want to add
that, so it's explicitly a TODO.
Note GetTokenText re-lexes literal values, so there's a bit of potential
overhead there. Not sure if we want a more efficient manner for
comparing in cases like this.
This is for consistency with #4120. Similar to that, we can use
overloads on the typed NodeId rather than individually named handlers.
There isn't the same caller benefit here though, since the calls from
check.cpp are already boilerplate.
We can't use the instruction from the generic directly, because it
doesn't have the right constant value. Instead add an instruction that
models the transition from the constant value in the generic to the
constant value in the generic instance.
Also start associating the self generic instance with unqualified
lookups that find results in an enclosing generic, so that we track the
information necessary to create the new instruction.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
When checking a declaration or definition of a generic, track a list of
created instructions that depend on the generic's parameters in some
way, along with information on how they depend on the parameters. This
will eventually be used to determine what information we need to compute
when creating instances of the generic, but for now we're just building
the list.
Information is tracked separately for the declaration region and the
definition region of the generic, because in general these may be first
provided in separate declarations, and they should be substituted into
at different times.
In a `class C(T:! type)`, the type `Self` should be `C(T)`, not merely
`C`. Similarly, in an `interface I(T:! type)`, the type of self should
be `I(T)`, not merely `I`.
In `ClassType`s and `InterfaceType`s, track a `GenericInstanceId` for
the instance rather than just the argument list.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Jon Ross-Perkins
Boaz Brickner
Geoff RomerandJon Ross-Perkins
