Remove the type canonicalization mechanism and instead rely on constant
canonicalization to deduplicate types.
Rename the `Canonicalize*Type` functions to reflect that they're no
longer performing canonicalization. Switch code that creates types due
to semantic checking, rather than due to source syntax, to directly
create type constants through evaluation rather than creating an
instruction and evaluating it to produce a separate constant
representation.
The mapping from `const (const T)` that was previously performed by type
canonicalization is now implemented in expression evaluation instead.
The value `<error>` is now treated as a constant value, with a special
property that an instruction involving `<error>` that could possibly be
constant evaluates to `<error>`. This helps avoid producing follow-on
errors when an error occurs as a subexpression of an expression, such as
a type, that is intended to be constant.
Rather than producing multiple constants with the same value, fold all
instances of a given constant to the same constant instruction.
A future PR will use this to replace the current type canonicalization
system.
Instructions created by splices during conversion are now evaluated, as
are instructions created in cases where we first create a placeholder
instruction and later replace it by a different instruction.
This also removes the ability to set a parse node and instruction
independently after creating an `InstId`, which could lead to them
accidentally not matching.
This is accomplished by tracking an extra bit on the ID we store in the
constant values table, and propagating that from subexpressions to the
enclosing expression. This extra bit is not yet computed correctly for
types; that will be addressed in later PRs.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Form a side table with constant values for each instruction. Evaluation
is only supported for a few very simple kinds of instruction for now.
This is not observable outside of the SemIR output, because nothing
depends on expressions having a constant value phase yet.
Namespaces are copied, which means also adding their name to the
underlying instruction. It happened not to be done previously; the name
was only in name lookup.
Since the only import supported right now is the default import,
functionality is limited; in the future I'll need to deal with namespace
vs package conflicts.
Tests of namespace imports are under "namespace" -- I figured this would
be best for scaling as more instructions get support.
This also improves some debugging-related output that I was trying to
use while trying to build the support.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This is a step towards adding enclosing scopes for imports. It creates
an indirection for all bind names.
We discussed specializing for bindings that are in function scope (i.e.,
not a useful enclosing scope for imports or diagnostics). However, the
thought is to go ahead with this singular approach for now, and only
change structure if it's a performance issues so that we have
incrementally fewer instructions to handle.
When formatting special values, we currently have `NameId` doing `if
(*this == SelfValue)`, `BoolValue` doing `case 0:`, and `TypeId` doing
`if (index == TypeType.index)`. I'm suggesting we consolidate onto the
`*this == SelfValue` approach for consistency, it seems the easiest to
see the mapping of values.
This mixes in #3552 because I'm adding `Exports` there. I'm suggesting
reformatting `Empty` and `Exports` consistent with `Unreachable`.
This adds a block for exported InstIds, rather than scanning the package
scope. This working down a path discussed last month, which we'll need
to add enclosing namespaces to the Inst in order to complete import of
something like `namespace NS; var NS.a;`
Exports could've been a separate `vector<InstId>` on `SemIR::File`, but
using an entry in `inst_blocks` felt more consistent.
The goal here is to make the representation more uniform so that we can
start adding different kinds of binding -- checked generic bindings and
template bindings -- across both function parameters and local `let`
declarations.
With this change, the entry in the parameter list for the function is
the name binding, not the Param itself, which has some ripple effects on
consumers of that list that want to access the parameter rather than the
binding. This is expected to change again when we start adding more of
the pattern matching SemIR, but this seems good enough for now.
Another incremental change to diagnostic formatting. I simply recurse
over all the tokens in the subtree of a parse node and construct a
`DiagnosticLocation` that covers all of the tokens.
I believe it's nicer for the user to be directed at the entire chunk of
source where the error is occurring rather then just pointing at the
bracketing/terminator tokens, but let me know if you all agree.
BinaryOperatorAdd had been added early on as a proof-of-concept for
toolchain design for parsing -> checking flow. However, it doesn't
reflect the interface direction for operators, and now other portions of
the toolchain demonstrate the relevant logic. Instead, it's just a bit
of an outlier versus other instructions which have evolved over time.
This creates a namespace for `package` scope.
It looks like names of class_decls in namespaces lead to an unexpected
instref. This is already true, as best as I can tell. I'm not sure if
there's a preferred approach to address that, so I've left a TODO for
now.
This removes the filename from the file-scoped block, and places it
above to make it clear where the full SemIR begins (with multifile,
providing a barrier between).
I was suggesting this because `FloatingPoint` is pretty long. `int` and
`float` should be familiar abbreviations. `unsigned` should be familiar
to developers too, but `UnsignedInt` still feels usefully clearer for
the additional chars.
This is an incremental improvement on our diagnostic messages that
simply underlines an entire token if the token is larger than 1 char
(else it points to the single char with a caret like it used to).
Instead of ad-hoc conversion tracking on some kinds of nodes that
conversion creates, consolidate tracking into a single node kind. This
frees up an operand on `Init` instructions that can be used to store the
destination.
Add a `NameId` that is effectively just a wrapper around a `StringId`,
with
some additional predefined values for names that don't correspond to
strings, such as the name of `self` or the function's return slot.
Following up on discussion yesterday regarding this split.
Note, I'm expecting #3341 to do IdentifierId -> NameId in SemIR. It
might be worth adding NameId creation directly to StringStore if you're
content with this setup though.
Advantages:
- Allows lexing/parsing in parallel, since they are modifying fully
separate ValueStores.
- Allows SemIR to reliably be stored hermetically.
Disadvantages:
- Creates overlapping storage of duplicate strings when multiple files
are compiled together.
- Prevents Ids from being uniquely compared cross-file.
Per discussion, the decision is that the advantages are more important.
The looser ownership remains because both SemIR checking and
metaprogramming may still generate things we would want to deduplicate.
It would be somewhat odd if TokenizedBuffer owned something that
checking modified.
I'm looking at this due to the conversation on #3341. Although
diagnostics aren't where they should be, I thought it may help to start
adding raw identifier support (which may also help show how I was
thinking about this).
Note regarding the TODO on how to form the token, `GetTokenText` returns
the `string_id`'s reference value for an `Identifier`. So to make
`GetTokenText` work in a way that returns `r#foo` for a raw identifier,
I think there are a few options:
1. Add additional data indicating the end of the identifier.
2. Add `RawIdentifier` as a token kind to indicate that it's raw and
should be prefixed with `r#` (but also giving later stages one more
token kind to handle)
3. Make the `string_id` correspond to `r#foo`, and have later stages add
`foo` to the strings table whenever `r#foo` is encountered (with map
lookups leading to deduplication).
4. Add `StringId::RawKeyword` special values for each keyword.
- This would mean `self` prints as `self`, `r#self` prints as `r#self`,
but `r#foo` is not a keyword so prints as `foo`.
- This means keywords would need to be listed in a place `StringId` can
depend on them, one way or the other (e.g., a `keywords.def` file in
`base/` should work).
5. Say that it _is_ an `Identifier`, and if it's a keyword spelling, it
must have been a raw identifier.
- Same limitation as above: This would mean `self` prints as `self`,
`r#self` prints as `r#self`, but `r#foo` is not a keyword so prints as
`foo`.
I'm hoping to resolve this issue separately though. :)
For now, we treat class types and `String` as non-copyable, because we
don't know how to emit SemIR to copy them yet. This will change as we
add support for copying those types when appropriate.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Adds a `BoundMethod` SemIR node to represent an `x.F` bound method, with
a new builtin type `BoundMethodType`. Reorganized conversion of call
expression arguments to also check and convert a `self` parameter in the
implicit parameters list.
In passing, improved diagnostics and error recovery for bad call
expressions. We now build a `call` node with the appropriate type and
value category, but with invalid arguments, if the argument conversion
failed, and diagnose calls to non-callable expressions.
`addr self` methods don't work properly yet; the `addr` is ignored for
now.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Provides an adapter for the llvm::yaml API because it otherwise needs a
bunch of const/non-const definitions, and the traits are difficult to
diagnose issues with. The current approach is pretty simple to use, even
if it's not super efficient (which, yaml output is more of a debugging
thing so I'm not really expecting it to be an issue).
Changes the format of yaml output to provide more index information,
just as reminders when seeing something like `node+0`. Note this would
create more churn in deltas if we were reliant on the output yaml in
tests, but we aren't so it should be okay.
This replaces the use of `VarStorage` in this case.
Add an `UnboundFieldType` type as the type of a field, in cases where
it's referenced without an accompanying object.
Add a `BindName` node to describe the name binding performed for both
variables and fields so that we can handle them more uniformly.
This standardizes on having ValueStore and related structures provide
printing, removing the handlers in file.cpp.
The `[]` is provided for empty sequences versus if there was simply
nothing, in which case it would be a sequence when non-empty, and a null
value when empty. Consistently (and explicitly) providing sequences
feels easier to understand.
The changes to the output yaml are overall more terse. My hope is that
this is an improvement for most readers.
Also fixes printing of APInt, defaulting to unsigned for consistency
with Carbon's use.
Building on #3311, change SemIR to use the SharedValueStore. Since this
removes hermeticity, raw output no longer prints ints, reals, and
strings. TokenizedBuffer accessors are modified to return IDs because
values are often passed through in semantics without needing to read
them.
I would've put SharedValueStores on Context, except for the
GetArrayBoundValue convenience method. I felt awkward removing that, so
it's on File, at least for now. That's then used by the formatter and
Lower too. The flipside of this is that TokenizedBuffer has a
SharedValueStores only for printing, so maybe that's similar enough to
what File is doing.
This doesn't start shifting other SemIR members to ValueStore, but that
seems like a next step.
This includes being able to define a class that was previously
forward-declared, and being able to define a member function out-of-line
that was previously declared inside a class.
No support for fields or methods yet, and a class definition doesn't yet
cause the class to be treated as a complete type.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Incomplete types may be nested within other types; for example, a tuple
type might have an incomplete type as an element. Handle such cases by
walking through nested incomplete types when completing a type. This is
done non-recursively in case a very complex type is formed.
Types are generally no longer completed at the point where they're
formed. Instead, we attempt to complete a type when it is used in a
context that requires a complete type, and diagnose if the type cannot
be completed at that point. This will be necessary for classes, which
can become complete after their first use, and helps tease out bugs
where a type completeness check is missing.
Using the computed value representation, fix lowering of struct and
tuple values to use the value representation rather than the object
representation. Fixes an issue found in the review of #3257.
This currently causes us to compute value representations of all types
as they are created, which generates substantially more SemIR to
represent types. We can get some of that back by deferring computation
of the value representation until the type is required to be complete,
but some of the additional cost here will persist with this approach.
I also considered making the computation of the value representation
type be something that lives entirely within the lowering phase, but I
think that's not the right approach in the longer term, because the
value representation will be semantically visible and relevant once we
start allowing it to be customized.
We should consider moving the nodes that exist to compute canonical
non-local types, including value representations, out into a separate
global block. That will clean up the SemIR representation substantially,
and make the SemIR produced for a function not depend on which types we
happen to have encountered beforehand. But that's not being done in this
PR.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Bug found by fuzzing. Problem was untyped SemIR nodes had an invalid
type id, which was retrieved by `HandlePrefixOperator` and then passed
to `context.GetUnqualifiedType`, ultimately performing an invalid access
in `semantics_ir_->GetNode`.
We prefer to make a placeholder type for functions and namespaces to
remove the need for checking for the untyped case everywhere. Eventually
functions will have their own types, but this approach will be needed
for namespaces (and perhaps other non-first-class entities like unbound
methods and interface members) long term.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Replace `SemIR::Node::GetAsFoo` and `SemIR::Node::Foo::Make` with
`SemIR::Foo` class that represents a particular kind of node, with named
fields.
Rename `SemIR::IntegerLiteral` and `SemIR::RealLiteral` to
`IntegerValue` / `RealValue` to better reflect their purpose and avoid a
name collision with the corresponding `SemIR` node kinds.
Remove `NodeKind::Invalid` and the `SemIR::Node` default constructor
entirely, as they were not used for anything.
Also add `name_reference_untyped` for references to non-first-class
names without types, which currently covers namespaces and functions.
This improves the fidelity of the SemIR representation, and fixes some
issues where we would use the wrong location for nodes and diagnostics
downstream of a name reference.
We're still missing a representation for dotted name expressions, such
as `Namespace.Function`, and we don't use the `untyped` node as an
operand of any other node yet.
Combine the initialization, implicit conversion, and value category
conversion functions into a single function.
This substantially reduces the duplication between these steps, and
ensures that we support the same set of conversions in all these
contexts. This also fixes some issues where we would not use the proper
value representation for tuples and structs after performing implicit
conversions.
Richard Smith
Jon Ross-PerkinsandRichard Smith
josh11b
Jacob Schneider