As discussed around #3792, identify the import a diagnostic message came
from prior to the diagnostic message itself. This occurs during location
translation so that the logic can be central.
I'd considered associating the parse node with ImportRef instructions,
but I realized about halfway through that because I need to store the
ImportDirectiveId on the ImportIR for cross-package imports, it's there
for use in location translation without extra work. That saves a fair
amount of stringing it through declarations, as well as an oddity where
ImportRef instructions would have a node that didn't really represent
them.
The purpose of the newline is to make it clearer where a given
diagnostic begins and ends, particularly as the first message of a
diagnostic may not be the error.
This is a trivial code change, but ripples edits through test files.
This handles toolchain failures per-file. The intent is to allow placing
both "success" and "fail" tests in the same file, using splits. However,
this PR only adds support and updates existing tests to continue
passing.
This doesn't add full support. I'm separating it out to make the effects
of the modifier changes clearer for review. I'm restructuring a little
with the expectation that we'll have some more categories of modifier
keywords in the future (similar to `extern`, these may not be in a "set"
such as access), and thus easily scaling up to a few more would be
useful.
This detects more cases of incorrect matches between declaration and
definition. It also factors the logic out to a separate file for easier
sharing, particularly when it comes to merging imports (I'm not sure if
this exact API will be reshared, but the core logic should apply).
Previously we used the default inst block formatting, which writes out a
parenthesized list of references, which would always be 'unexpected
instref's because nothing else prints the instructions in the decl
block.
In addition, track the decl block for function declarations like we do
for other kinds of declaration. This means that the parameter
declarations for a function are now properly rendered into the formatted
IR. Note that this adds a lot of verbosity to `function_decl`, but it
does accurately reflect the IR, and we'll probably want this information
to be printed once we start supporting more complex generic function
declarations.
This removes almost all the 'unexpected instref's in our formatted
output. There are remaining cases when a declarative scope contains
multiple blocks, where we only track one of those blocks. That happens
when there is control flow within declarative scopes, and for error
recovery when a class or interface or similar is defined more than once.
As requested in #3730.
I believe this PR is sufficient to pull in all current class features,
including the current bits of inheritance which have been implemented.
Because a class declaration can reference its own type, this creates an
incomplete type prior to constant loading.
Right now, the object representation is imported proactively, but
individual fields are left as ImportRefUnused. This means that member
functions and similar will only be imported if called.
This also adjusts how function parameters are being handled, to match
the expectations of Self param structure.
When formatting, I'm starting to look into constants. Otherwise we get
"unexpected instref".
Overall, there are a few things that may be worth further discussion:
- The lack of a constant corresponding to the ClassType on ClassDecl is
inconvenient -- I'd like to see how zygoloid feels about trying to
restructure this. i.e., I'm setting a constant in order to be able to
track things down later, it'd be nice if the normal IR did this simply
for consistency, or if we were able to combine these rather than having
separate instructions.
- Should we shift the parse node tracking further, and go with a setup
wherein imports can embed import references into that? e.g., negative
values go to another array which includes a ImportIRId for printing
diagnostics, replacing the invalid NodeId.
- Can the formatter switch to a more general scan of instructions for
naming, to eliminate the ImportRef constant approach added here?
- GetExprValueForLookupResult special-casing instructions felt
surprising, I might see if there's a way to restructure to avoid that.
But I think these issues are things that can be separated out.
This works by creating a faux FunctionDecl in the context of the current
IR, which seems to be working for function calls. Deduced params are
there, but won't really be tested until classes are up and running. Also
I may need to look further at return_slot_id to ensure it's working. But
the basics, I think, are here.
Reorganizes some other ImportRef work from `has_unresolved` that'd
relied on manual calls to a more detection-based `HasUnresolved`
approach that doesn't require as much checking.
Adds `BindAlias` with a hybrid of `BindName` and `NameRef` semantics. I
think it's slightly closer to `BindName` because it introduces a name,
so I'm going more in that direction. This also matches the need for
`bind_name_id` with imports on enclosing scopes.
Note, only things that look like a name reference are being allowed on
the RHS of `alias`. This includes builtins that look like name
references, such as `bool`, but not ones that turn into values
underneath, such as `false`.
`GlobalInit` block is now static block within a `SemIR` which will be
used to emit initialization instructions for variables in the `Package`
scope.
inst_block_stack now has additional methods to handle `GlobalInit` block
separately, this block can be popped without being finalized allowing to
accumulate between all instances of variables.
At the end of the `check` phase, if this block is not empty , the
function `__global_init` will be added with this block being inserted
into it.
This block is pushed to `inst_block_scope` at the end `BindName`,
allowing instruction to be emitted into it, then popped at the semicolon
(VariableDecl).
This significantly changes the `SemIR` output, that's why this commit
updates a lot of the test cases.
I'm basically just nudging down the path I think is right here. Adding a
small bit more support, but also more tests to capture cases that I
think will need to be verified as working.
With diagnostics like "Value of type `<function>` is not callable.",
that's because it expects a FunctionDecl but is instead finding a
ImportRefUsed. I'll need to work out the necessary support for a
callable function.
Builds on #3656.
Under the prior "lazy" model, we had been planning to copy instructions.
Under the current "unused+used" model, we're restricting that to
constants. I'm trimming back some of the ResolveIfImportRefUnused logic
because it was more appropriate for the former model.
Right now, I'm adding ImportRefUsed direct creation in import.cpp for
namespaces. I think I'll need to do something similar in
RseolveIfImportRefUnused... I'm still trying to think about how to
manage type information there (which needs to come in as an import
reference itself, and probably have some amount of deduplication before
forming a TypeId). So ImportRefUnused lacks a type because I'm hesitant
to aggressively load it, whereas ImportRefUsed should *always* have a
type but it's just an error while I think things through.
For reference, ImportRefUnused and ImportRefUsed are mainly split in
order to track the boolean "used" without making fundamental
modifications to Inst for bit packing (this effectively instead packs a
bit into InstKind). AnyImportRef currently excludes the type because
it's mainly for diagnostic printing at the moment. It could end up with
a TypeId that would end up Invalid for ImportRefUnused, though it could
also be that the TypeId is only accessed when using ImportRefUsed
explicitly.
This makes some changes to the formatter so that ImportRefUnused and
ImportRefUsed will both be labeled as "import_ref" with an "unused ->
used" argument change in textual IR, but is otherwise not changing
logic.
Per discussion with zygoloid, namespace declarations will merge both
when repeated in a given file, and across files. This echoes how forward
declarations of other entities are allowed to repeat. When merging with
an imported namespace, fill in the parse node so that future diagnostics
point at the declaration in the same file rather than a declaration in a
different file, just for locality.
Although it might be desirable to issue a diagnostic when a namespace
declaration is repeated within a given file, that's similarly true in
other cases, but may be more desirable as a tidy-style issue rather than
preventing code from compiling. Allowing the repetition also makes the
import versus non-import cases more consistent: the namespace
declarations merge regardless of the source.
Building on #3636 which handles the general import case, add special
casing for namespaces. Namespaces can be combined cross-IR, so it's a
little more complex.
The implementation adds import_id to the Namespace instruction as a
reference to find the original using the normal structure. This is
achieved by moving the name_id to NameScope to free up space.
---
I considered a few alternatives...
I considered adding import_id to NameScope, but:
1. It's more consistent with things such as Function or Class that
provide name_id on the info object rather than the instruction.
2. I thought it more likely that there would be more NameScope cases
that might want a name_id rather than the import_id, since ImportRef
will typically be used.
I considered putting the import source (cross-ref IR id + inst id) on
the NameScope versus a separate ImportRef, which seems like the
strongest argument towards the NameScope approach because it removes an
instruction. That just felt inconsistent though, and the overhead of
instruction-per-imported-namespace should be low (theoretically few
namespaces should be used). Plus I feel a bit odd adding two
generally-unused ids to NameScope.
A specialized Namespace structure could also have been created to store
the import_id, but that would add an indirection to the NameScope.
This makes duplicate and previous definition handling match. While we
may want to make both point more fine-grained at the name, the necessary
logic seems likely to be equivalent.
Note, I'm looking at this mainly due to duplicate names in imports,
where it's especially helpful to take an instruction instead of a parse
node. We'll eventually want to handle parse nodes from other imports
better, and I think this is the way it would most likely work.
Rather than producing multiple constants with the same value, fold all
instances of a given constant to the same constant instruction.
A future PR will use this to replace the current type canonicalization
system.
This is accomplished by tracking an extra bit on the ID we store in the
constant values table, and propagating that from subexpressions to the
enclosing expression. This extra bit is not yet computed correctly for
types; that will be addressed in later PRs.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Form a side table with constant values for each instruction. Evaluation
is only supported for a few very simple kinds of instruction for now.
This is not observable outside of the SemIR output, because nothing
depends on expressions having a constant value phase yet.
Namespaces are copied, which means also adding their name to the
underlying instruction. It happened not to be done previously; the name
was only in name lookup.
Since the only import supported right now is the default import,
functionality is limited; in the future I'll need to deal with namespace
vs package conflicts.
Tests of namespace imports are under "namespace" -- I figured this would
be best for scaling as more instructions get support.
This also improves some debugging-related output that I was trying to
use while trying to build the support.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Another incremental change to diagnostic formatting. I simply recurse
over all the tokens in the subtree of a parse node and construct a
`DiagnosticLocation` that covers all of the tokens.
I believe it's nicer for the user to be directed at the entire chunk of
source where the error is occurring rather then just pointing at the
bracketing/terminator tokens, but let me know if you all agree.
In theory because none are allowed. This is to improve consistency in
handle_decl_name_scope's modifier handling, removing the namespace
special-case.
I noticed there's a crash bug on `impl <declaration>` which I'll address
separately.
This builds on #3461.
This creates a namespace for `package` scope.
It looks like names of class_decls in namespaces lead to an unexpected
instref. This is already true, as best as I can tell. I'm not sure if
there's a preferred approach to address that, so I've left a TODO for
now.
This removes the filename from the file-scoped block, and places it
above to make it clear where the full SemIR begins (with multifile,
providing a barrier between).
This is an incremental improvement on our diagnostic messages that
simply underlines an entire token if the token is larger than 1 char
(else it points to the single char with a caret like it used to).
Add a `NameId` that is effectively just a wrapper around a `StringId`,
with
some additional predefined values for names that don't correspond to
strings, such as the name of `self` or the function's return slot.
When declaring a name such as `fn Ns.Class.F() { ... }`, enter the
scopes of `Ns` and `Ns.Class` as we form the name, and remain in those
non-lexical scopes until the end of the declaration.
When performing an unqualified lookup, look in any enclosing non-lexical
scopes in addition to looking into the lexical name table.
We now track a scope index with each lookup result in the lexical name
lookup table. This is used to determine whether a lexical or non-lexcial
result is the innermost result and whether a declared name is in the
same scope as some previous introduction of that name or in a nested
scope. For now, this could just be the index into the scope_stack, but
the intent is to also use this to detect names being declared after they
are first looked up, which requires the indexes to outlive their scopes,
so we use a persistent numbering of all scopes instead. The persistent
numbering also permits more invariant checking.
Retain the `ClassDeclaration` node to represent a syntactic declaration
of a class (including possibly a declaration of a generic class), but
use a separate SemIR node to represent the class type itself. This
allows us to give the two separate treatment.
The `ClassDeclaration` is still entered into the name lookup table for
its enclosing scope, but when it is named in an expression, the class
type is produced instead. When the class declaration is named in a
declaration name, it can be used to define members of the class, but an
expression that resolves to the class type cannot be used to define
members of the class.
In order to distinguish these cases, use `Name` rather than
`NameExpression` for the left-hand side of a `QualifiedName` parse node.
This removes the only use of the `Expression` form of a declaration
name, so that is also removed.
In the future, `ClassType` will also be used to describe types such as
`Vector(T)`, for which there is no corresponding `ClassDeclaration`.
This includes being able to define a class that was previously
forward-declared, and being able to define a member function out-of-line
that was previously declared inside a class.
No support for fields or methods yet, and a class definition doesn't yet
cause the class to be treated as a complete type.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Track the callee expression in full, instead of only tracking the
callee's FunctionId. This results in the `name_reference` denoting the
function actually being used.
Lowering now propagates a `llvm::Function*` as the value associated with
expressions of type `<function>`.
We were not creating `NameReference` node for names produced by member
access into a namespace, such as the second name in
`Namespace.Function`, which caused lowering of calls to such names to
fail. This is now fixed, but the resulting `NameReference` node only
refers to the name and the lookup result, not to the `Namespace.`
qualifier. We'll need to decide how to fit a third operand into that
node (perhaps we can stop storing the `name_id`, since it can be derived
from the lookup result) but for now the qualifier is not tracked.
Bug found by fuzzing. Problem was untyped SemIR nodes had an invalid
type id, which was retrieved by `HandlePrefixOperator` and then passed
to `context.GetUnqualifiedType`, ultimately performing an invalid access
in `semantics_ir_->GetNode`.
We prefer to make a placeholder type for functions and namespaces to
remove the need for checking for the untyped case everywhere. Eventually
functions will have their own types, but this approach will be needed
for namespaces (and perhaps other non-first-class entities like unbound
methods and interface members) long term.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Also add `name_reference_untyped` for references to non-first-class
names without types, which currently covers namespaces and functions.
This improves the fidelity of the SemIR representation, and fixes some
issues where we would use the wrong location for nodes and diagnostics
downstream of a name reference.
We're still missing a representation for dotted name expressions, such
as `Namespace.Function`, and we don't use the `untyped` node as an
operand of any other node yet.
This makes the difference between errors and lower-level diagnostics
visible to users, and aligns the toolchain's behavior with the
expectations in `driver_fuzzer.cpp`.
Building on #3214 and #3215, updates sem_ir yaml to be:
```
- filename: name
sem_ir: [ ... ]
```
Also, changes the textual format from `package { ... }` to `file
<filename> { ... }`. My thought on packages there is:
```
file "foo.carbon" {
package MyPackage
...
}
```
The reason for putting the file first is that it's easier if we put what
we're grouping on first, whereas the package is an "annotation" on the
file.
Continuing along with #3070. Note this is just a file rename, with BUILD
edits; every file previously in semantics/ should show as moved (except
maybe BUILDs, which split).
Jon Ross-Perkins
Richard Smith
Gıyaseddin Tanrıkulu
josh11b
Jacob Schneider
Geoff Romer