This adds a `BindExport` instruction in order to better track the
location of the `export` itself, but a `bind_name_id` is also added to
`ImportRef` so that we know quickly where to put it in name lookup.
Merging identical names is a TODO. I haven't quite decided how best to
achieve that, because I do think the BindExport should be what's
actually added to name lookup.
Also, I will probably add a mode to DeclNameStack that blocks
non-namespace scopes. This seems to already be an error, but the wrong
one (maybe due to lack of support for cross-file decl/def support).
This provides `export import` logic in lex, parse, and check; `export
name` logic is only in lex and parse, not check.
I think with `export name` I'm going to need to modify import_ref and
some consolidation logic, whereas `export import` seems feasible to keep
as primarily import logic. Given the implementations were looking like
they'd diverge more substantially, I thought it'd be helpful to cut the
PR here.
This was to track use of a declaration after import, prior to a
redeclaration. Per [discussion on
Discord](https://discord.com/channels/655572317891461132/1217182321933815820/1236016521059237962),
we likely don't need this check due to the change in behavior of
`extern`.
Rather than potentially getting one of many `extern` decls and depending
on it by accident, it is now planned to be _required_ to be imported,
and the library doing a non-`extern` decl must _know_ it's importing the
`extern` decl. The stricter requirement on the library means it now
seems more reasonable to use the `extern` decl.
So kind of rolling back #3831, though keeping `ImportIRInstId` (at least
for now) and keeping `Loaded`/`Unloaded` terminology (seems a nicer
fit).
This removes the builtin FunctionType, replacing it with a FunctionType
instruction. The constant for a FunctionDecl is now a StructValue with
type of FunctionType.
Note this means a function declaration produces _both_ a type, and a
value of the type. This has some consequences in terms of circularity,
and makes the importing of function declarations a little more complex.
It'll get particularly peculiar for imports because of the behavior of
the reference, but that's a known issue due to other things such as
`alias`. The impact will hopefully be contained to
ResolvePrevInstForMerge (and ImportRefs).
To note a small formatting change in diagnostics:
```
- // CHECK:STDERR: fail_member_lookup.carbon:[[@LINE+4]]:3: ERROR: Value of type `<associated <function> in Interface>` is not callable.
+ // CHECK:STDERR: fail_member_lookup.carbon:[[@LINE+4]]:3: ERROR: Value of type `<associated F in Interface>` is not callable.
- // CHECK:STDERR: fail_todo_facet_lookup.carbon:[[@LINE+4]]:3: ERROR: Value of type `<associated <function> in Interface>` is not callable.
+ // CHECK:STDERR: fail_todo_facet_lookup.carbon:[[@LINE+4]]:3: ERROR: Value of type `<associated F in Interface>` is not callable.
```
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
We don't need it any more, and removing it simplifies a few things:
- One fewer predefined `File` and reserved ID.
- We now have simply `Builtin` instructions for builtins, instead of
having an `ImportRef` that indirectly references a `Builtin`.
- `ConstantId`s now always refer directly to a local constant, instead
of sometimes referring to an `ImportRef` for a constant in the builtins
IR.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Factor out `SemIR::InstNamer` and also use it when lowering to LLVM IR.
Automatically name all instructions created with our `IRBuilder` based
on the name computed by the `InstNamer`, and likewise name basic blocks
using the label generated by the `InstNamer`.
Move some of the existing naming logic out from lower into `InstNamer`
so that it's also used in SemIR. In particular, we now name call
instructions after their callee, or after the builtin name for calls to
builtins.
Computing and adding these names isn't completely free. This instruction
naming is designed to be optional, so that we can turn it off for builds
where the LLVM IR will only be converted to assembly and won't be seen
by a human, but so far it's enabled unconditionally. We can tune that
later as needed.
While most of the examples in our tests are made clear by the code
snippet, sometimes that's not the case. This makes it more clear when
the implicit `Main` package is used for example. I added this because I
got an error message without any source code to show in the snippet
(importing the prelude for example) and this makes the error much more
understandable.
- Adds an empty prelude.carbon file
- Imports that file in any non-Core package file
- Adds --disable-prelude-import to avoid that
- Adds --exclude-dump-file-prefix to be able to hide files from dumping
- Used to hide core files (we can't do this by package name due to lex
dumps, for example)
- Restructures some tests to not rely on `i32`, particularly `alias`
tests (which rely on a name ref) and tests with no prelude.
I'm adding the framework for switching i32 to calling Int32 in the
prelude, but I'm running into a separate error actually switching over.
So that *mostly* works, but isn't quite ready for prime time. However,
maybe the current state of this PR is still useful to review since it
does a lot of the infrastructure work and adds the %Core everywhere?
Adds ImportIRId::ApiForImpl to reserve a specific slot for the `api`
import, so that the code can trivially determine whether an import is
from the same library. This is then used for merging function
declarations, because the rules for redeclarations in the same library
slightly differ as compared to other imports (note they're also not
identical to same-file rules).
The main thing this leaves from the recent #3762 is verifying that
entities forward declared in the `impl` file are also defined, but
that's not in-scope for merging; it's moreso post-checking validation.
Note, a lot of our `invalid <entity> ID` comments in ids.h were
incorrectly copy-pasted, so I've cut `<entity>`.
LookupNameInDecl is only called from DeclNameStack, but I'm adding
mark_imports_used there because it feels more consistent. Not sure if we
want a better API boundary. I admit I'm also suspicious of its call to
LookupInCurrentScope but maybe it's okay due to how imports work.
I was choosing to print multiple diagnostics when a declaration is
previously used _and_ doesn't match because I think the "previously
used" is more important, but the "doesn't match" may give an additional
hint about why it didn't work.
The merge.h utility function is because I think we can follow a similar
model for identifying errors with other declarations: classes,
interfaces, etc.
This doesn't significantly change logic, although I'm trying to add the
location to used state.
The issue I'm trying to address is how to identify a declaration as
"allowed to be redeclared". Consider:
```
library "a" api;
extern fn F();
```
```
library "b" api;
extern fn F();
```
```
library "c" api;
import library "a";
import library "b";
var x: auto = F();
fn F();
```
What currently happens is:
1. On import of "a", `F` becomes ImportRefUnused
2. On import of "b", `F` becomes ImportRefUsed in order to merge.
3. In "c", the call `F()` doesn't change the state.
4. In "c", the declaration `fn F();` needs some breadcrumb to understand
whether "F" has been referenced, as in step (3) here.
What I want to happen is:
1. On import of "a", `F` becomes ImportRefUnloaded
2. On import of "b", `F` becomes ImportRefLoaded in order to merge.
3. In "c", the call `F()` causes `F` to become ImportRefUsed
4. In "c", the declaration `fn F();` detects that `F` is already
ImportRefUsed, and can use the associated `used_id` for a diagnostic
about why redeclaring is invalid.
Note this PR isn't implementing (4). I'm focused on the refactoring to
add a new ImportRef state here.
Just connecting this together. Note the diagnostics still need to be
cleaned up/improved, although this is correctly handling some cases.
This exposed an incorrect parameter passed in DiagnosticEmitter, which
is fixed here -- context_fn shouldn't include args. Note this was caught
by the CHECK, I'm just expanding on the message since I used the detail
to help figure out the bug.
Note we only identify conflicts between libraries in the current package
during import.
This restructures the BUILD because of dependency cycles between cpp
files... We're going to need context's name lookup to handle things such
as merging, merging requires function logic, function logic requires
context access. Per discussion, going with a single large cc_library for
now rather than trying to split out small libraries.
I'm envisioning the new merge.* as a hub for cross-declaration merge
logic. Note function.cpp is already pretty sizable, and I think it may
lean a little function-specific even if there are some utilities that
could be split out.
This replaces all invalid node IDs in import_ref.cpp with references to
the imported instruction.
This splits out ReplaceInstBeforeConstantUse into a separate function
when the LocationId is replaced, as for splicing. That's the less common
case, whereas others would need to provide the LocationId in order just
to not change the value.
This starts propagating is_extern on import, and warns when merging an
imported non-extern declaration with a local non-extern declaration.
Note this doesn't address import conflicts yet (i.e., two libraries
define an equivalent name) because they don't call merge logic.
On merge, I'm only setting values when new_is_definition because I think
it better matches the comment and resulting behavior. Note I now set
them even for bad redefinitions; I think this matches the comment, and
there's not a perfect choice here. I could change the flow back if
preferred.
Note this puts a spotlight on invalid nodes on imported decls, which I
think I'm going to need to address now. This isn't addressed by
ImportRef logic directly because the Function's decl_id is a
FunctionDecl, rather than the ImportRef that led to it. While I could
add the ImportRef link to each decl, I think adjusting the associated
NodeId is a better approach.
As discussed around #3792, identify the import a diagnostic message came
from prior to the diagnostic message itself. This occurs during location
translation so that the logic can be central.
I'd considered associating the parse node with ImportRef instructions,
but I realized about halfway through that because I need to store the
ImportDirectiveId on the ImportIR for cross-package imports, it's there
for use in location translation without extra work. That saves a fair
amount of stringing it through declarations, as well as an oddity where
ImportRef instructions would have a node that didn't really represent
them.
The purpose of the newline is to make it clearer where a given
diagnostic begins and ends, particularly as the first message of a
diagnostic may not be the error.
This is a trivial code change, but ripples edits through test files.
Fix a collection of issues that were preventing lowering for overloaded
operators from working.
Instead of creating `import_ref` instructions during name lookup in the
current block, whatever that might be, we now create them in the `file`
block always. This avoids inserting them into blocks that might not be
intended to contain them, such as functions, and avoids the IR generated
for a function depending on which names we happen to have looked up
first.
When importing a class, function, or interface, import its enclosing
scope ID. This is necessary to allow us to distinguish between functions
at interface scope, which shouldn't be lowered, and other functions, and
will also be used in future to provide qualified names for declarations
when printing types. In order to support this:
- Track the constant values of namespaces created during importing so
that we can find them when resolving an import ref. Use those constant
values to convert an enclosing scope ID from the imported IR into a
corresponding ID in the current IR.
- Change how we do two-pass import of classes and namespaces so that we
can do two-pass import even for non-defining declarations, so that we
can import the enclosing scope.
While working on the final point above, I reworked `TryResolveInst` to
return a flag indicating whether another pass is necessary instead of
implicitly encoding this in the `ConstantId`. This permits the handling
of classes to be simplified; now `import_ir_constant_values` is only
accessed in a single place.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Note, I'm annotating the lookup partly so that the reason the conflict
comes up is clear, partly so that there's actually a diagnostic line
associated with the root cause as more tests get packed into a single
file.
This revamps the support for cross-package imports, making them look
more like a namespace. The planned model is mentioned on
[#toolchain](https://discord.com/channels/655572317891461132/655578254970716160/1217586076022210670).
This does not implement name lookup into the new namespace structure.
A few key changes in this PR (it's a little sprawling) are:
- Moves logic for adding package imports from context.* to import.*
- Remove SemIR::Import, which was the prior model. This is instead now a
SemIR::Namespace with the NameScope getting a new import_ir_scopes
field.
- Allow SemIR::Namespace to use Parse::ImportDirectiveId in addition to
the prior Parse::NamespaceId
- The import_ir_scopes field includes a NameScopeId so that as we
traverse to child namespaces, we can directly perform name lookup in the
other IR.
- is_closed_import now tracks whether a namespace comes from a different
package. This has a diagnostic implemented in decl_name_stack.
This handles toolchain failures per-file. The intent is to allow placing
both "success" and "fail" tests in the same file, using splits. However,
this PR only adds support and updates existing tests to continue
passing.
Previously we used the default inst block formatting, which writes out a
parenthesized list of references, which would always be 'unexpected
instref's because nothing else prints the instructions in the decl
block.
In addition, track the decl block for function declarations like we do
for other kinds of declaration. This means that the parameter
declarations for a function are now properly rendered into the formatted
IR. Note that this adds a lot of verbosity to `function_decl`, but it
does accurately reflect the IR, and we'll probably want this information
to be printed once we start supporting more complex generic function
declarations.
This removes almost all the 'unexpected instref's in our formatted
output. There are remaining cases when a declarative scope contains
multiple blocks, where we only track one of those blocks. That happens
when there is control flow within declarative scopes, and for error
recovery when a class or interface or similar is defined more than once.
As requested in #3730.
I believe this PR is sufficient to pull in all current class features,
including the current bits of inheritance which have been implemented.
Because a class declaration can reference its own type, this creates an
incomplete type prior to constant loading.
Right now, the object representation is imported proactively, but
individual fields are left as ImportRefUnused. This means that member
functions and similar will only be imported if called.
This also adjusts how function parameters are being handled, to match
the expectations of Self param structure.
When formatting, I'm starting to look into constants. Otherwise we get
"unexpected instref".
Overall, there are a few things that may be worth further discussion:
- The lack of a constant corresponding to the ClassType on ClassDecl is
inconvenient -- I'd like to see how zygoloid feels about trying to
restructure this. i.e., I'm setting a constant in order to be able to
track things down later, it'd be nice if the normal IR did this simply
for consistency, or if we were able to combine these rather than having
separate instructions.
- Should we shift the parse node tracking further, and go with a setup
wherein imports can embed import references into that? e.g., negative
values go to another array which includes a ImportIRId for printing
diagnostics, replacing the invalid NodeId.
- Can the formatter switch to a more general scan of instructions for
naming, to eliminate the ImportRef constant approach added here?
- GetExprValueForLookupResult special-casing instructions felt
surprising, I might see if there's a way to restructure to avoid that.
But I think these issues are things that can be separated out.
This works by creating a faux FunctionDecl in the context of the current
IR, which seems to be working for function calls. Deduced params are
there, but won't really be tested until classes are up and running. Also
I may need to look further at return_slot_id to ensure it's working. But
the basics, I think, are here.
Reorganizes some other ImportRef work from `has_unresolved` that'd
relied on manual calls to a more detection-based `HasUnresolved`
approach that doesn't require as much checking.
`GlobalInit` block is now static block within a `SemIR` which will be
used to emit initialization instructions for variables in the `Package`
scope.
inst_block_stack now has additional methods to handle `GlobalInit` block
separately, this block can be popped without being finalized allowing to
accumulate between all instances of variables.
At the end of the `check` phase, if this block is not empty , the
function `__global_init` will be added with this block being inserted
into it.
This block is pushed to `inst_block_scope` at the end `BindName`,
allowing instruction to be emitted into it, then popped at the semicolon
(VariableDecl).
This significantly changes the `SemIR` output, that's why this commit
updates a lot of the test cases.
This provides support for Const, Pointer, Struct, and Tuple types. It
does not cover Class, Function, or Interface which have their own Id and
are tracked slightly differently.
I'm basically just nudging down the path I think is right here. Adding a
small bit more support, but also more tests to capture cases that I
think will need to be verified as working.
With diagnostics like "Value of type `<function>` is not callable.",
that's because it expects a FunctionDecl but is instead finding a
ImportRefUsed. I'll need to work out the necessary support for a
callable function.
Builds on #3656.
Under the prior "lazy" model, we had been planning to copy instructions.
Under the current "unused+used" model, we're restricting that to
constants. I'm trimming back some of the ResolveIfImportRefUnused logic
because it was more appropriate for the former model.
Right now, I'm adding ImportRefUsed direct creation in import.cpp for
namespaces. I think I'll need to do something similar in
RseolveIfImportRefUnused... I'm still trying to think about how to
manage type information there (which needs to come in as an import
reference itself, and probably have some amount of deduplication before
forming a TypeId). So ImportRefUnused lacks a type because I'm hesitant
to aggressively load it, whereas ImportRefUsed should *always* have a
type but it's just an error while I think things through.
For reference, ImportRefUnused and ImportRefUsed are mainly split in
order to track the boolean "used" without making fundamental
modifications to Inst for bit packing (this effectively instead packs a
bit into InstKind). AnyImportRef currently excludes the type because
it's mainly for diagnostic printing at the moment. It could end up with
a TypeId that would end up Invalid for ImportRefUnused, though it could
also be that the TypeId is only accessed when using ImportRefUsed
explicitly.
This makes some changes to the formatter so that ImportRefUnused and
ImportRefUsed will both be labeled as "import_ref" with an "unused ->
used" argument change in textual IR, but is otherwise not changing
logic.
Building on #3636 which handles the general import case, add special
casing for namespaces. Namespaces can be combined cross-IR, so it's a
little more complex.
The implementation adds import_id to the Namespace instruction as a
reference to find the original using the normal structure. This is
achieved by moving the name_id to NameScope to free up space.
---
I considered a few alternatives...
I considered adding import_id to NameScope, but:
1. It's more consistent with things such as Function or Class that
provide name_id on the info object rather than the instruction.
2. I thought it more likely that there would be more NameScope cases
that might want a name_id rather than the import_id, since ImportRef
will typically be used.
I considered putting the import source (cross-ref IR id + inst id) on
the NameScope versus a separate ImportRef, which seems like the
strongest argument towards the NameScope approach because it removes an
instruction. That just felt inconsistent though, and the overhead of
instruction-per-imported-namespace should be low (theoretically few
namespaces should be used). Plus I feel a bit odd adding two
generally-unused ids to NameScope.
A specialized Namespace structure could also have been created to store
the import_id, but that would add an indirection to the NameScope.
Name conflicts weren't previously tested, and the diagnostics were just
a TODO, so this is also adding testing for that. But handling too.
Removing AddEntry because I think it's hard to make helpful for this
use-case when we want to do a diagnostic followup (because really,
callers want the full `.insert` result of pointer + success), and unused
otherwise.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Remove the type canonicalization mechanism and instead rely on constant
canonicalization to deduplicate types.
Rename the `Canonicalize*Type` functions to reflect that they're no
longer performing canonicalization. Switch code that creates types due
to semantic checking, rather than due to source syntax, to directly
create type constants through evaluation rather than creating an
instruction and evaluating it to produce a separate constant
representation.
The mapping from `const (const T)` that was previously performed by type
canonicalization is now implemented in expression evaluation instead.
The value `<error>` is now treated as a constant value, with a special
property that an instruction involving `<error>` that could possibly be
constant evaluates to `<error>`. This helps avoid producing follow-on
errors when an error occurs as a subexpression of an expression, such as
a type, that is intended to be constant.
This is accomplished by tracking an extra bit on the ID we store in the
constant values table, and propagating that from subexpressions to the
enclosing expression. This extra bit is not yet computed correctly for
types; that will be addressed in later PRs.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Form a side table with constant values for each instruction. Evaluation
is only supported for a few very simple kinds of instruction for now.
This is not observable outside of the SemIR output, because nothing
depends on expressions having a constant value phase yet.
Namespaces are copied, which means also adding their name to the
underlying instruction. It happened not to be done previously; the name
was only in name lookup.
Since the only import supported right now is the default import,
functionality is limited; in the future I'll need to deal with namespace
vs package conflicts.
Tests of namespace imports are under "namespace" -- I figured this would
be best for scaling as more instructions get support.
This also improves some debugging-related output that I was trying to
use while trying to build the support.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This builds out a little infrastructure for one name scope to `extend`
another. We'll need more refinement here to cover other cases, but this
should provide some foundation for that future work.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Adds a `LazyImportRef` instruction. Versus `CrossRef`, this is intended
to represent an instruction which cannot be used directly, and must be
replaced when it comes up due to name lookup. The intent is to use this
to avoid recursive loading of imported IR instructions.
Note, under this model, when `ResolveIfLazyImportRef` is called, it
essentially needs to load both inst and type information to a sufficient
point where any further attempts would hit name lookup again. That will
probably be complex, and the current implementation is just touching the
surface of the issue. I was heading down this route because it would
mean we have a limited number of points that need to consider whether
they're going to talk about a `LazyImportRef`.
I'm considering whether `CrossRef` should be dropped in favor of more
specific `Builtin` special-casing, due to the divergence of desired
behaviors. This could mean dropping the `builtins` IR since it's not
looking useful right now.
Modify `NameScope` to track whether the scope is associated with a load
error. This is to handle cases where one or more imports failed, so we
do not want to issue warnings for related scopes.
The 0-size on `ValueStore` comes up due to the changes to `NameScope`,
which make it too large for the default handling. After discussion with
zygoloid, the thought was we might want to try reserving a roughly
correct value based on parse node counts, but the stack default wasn't
buying much.
Fixes a bug where the implicit import used the package name instead of
the invalid identifier.
This creates a namespace for `package` scope.
It looks like names of class_decls in namespaces lead to an unexpected
instref. This is already true, as best as I can tell. I'm not sure if
there's a preferred approach to address that, so I've left a TODO for
now.
This removes the filename from the file-scoped block, and places it
above to make it clear where the full SemIR begins (with multifile,
providing a barrier between).
This adds instructions so that we get printing. I may adjust the
instruction format a little further to add a type, but I think the basic
setup will remain.
Note this builds on #3414
Jon Ross-Perkins
Chandler Carruth
Gıyaseddin Tanrıkulu