Instead of attempting to reconstruct this based on the properties of the
target scope of the declaration name, check whether the enclosing scope
of the inner deferred definition scope is the enclosing deferred
definition scope.
Also add verbose logging to the deferred definition handling code to
make it easier to see when we switch between handling the regular parse
nodes and the deferred function bodies.
This is achieving a similar goal as #3849, using placeholders instead of
an ambiguous start node to clarify structure and incrementally simplify
checking. The benefit isn't quite as big here because both paths are
structs, and so checking is more consistent than paren exprs versus
tuples. But I think this removes the only other multi-purpose parse
node.
This uses StructLiteral/StructTypeLiteral naming, reflecting equivalent
SemIR naming. Note, I would lean towards renaming StructLiteral to
StructValueLiteral, but I think consistency in naming takes precedence.
Any renames of StructLiteral might be better in a separate PR.
StructFieldType/StructFieldValue -> StructTypeField/StructField is
trying to making the reading more consistent with
StructTypeLiteral/StructLiteral. SemIR has StructTypeField but not a
value equivalent.
I've been thinking about this since we decided to add placeholders in
the parse tree. This allows a clearer division of work in check
handling, where we were doing work for ExprOpenParen that's only
necessary for tuples (splitting/renaming handle_paren.cpp accordingly).
LookupNameInDecl is only called from DeclNameStack, but I'm adding
mark_imports_used there because it feels more consistent. Not sure if we
want a better API boundary. I admit I'm also suspicious of its call to
LookupInCurrentScope but maybe it's okay due to how imports work.
I was choosing to print multiple diagnostics when a declaration is
previously used _and_ doesn't match because I think the "previously
used" is more important, but the "doesn't match" may give an additional
hint about why it didn't work.
The merge.h utility function is because I think we can follow a similar
model for identifying errors with other declarations: classes,
interfaces, etc.
This doesn't significantly change logic, although I'm trying to add the
location to used state.
The issue I'm trying to address is how to identify a declaration as
"allowed to be redeclared". Consider:
```
library "a" api;
extern fn F();
```
```
library "b" api;
extern fn F();
```
```
library "c" api;
import library "a";
import library "b";
var x: auto = F();
fn F();
```
What currently happens is:
1. On import of "a", `F` becomes ImportRefUnused
2. On import of "b", `F` becomes ImportRefUsed in order to merge.
3. In "c", the call `F()` doesn't change the state.
4. In "c", the declaration `fn F();` needs some breadcrumb to understand
whether "F" has been referenced, as in step (3) here.
What I want to happen is:
1. On import of "a", `F` becomes ImportRefUnloaded
2. On import of "b", `F` becomes ImportRefLoaded in order to merge.
3. In "c", the call `F()` causes `F` to become ImportRefUsed
4. In "c", the declaration `fn F();` detects that `F` is already
ImportRefUsed, and can use the associated `used_id` for a diagnostic
about why redeclaring is invalid.
Note this PR isn't implementing (4). I'm focused on the refactoring to
add a new ImportRef state here.
Just connecting this together. Note the diagnostics still need to be
cleaned up/improved, although this is correctly handling some cases.
This exposed an incorrect parameter passed in DiagnosticEmitter, which
is fixed here -- context_fn shouldn't include args. Note this was caught
by the CHECK, I'm just expanding on the message since I used the detail
to help figure out the bug.
These can't be supported by `TryEvalInst`, because we don't track
sufficient information about predecessors and branch conditions in SemIR
to efficiently compute the constant result. Even if we could do so, we
may not want to treat all `BlockArg`s for which we can infer a constant
value as being constants. Instead, set the constant value explicitly
after creating the corresponding `BlockArg` instruction.
In parse, form a list of methods that are defined inline, tracking where
they start, where they end, and which other inline methods are nested
within them.
In check, when we reach an inline method body, skip it and add it to a
worklist to be processed later. We also track when we reach the start
and end of a context in which inline method bodies are deferred, so that
we know when to replay the bodies.
When suspending a function definition to be processed later, the
`DeclNameStack` entry is moved to separate storage, including popping
the corresponding scopes from the scope stack and removing the
corresponding lexical names from lexical lookup. Later, when we return
to the function and parse its definition, the `DeclNameStack` entry is
restored. The same is done when we reach the end of a nested context
that can have inline methods, so that we can reenter the nested scope
before processing its members.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Our testing had started to take more noticable time so I looked at where
the time went to see if it could be improved easily. Almost all of the
time was spent building regex matchers. That code path allocates a lot
of memory and does a lot of processing in order to make the regex fast
to run over the input. While that's not really a great tradeoff for how
we use these matchers, the bigger issue is that we don't need a regex is
the vast majority of cases.
This change inspects the pattern more deeply to avoid most of the work.
First, it looks for the
cases where the pattern doesn't require any adjustment at all and
directly builds a matcher from that if it can. This avoids an extra
string allocation entirely. This is probably only a minor improvement,
but it is also very easy.
The larger change is to process the string in two phases. First, we
expand the keywords and check for a regex region. If we find no regex
region, we can directly build a string equality matcher for the expanded
string. This still allocates an extra copy of the string but is
*dramatically* cheaper that building a regex.
Finally, if we *do* find a regex, we re-process the string to escape
everything and transform the regex sequence into its valid form before
building the matcher.
The result for me is an over 5x reduction in test time. Profiling
afterward shows a lot more opportunities for optimization here if things
get slow again. The actual toolchain code isn't really visible in the
profile yet.
Note, I was profiling the normal build, which has asserts and ASan and
such. I've not looked at the optimized build.
Note I don't think this affects behavior just due to how the
implementation is, but I was trying to figure out a bug that was hit
during sorting consumer flushing, and this seems like it should be done
for consistency.
Note we only identify conflicts between libraries in the current package
during import.
This restructures the BUILD because of dependency cycles between cpp
files... We're going to need context's name lookup to handle things such
as merging, merging requires function logic, function logic requires
context access. Per discussion, going with a single large cc_library for
now rather than trying to split out small libraries.
I'm envisioning the new merge.* as a hub for cross-declaration merge
logic. Note function.cpp is already pretty sizable, and I think it may
lean a little function-specific even if there are some utilities that
could be split out.
This replaces all invalid node IDs in import_ref.cpp with references to
the imported instruction.
This splits out ReplaceInstBeforeConstantUse into a separate function
when the LocationId is replaced, as for splicing. That's the less common
case, whereas others would need to provide the LocationId in order just
to not change the value.
The purpose of this change is to allow something such as a FunctionDecl
instruction to note an imported instruction as the "loc_id". Note that
doesn't occur here: this change is already very sweeping in edits. There
is no testdata affected, intended to show equivalent behavior.
We might want to consolidate NodeId references towards LocationId, but
if that's preferred, I'd still like to split it out. A lot of this just
piping through LocationId where it's a build error otherwise, enough
that imports should be able to start using it for diagnostics.
ValueStores are added but still unused -- just flushing out structure
for review.
Restructuring SemIRLocation is necessary to use LocationId this way. For
TokenOnly, it's not getting used in Parse, so I migrated it to Check and
it's now specific to SemIRLocation.
I also considered making LocationId reference an InstId (which would
need to be an ImportRef) instead of an ImportIRInstId. However, that
would've required import.cpp to add instructions for decls which are
reached during resolution -- we typically don't have an inst ready for
use. An extra inst is essentially 16 bytes in InstId's ValueStore + 4
bytes in LocationId's ValueStore, whereas this is 8 bytes per.
Per #3714, some of the details here are not yet settled. In particular,
we might want `Self` to come into scope at the start of the definition,
not at the `as` keyword. However, this change allows us to accept the
uncontroversial examples.
This starts propagating is_extern on import, and warns when merging an
imported non-extern declaration with a local non-extern declaration.
Note this doesn't address import conflicts yet (i.e., two libraries
define an equivalent name) because they don't call merge logic.
On merge, I'm only setting values when new_is_definition because I think
it better matches the comment and resulting behavior. Note I now set
them even for bad redefinitions; I think this matches the comment, and
there's not a perfect choice here. I could change the flow back if
preferred.
Note this puts a spotlight on invalid nodes on imported decls, which I
think I'm going to need to address now. This isn't addressed by
ImportRef logic directly because the Function's decl_id is a
FunctionDecl, rather than the ImportRef that led to it. While I could
add the ImportRef link to each decl, I think adjusting the associated
NodeId is a better approach.
As discussed around #3792, identify the import a diagnostic message came
from prior to the diagnostic message itself. This occurs during location
translation so that the logic can be central.
I'd considered associating the parse node with ImportRef instructions,
but I realized about halfway through that because I need to store the
ImportDirectiveId on the ImportIR for cross-package imports, it's there
for use in location translation without extra work. That saves a fair
amount of stringing it through declarations, as well as an oddity where
ImportRef instructions would have a node that didn't really represent
them.
The stream needs to be declared before the scoped exit (for flush in
particular). Trying to also adjust the code to make the issue clearer.
Caught by asan.
Rename `BoolValue::FromBool` to `BoolValue::From` as requested in #3816.
Add `BoolValue::ToBool`.
Convert existing code to use these where appropriate.
In preparation for adding more builtins, factor out the handling of
builtin function kinds into separate files.
Add checking for builtin function signatures. The mechanism used here is
intended to provide a lot of flexibility for declaring generic builtin
functions and pretty arbitrary constraints on the types of parameters of
builtin functions. For now, these constraints are checked when the
builtin function is declared. The hope is that this will suffice, but if
not, it should be straightforward to switch to doing some of the
checking on call and share logic between the checks.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
This is temporary: eventually per the design we should be forming
integer literals whose types reflect their values. But for now we should
ensure that values fit within their types.
This also fixes canonicalization of integer constants and hence of array
types, because we no longer have multiple different representations of
each `i32` value depending on the bit-width used for the literal.
Update broken links to aid in following links while perusing the
documentation. Closes#3778
I don't imagine I got every one of these right. Some 404s may be
reported due to permissions.
Using the lychee command in #3778, only the following link is reported
as not found, but it's just a permission issue:
```
[proposals/p1367.md]:
✗ [404] https://github.com/carbon-language/carbon-lang/settings/access | Failed: Network error: Not Found
```
I tried to get it down to as few as possible, partially so that future
link scans won't run into repeat errors.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
The purpose of the newline is to make it clearer where a given
diagnostic begins and ends, particularly as the first message of a
diagnostic may not be the error.
This is a trivial code change, but ripples edits through test files.
On #3792 it was requested to put context-related diagnostic messages
prior to the diagnostic error. In order to support that, remove the
distinction that an error needs to come first on DiagnosticMessage.
Since the addition of TranslateArg, I don't think this type is going to
go away (cutting a TODO). Refactoring names slightly to fit the current
role, and adding const to ConvertLocation.
- Add the `extern` keyword for forward declarations in libraries that
don't
provide the definition.
- Treat repeated forward declarations as redundant.
- Allow them when they prevent a dependence on an imported name.
- Clarify rules for when modifier keywords should be on forward
declarations
and definitions.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
For now, a builtin function is defined by specifying a string literal
initializer in a function declaration:
```carbon
fn MyBuiltin(a: i32) -> i32 = "builtin.name";
```
End-to-end support is included for a sample `"int.add"` builtin
performing integer addition, covering constant evaluation and code
generation.
The implementation here needs substantial refactoring before we'll be
ready to start adding more builtins. That refactoring work will be
coming next. This change is aiming to checkpoint some incremental
progress.
This is to make it easier to locate various diagnostic types. I was
thinking to separately rename DiagnosticLocationTranslator to
DiagnosticTranslator, because it's now handling some type translations.
This is mostly keeping classes in one place, but FormatFn handling is
moved from DiagnosticMessage to DiagnosticEmitter for a cleaner
dependency graph where DiagnosticTypeForArg is concerned.
Fix a collection of issues that were preventing lowering for overloaded
operators from working.
Instead of creating `import_ref` instructions during name lookup in the
current block, whatever that might be, we now create them in the `file`
block always. This avoids inserting them into blocks that might not be
intended to contain them, such as functions, and avoids the IR generated
for a function depending on which names we happen to have looked up
first.
When importing a class, function, or interface, import its enclosing
scope ID. This is necessary to allow us to distinguish between functions
at interface scope, which shouldn't be lowered, and other functions, and
will also be used in future to provide qualified names for declarations
when printing types. In order to support this:
- Track the constant values of namespaces created during importing so
that we can find them when resolving an import ref. Use those constant
values to convert an enclosing scope ID from the imported IR into a
corresponding ID in the current IR.
- Change how we do two-pass import of classes and namespaces so that we
can do two-pass import even for non-defining declarations, so that we
can import the enclosing scope.
While working on the final point above, I reworked `TryResolveInst` to
return a flag indicating whether another pass is necessary instead of
implicitly encoding this in the `ConstantId`. This permits the handling
of classes to be simplified; now `import_ir_constant_values` is only
accessed in a single place.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Support is added for all overloaded operator interfaces in the current
design apart from `Assign`, which is going to require some more work to
properly handle, given that primitive assignment currently has a special
implementation for quite a few builtin types.
As we don't have support for generics yet -- in particular, generic
interfaces -- there is no support for `*With` interfaces, but homogenous
interfaces such as `Add` are supported instead.
Factor out building of call expressions so that overloaded operators can
generate calls.
Switch a few places from using specific kinds of NodeId to a general
NodeId. Because overloaded operators and other things like implicit
conversions can result in member access and function calls, those
operations can't require a specific kind of NodeId.
Add import support for associated entities, and fix import support for
interfaces and symbolic bindings. We now import interfaces in two steps,
first importing a forward declaration then a definition, just like we do
for classes. For symbolic bindings, we ensure that each BindSymbolicName
is imported only once, because its ID is used as its symbolic identity.
This is necessary because we (only) support operator interfaces that are
defined in an imported Carbon package for now.
The entire contents of `check/operator.cpp` should probably be
rethought. In particular, doing a lot of name lookups on each operator
is likely to be bad for performance. But this gets us to the point where
overloaded operators are basically working, which seems like a good
place to iterate from.
For now, the tests that the individual operators map to the right
interfaces are mostly generated by a script, but that's just because I'm
expecting a fair bit of churn in how we define the prelude and the
`impl`s -- in particular, when we add support for `AddWith`, we'll need
to update all the tests. The plan is to remove the script once things
settle down.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
Still needs more merge/redeclaration logic for import semantics, but
this felt like a reasonable point to send a PR.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This change implements the check behavior for the arrow operator.
`ptr->Foo()` is rewritten as `(*ptr).Foo()` and `ptr->(X.y)` is
rewritten as `(*ptr).(X.y)`
I wanted to choose one or the other. I think some code has been using
each from early on. We're predominately using `filename`, so
consolidating on that. This conveniently matches the [Google dev doc
style guide](https://developers.google.com/style/word-list#filename)
(which we use for docs) which says "filename: Not file name".
In toolchain:
```
╚╡git grep file_name . | wc -l
35
╚╡git grep filename . | wc -l
489
```
josh11bandJosh L
Richard Smith
Jon Ross-Perkins
Chandler Carruth
dongjinlong
CJ Johnson