This is trying to document the status quo. Note
https://github.com/carbon-language/carbon-lang/pull/4497 placed
restrictions on a bit, and this is in part flowing back to restrictions
on both NodeId and ImportIRInstId limits.
Adds subset conversion of `NodeIdOneOf` due to the choice usage, plus
the pre-existing TODO. Fixes incorrect information about nodes on
StructLiteral and TupleLiteral.
After this change, `UncheckedLoc` is only used in a couple import
contexts (hard to verify) plus `InstStore::GetWithLocId`.
This flows out of #5084 and trying to reduce UnsafeMake use. It turns
out imports and namespaces were using unexpected node kinds (previously
ImportIntroducer instead of ImportDecl, for example). This fixes and
adds validation.
I was uncertain about whether to just remove the is_convertible check,
since I don't see it as motivating creation of a conversion between
NodeIdOneOf types. So I've just left a TODO for now.
This doesn't change functionality, but I was seeing better diagnostics
in VS Code.
This also changes the NodeId constructors for related types (also
NodeCategory and NodeIdForKind) to use UnsafeMake for construction. That
originated from avoiding ambiguity coming from `requires`, but the
constructor mode is also one we should typically avoid (e.g., preferring
`Parse::Tree::As`).
This adds support for choice types at a similar level to that of a C
enum, where each alternative has a name but no additional
data/parameters attached to it. We generate a TODO diagnostic if
parameters are specified.
Because there's no extra data, the storage is a simple unsigned integer
discriminant of the smallest possible size.
A choice without any alternatives is not constructible. A choice with a
single alternative is, and has an empty tuple in place of a discriminant
since it has only one state. The empty tuple is used to make the class
non-constructible. This can be improved.
Each alternative is turned into a let binding on the choice that is a
value of the choice with that alternative set as the active one in the
discriminant. This isn't possible to write in user code with a class
right now, since the let binding has the same type as the choice
(which is a class) it is within. It's possible to generate it in semir
however by adding the binding after the class is marked complete.
Change representation of package names from `IdentifierId` to
`PackageNameId`, and add a special value `PackageNameId::Core` for the
Core package. Add a `Core` expression to name the Core package, and
support for parsing the `Core` keyword in `package` and `import`
declarations.
For now, I've made no changes to instruction fingerprinting or name
mangling. This means that fingerprints and mangled names will collide
between names in the `Core` package and names in a `r#Core` package. See
#4908.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This is a followup from #4834, I searched for "invalid" uses in our
codebase. This is mostly changing comments, and a couple debug
functions, but shouldn't affect testable behavior.
Note a couple things I'll highlight as not changing (but could) are:
- `ReturnTypeInfo::is_valid`
- `"invalid"` uses in the formatter
- `AddInvalid` for `!has_value` in `inst_fingerprinter` (because the
cases it's called sound invalid-ish)
High level, replacing `Id::Invalid` with `Id::None` and `Id::is_valid`
with `Id::has_value` for clarity, as discussed
[here](https://discord.com/channels/655572317891461132/655578254970716160/1331664574545395794).
The `IntId` refactoring is needed together with `AnyIdBase` because it's
also used with `ValueStore`.
Note, trying to be careful not to rewrite `EnumBase::InvalidIndex`, or
`is_valid` in general (e.g., `IdKind::is_valid`).
I've tried to sequence commits here:
1. Automatic replacements:
- `((?:Id|Index)(?: |::|\(|Base(?:\(|::)))Invalid((?:Index)?\W)` ->
`$1None$2`
- `<invalid>` -> `<none>`
- `InvalidNodeId` -> `NoneNodeId`
- `/\*invalid\*/` -> `/*none*/`
- `id((?:_|\(\))(?:\.|->))is_valid` -> `id$1has_value`
2. Manual edits:
- In `int.h` and `int_test.cpp`
- `IntT` has `is_value`, which I'm renaming to `is_embedded_value`.
- Manual edits to comments in this file.
- `AnyIdBase` and `IdBase`
- Declaration of `is_valid` -> `has_value`, `InvalidIndex` ->
`NoneIndex`.
- In `ids.h` and `ids.cpp`
- `is_valid` -> `has_value`
- `// An explicitly invalid ID.` -> `// An ID with no value.`; similar
for index
- Various math on `InvalidIndex` -> `NoneIndex`
- Various mentions of "valid" in comments
- In `value_store.h`, for `IdT::Invalid`, plus one comment
- In `impl.h` and `tokenized_buffer.h`, we had different initialization
of `::None` values (versus `ids.h` syntax) that I fixed manually.
- Spot checks to compile
- Particularly where `is_valid` replacements didn't catch spots due to
different naming.
3. Autoupdate tests
4. verbose.carbon (NOAUTOUPDATE)
5. Comment spot checks
Note there are probably other mentions of "Invalid" that should be swept
up, but I'd like to argue for merging and separating out remaining
cleanup since this is so sweeping (and likely to hit merge conflicts
from churn). We'll probably have lingering mentions of "invalid" for a
bit regardless, just because there are uses of "invalid" in non-Id APIs.
This change splits `NodeKind::IdentifierName` into separate node kinds
depending on whether the identifier is followed by parameters, and
similarly splits `NameQualifier` based on whether the qualifier has
parameters. This enables us to only push a pattern block when it's
actually needed, rather than "defensively" pushing one when it might be
needed.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
These don't fully work in check and beyond yet, because they're not
added into lexical lookup, but already mostly do the right thing.
Per #3407, disallow namespace declarations anywhere other than at file
scope for now.
We don't treat statements starting with a packaging introducer keyword
(`package`, `library`, `import`) as declarations because they're
sufficiently unlikely to occur that the error recovery doesn't seem
important, and this avoids needing to disambiguate `package.` at the
start of an expression.
This removes a lot of boilerplate `Print` functions in favor of a
CRTP-based approach that uses a `Label` field as an automatic prefix.
This `Label` is also made available for other purposes, particularly
`IdKind` crash messages in this change. In particular, for
`RequireIdKind` in node_stack.h from using numeric IdKinds (e.g., 5 and
24) to something that will print `IdKind(<label>)` (this came up
recently on #toolchain).
While I'm in here, also doing some other tinkering:
- Moving operators to be `friend` members, to reduce the extra
templating now that the base types are templated.
- Adjusts IntId diagnostics from `int [...]` to `int(...)` for
consistency with other id printing.
- Changes InstBlockId's label from "block" to "inst_block", since we
have multiple blocks now.
- Fixes StructTypeFieldsId to use "struct_type_fields" instead of
"type_block" (from `TypeBlockId`)
- Does some more adjustments from camelCase to snake_case for
consistency
Includes support for the `impls`, `=`, and `==` requirement operators to
the right of a `where`, but `and` to allow multiple requirements is
still a TODO.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
As discussed in toolchain meeting, we want to avoid overloading the
meaning of "instance", and "specific" was the best name we found. It's a
little unorthodox and inventive, but hopefully over time will become as
unsurprising as the term "generic" is.
Instead of adding a `Token<Kind>` field to typed parse nodes that
contains a `TokenIndex`, make the parse node directly hold a
`TokenIndex` of a more specific kind.
Instead of tracking the token associated with a parse node in the `.def`
file macro, track it on the typed node instead. List the token as a
field inside the node structure to show the order of the token relative
to the other components of the grammar production, and to allow the
token index to be accessed when the node is extracted.
Remove the corresponding information from the `.def` file, leaving
behind just a list of parse node kinds in the majority of cases.
This also removes the checking of the token kind associated with a parse
node in the case where the parse node has errors. Previously we had a
flag on the node kind to indicate whether we should check this, but per
[discord
discussion](https://discord.com/channels/655572317891461132/655578254970716160/1246214418979881052),
we have decided to remove this.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Mirroring #4003 for NodeCategory.
Note we template a lot more on NodeCategory's enum, so this is a
slightly more awkward delta.
Also, switch from Enum in KeywordModifierSet to RawEnumType for
consistency with EnumBase. The templating on NodeCategory had me
thinking about that more.
Parse the name of a declaration as a sequence of `NameQualifier`s --
which have a name, possibly parameters, and a trailing period --
followed by a name and possibly parameters. This prepares us for parsing
declarations of members of generic classes and similar cases, but
actually supporting such member redeclarations is left to a future
change.
We previously required functions to have parameters, but no longer do,
following the direction of #3848. Cases like namespaces that can't
actually have parameters are now diagnosed in check instead of in parse.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
For now, a builtin function is defined by specifying a string literal
initializer in a function declaration:
```carbon
fn MyBuiltin(a: i32) -> i32 = "builtin.name";
```
End-to-end support is included for a sample `"int.add"` builtin
performing integer addition, covering constant evaluation and code
generation.
The implementation here needs substantial refactoring before we'll be
ready to start adding more builtins. That refactoring work will be
coming next. This change is aiming to checkpoint some incremental
progress.
Support is added for all overloaded operator interfaces in the current
design apart from `Assign`, which is going to require some more work to
properly handle, given that primitive assignment currently has a special
implementation for quite a few builtin types.
As we don't have support for generics yet -- in particular, generic
interfaces -- there is no support for `*With` interfaces, but homogenous
interfaces such as `Add` are supported instead.
Factor out building of call expressions so that overloaded operators can
generate calls.
Switch a few places from using specific kinds of NodeId to a general
NodeId. Because overloaded operators and other things like implicit
conversions can result in member access and function calls, those
operations can't require a specific kind of NodeId.
Add import support for associated entities, and fix import support for
interfaces and symbolic bindings. We now import interfaces in two steps,
first importing a forward declaration then a definition, just like we do
for classes. For symbolic bindings, we ensure that each BindSymbolicName
is imported only once, because its ID is used as its symbolic identity.
This is necessary because we (only) support operator interfaces that are
defined in an imported Carbon package for now.
The entire contents of `check/operator.cpp` should probably be
rethought. In particular, doing a lot of name lookups on each operator
is likely to be bad for performance. But this gets us to the point where
overloaded operators are basically working, which seems like a good
place to iterate from.
For now, the tests that the individual operators map to the right
interfaces are mostly generated by a script, but that's just because I'm
expecting a fair bit of churn in how we define the prelude and the
`impl`s -- in particular, when we add support for `AddWith`, we'll need
to update all the tests. The plan is to remove the script once things
settle down.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
This change implements the check behavior for the arrow operator.
`ptr->Foo()` is rewritten as `(*ptr).Foo()` and `ptr->(X.y)` is
rewritten as `(*ptr).(X.y)`
On the parsing side, we treat `a.(b)` as a member access whose second
operand is a `ParenExpr` rather than a `MemberName`. A new node category
is added for the union of `MemberName` and `ParenExpr` to support this.
Checking is mostly reusing the same pieces we already have for simple
member access. Compound member access is in most ways a simplified form
of simple member access because it doesn't need to do any lookup.
This revamps the support for cross-package imports, making them look
more like a namespace. The planned model is mentioned on
[#toolchain](https://discord.com/channels/655572317891461132/655578254970716160/1217586076022210670).
This does not implement name lookup into the new namespace structure.
A few key changes in this PR (it's a little sprawling) are:
- Moves logic for adding package imports from context.* to import.*
- Remove SemIR::Import, which was the prior model. This is instead now a
SemIR::Namespace with the NameScope getting a new import_ir_scopes
field.
- Allow SemIR::Namespace to use Parse::ImportDirectiveId in addition to
the prior Parse::NamespaceId
- The import_ir_scopes field includes a NameScopeId so that as we
traverse to child namespaces, we can directly perform name lookup in the
other IR.
- is_closed_import now tracks whether a namespace comes from a different
package. This has a diagnostic implemented in decl_name_stack.
Consume the components of the `impl` declaration, and set up scopes for
the child elements. We don't yet build a representation for the impl
itself.
Also, add an interface type value. This is necessary so that we have a
value for the expression on the right-hand side of `as` in an `impl`.
This patches bazel_clang_tidy handling of headers. I found an equivalent
change at https://github.com/erenon/bazel_clang_tidy/pull/13, but that
was [already
rejected](https://github.com/erenon/bazel_clang_tidy/pull/13#issuecomment-1047007424).
Per the criticism, this will result in redundant processing of headers.
The project instead uses `HeaderFilterRegex: ".*"`, but that results in
two problems:
1. When running with `-k`, errors are repeated when a header is included
more than once, which is common.
2. clang-tidy including errors from headers that are included from other
modules (e.g., abseil-cpp); filtering correctly is difficult.
Given the trade-offs and options (including forking), I thought patching
was preferable so long as it remains narrow.
This involves a number of supporting changes:
* The `parse_node;` member of instruction types may now have any type
derived from `Parse::NodeId` and is no longer required to have that
exact type.
* `Parse::Node::Invalid` is now a singleton object of a separate type
that is convertible to `Parse::NodeId` and its descendants. This
replaces the `Invalid` member of its descendants, and avoids having to
write long `NodeIdOneOf<...>` types when initializing variables to
invalid.
* `IndexBase` now allows `==` and `!=` comparisons between its derived
classes and types that are convertible to those types.
* A number of functions in the check stage have been changed to preserve
more type information instead of using `Parse::NodeId`.
* `NodeIdForKind<K>` (also known as `KId`) now has a `Kind` member so it
may be used to declare `NodeIdOneOf<T, U>` types without #including
`parse/typed_nodes.h`.
* `NodeIdForKind<K>` (also known as `KId`) may be implicitly converted
to `NodeIdOneOf<T, U>` if `T::Kind == K` or `U::Kind == K` (executing a
TODO).
Many of the `parse_node` members were not converted since they would
have required more extensive changes. They have been marked with "TODO"
comments.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Goal is to increase type safety, though more work needs to be done (see
added TODOs).
Note that, after this change, check handlers corresponding to deleted
parse node kinds will no longer compile.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Use this new category to replace the unconstrained `NodeId` child of
`MemberAccessExpr ` and `PointerMemberAccessExpr`. For now this new
category matches `IdentifierName` and `BaseName`, but later this will be
expanded to support `a.(b.c)` and `p->(b.c)` syntactic forms.
QUESTION: Is it time to make a `node_category.def` x-macro file?
ANSWER: Not yet.
These are intended to allow the structure of a parse tree node to be
described more precisely in code, to support these use cases:
- Automated checking that the parse tree conforms to the expected
structure. (Added to `Tree::Verify`.)
- Easier reading and understanding of the structure of the parse tree by
toolchain developers. (See `parse/typed_nodes.h`.)
- Easier navigation of the parse tree, for example for tooling uses and
for use when forming diagnostics.
On this last point, an object representing the file may be inspecting
using `Tree::ExtractFile`, as in:
```
auto file = tree->ExtractFile();
for (AnyDeclId decl_id : file.decls) {
// `decl_id` is convertible to a `NodeId`.
if (std::optional<FunctionDecl> fn_decl =
tree->ExtractAs<FunctionDecl>(decl_id)) {
// fn_decl->params is a `TuplePatternId` (which extends `NodeId`)
// that is guaranteed to reference a `TuplePattern`.
std::optional<TuplePattern> params = tree->Extract(fn_decl->params);
// `params` has a value unless there was an error in that node.
} else if (auto class_def = tree->ExtractAs<ClassDefinition>(decl_id)) {
// ...
}
}
```
The `Extract...` functions collect the child nodes into the typed parse
node's fields (internally using a `Tree::SiblingIterator`) for easy
access. However, this is not as fast as directly observing the tree
structure using the postorder strategy being used by the check stage.
These functions rely on using struct reflection on the typed parse node
definitions from `parse/typed_nodes.h` to get the expected structure of
child nodes and then populate them.
Note that validating these in `Tree::Verify` adds significant cost to
it, and is currently included in the parsing stage. Without this change,
a 10 mloc test case of lex & parse takes 4.129 s ± 0.041 s. With this
change, it takes 5.768 s ± 0.036 s.
This builds upon and completes #3393.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Jon Ross-Perkins
Geoff RomerandJon Ross-Perkins
Dana Jansens
Richard SmithandJon Ross-Perkins
Brymer Meneses
CJ Johnson
