Continuing on #3070.
I moved ParseTree::Node to just Parse::Node, versus Parse::Tree::Node.
Other name changes are just removing "Parse" or "Parser" prefixes.
In EnumBase, I'm directly defining operator<< because the ostream.h
approach just isn't working, not for either of Parse::State nor
Parse::NodeKind. Errors look like:
```toolchain/parser/parser_context.cpp:449:34: error: invalid operands to binary expression ('llvm::raw_ostream' and 'const Carbon::Parse::State')
output << "\t" << i << ".\t" << entry.state;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~
```
The expected template in `Carbon::` is not in the error list; I only see
the:
```
./common/ostream.h:112:6: note: candidate template ignored: requirement 'std::is_base_of_v<std::ostream, llvm::raw_ostream>' was not satisfied [with S = llvm::raw_ostream, T = Carbon::Parse::State]
auto operator<<(S& standard_out, const T& value) -> S& {
^
```
I'm still prodding at this, but not seeing an obvious fix.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
A lot of this is more boring "remove unused header", plus some other
minor cleanups. I think the most significant changes were:
- yaml_test_helpers.cpp is doing a switch on an unsigned int, comparing
to enum values.
-
[bugprone-switch-missing-default-case](https://clang.llvm.org/extra/clang-tidy/checks/bugprone/switch-missing-default-case.html)
is unhappy with EnumBase, but correctly identified
yaml_test_helpers.cpp, so I'm opting to address it rather than disabling
it even though it needs NOLINT in several locations as a result, in
addition to what I think are some low-value `default` cases. I'd be fine
going the other way with this too and disabling it globally (I could see
it being noisier in the explorer).
- MarkInitializerFor swaps the argument names between the .h and .cpp. I
think the .cpp had the order as intended.
- There's a new-ish
[performance-enum-size](https://clang.llvm.org/extra/clang-tidy/checks/performance/enum-size.html)
which I'm basically treating as "add int8_t to enums".
My main motivation here is to just clean up as many of these as I can so
that I stop seeing them in vscode.
Trying to improve diagnostic names and code flow. I was thinking
"operand" might be a better name than "name" in the context of what `a`
is in `a[b]`, where we already call `b` the `index`.
Flow-wise, note this replaces pushing BuiltinError with instead creating
Index nodes that may contain an error. This is intended to aim towards
producing a more consistent SemIR in the face of errors. There are pros
and cons of both approaches, but I'm aiming for simpler control flow.
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
We already needed to manually flush diagnostics along every path out of
the driver, so switch to a CHECK-failure if we get this wrong rather
than a potential use-after-lifetime bug.
Switch the driver to flush explicitly rather than using a bunch of
cleanup lambdas, now that we have checking that we get this right.
This is a follow-up after #3126.
- Only allow assignment at the top level in an expression statement.
- Allow both negation and complement as subexpressions of both bitwise
and numeric operators.
- Remove parsing support for postincrement and postdecrement.
- Add parsing support for `as` operator.
- Use the same ambient precedence for types and non-type expressions.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Refactor type canonicalization so that we can reuse the same code for
building a `T*` expression and for forming the type of an `&x`
expression.
Add basic computation of expression category in order to check that we
only take the address of durable reference expressions. This is
currently computed on demand rather than being tracked as part of the
semantics node, but in most cases can be determined by looking at only a
single expression, so caching it in the node doesn't seem worthwhile
yet. This decision should be revisited if we start doing more complex
category calculations.
Also add trivial lowering support, but it doesn't work properly yet
because lowering doesn't yet take the expression category into account.
Semantic handling for use of `T*` and `const T` as types.
There's no way to form values of these types yet, and no conversions for
them are supported.
Factor out the common code to canonicalize types using a folding set,
and switch to using the same folding set for all kinds of type by adding
the kind as part of the folding set key.
Improve type printing to not include the `as type` portion when the type
is printed in a context within another type where a conversion to `type`
is implied, as in `{}*` and pre-existing cases like `({}, {}) as type`
(which we used to print as `({} as type, {} as type}) as type`.
This also makes the filename a reference to the buffer since the line
seems better to have as a reference (versus copying a string per error).
Most tests now have different line deltas due to the extra output, but
the actual errors should overall stay the same.
Some of the error locations look like they could be improved, but this
change is only making it clear where they were before.
The lack of comments makes it difficult to be sure, but if any of these
are necessary, I'd like to make sure there are comments to explain why a
specific value was shown (which I do add for diagnostics_ in this
change).
This handles namespacing of functions. Parsing and semantics are changed
significantly, while lowering works without changes. Variables can't be
namespaced yet because they're dealing with patterns, and I didn't dig
through that code.
Most of the logic is done through the new name declaration stack, which
is necessary because semantics isn't quite sure where the declaration
name ends. It'd be complex for parsing to send a signal about this,
probably involving node variants and rewrites of the tree, and this
solution seems to work well. Unfortunately this means a new stack, but
that may be inevitable due to the extra information needing to be
tracked.
Note this doesn't deal with scoped lookups of non-namespace things,
which we'll need for generics. That'll probably involve pushing resolved
scopes onto a stack (or maybe just setting a singleton value?) to affect
contextual name lookup. But, I think the basics are there to make it
work when we can test the behavior.
This renames "designator expression" to "qualified expression" and adds
"qualified declaration" in order to use terminology more consistent with
C++.
Namespaces will probably need to be considered for name mangling down
the line, but this still uses the basic name.
Add validation that every code block in a function is terminated by a sequence of terminating instructions, and that terminators don't appear anywhere else in code blocks.
This required tracking whether we're in a reachable code block. That's done on the fly when we create a new code block; the new `SemanticsNodeBlockId::Unreachable` is used to represent the case where we're not actually creating a code block because we're in unreachable code.
This is just a cleanup. Since we now have a testing directory, I think this is a better home for testonly libraries than //common. (I was thinking about this when I was considering adding more test_raw_ostream deps)
We model `if a then b else` as a prefix operator for parsing precedence purposes. The rule that a statement starting with `if` is never an `if` expression is handled implicitly because the statement parser never invokes the expression parser for a statement starting with `if`.
This exposed a bug in our diagnosis of the whitespace rule for prefix operators, which was incorrectly being applied to non-symbolic operators in some cases, and was producing a bogus second diagnostic in some cases, which is also fixed here.
This is addressing an issue left behind by the context switch, removing a few diagnostics that had been in the header rather than figuring out proper homes. I'm splitting one for semis a little further, sharing one, and then the other two are actually able to be moved into more specific homes as-is (one is only used in one place, clearly an oversight that it wasn't there already).
This builds on #2814 by adding the test framework to the toolchain. On Linux, this is 7s -> 4s for me. #2811 has more detailed timing for the explorer, which also had more dramatic changes because it's about 3x more tests run in 2x as many ways (6x total).
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This handles the basics of type and value for structs. Structurally, these look like parameters and arguments (respectively) because expressions/generics may result in multiple IR nodes being generated.
Because `{}` needs to be cast to a type for storage, I'm also adding some validation that's not specific to `{}`, e.g. that `1` shouldn't be valid as a type for storage (previously, nothing errored for that).
This adds more stringification of types, particularly literals, because they come up in value errors now.
ImplicitAs is the result of me mulling whether I'm taking the right approach on type conversions. I think it needs to return a value so that if the implicit cast rewrites the value, the result is accessible to the caller. I may reorient the current TryTypeConversion logic to be more based on the ImplicitAs logic.
This was bugging me after I saw all the strings; it feels like this is why we have EnumBase on the toolchain side.
I've included the move of EnumBase to //common because I figured it's reasonable to evaluate together; if we don't want EnumBase in this case, it doesn't make sense to move.
This makes it possible to specify both deduced and regular parameters on types. It reorganizes the handling of parameter lists in order to allow more reuse of code in this approach. Both functions and types use the new DeclarationNameAndParams handling. Overall the goal here is to take advantage of commonality in structure.
Regarding destructors, the likely approach would be to use ParameterListAsDeduced directly because `destructor` is a keyword with no declaration name and no regular parameters.
Unused diagnostics were incorrectly always returning "false" regardless of whether there was an issue. It was still looking for registry file changes, not kind file changes. Also, this should only be run once per pre-commit run (pass_filenames=false).
Remove the one unused diagnostic.
We could similarly add others -- this is intended to make it easy to add more that parse essentially the same.
The functionality expected is that types will use GetDeclarationContext in order to error on certain functionality in the declaration scope loop. e.g., with how constraints and interfaces currently don't allow definitions.
I've only moved out `package` because it's only valid on the top line. It might still be good to parse it later, but with slightly different logic because it would always be an error, and the declaration context isn't quite the right framing for that.
Also unifies some errors with `fn`.
This starts handling return types on functions, and comparing types with `return` statements.
Note, errors remain poor because the type literal is currently associated with a builtin, losing the parse_node that specified it. This means we don't have the original source location to associate with, even though it may be helpful to point at the type in source. We could point at the signature overall, but my leaning is that we wouldn't want that long-term, so TODOs for now and may want to change a little about how the parse node is tracked once things are a little further along.
This adds tracking of call information plus basic type checking. It adds a builtin for the empty tuple, mainly so that I have the basis for a default function return type.
As an aside, it also unifies printing within SemanticsIR, fixing a missing comma after callables.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This change is to make it clearer what is a TODO versus unexpected behavior. I'm doing this now because I feel it's been getting a little confusing in code.
So for example if I write the code a return type `-> i32`, I get the diagnostic output plus the dump of the (invalid) IR:
```
/carbon-lang/toolchain/semantics/testdata/function/basic.carbon:37:10: Semantics TODO: HandleReturnType
cross_reference_irs_size: 1
callables: [
]
integer_literals: [
]
strings: [
]
nodes: [
{kind: CrossReference, arg0: ir0, arg1: block0, type: node0},
{kind: CrossReference, arg0: ir0, arg1: block1, type: node1},
{kind: CrossReference, arg0: ir0, arg1: block2, type: node0},
{kind: CrossReference, arg0: ir0, arg1: block3, type: node0},
]
node_blocks: [
[
],
]
```
Although the implementation is similar in size, I think the consistency benefits are helpful. registry.def -> kind.def is also consistent with other implementations.
Also switching int32_t -> uint16_t because I think we it might be enough space (?). We use uint8_t elsewhere, but that's certainly going to be too small. Sticking with int32_t felt inconsistent.
Fundamentally this `.Print()` is wrong for debug output at present because `.fixed_spelling()` can be empty. It's also inconsistent with other enums to use it. We frequently print tokens for debugging, and it's easy to forget to specify `.name()` there.
Diagnostics use formatv, so we can provide a format_provider and address it in one spot that way. It also makes it harder to just forget to do the right thing.
Protobufs code hits a warning with the latest system headers on macOS.
I figured this may have been fixed so I updated protobufs and Bazel to
the latest releases. This generally cleaned things up.
However, it actually added *more* warnings. This clearly isn't a really
well tested path. In fact, we already have a disabled warning that we'd
like for Carbon code because LLVM isn't clean for that warning.
So I've switched our warning strategy to a more durable approach of
suppressing all warnings for external repository headers and source
files. This lets us re-enable the missing warning and should fix the
protobuf warning that started me down this twisty path.
Sadly, we *have* to update to Bazel 6 in order to have the necessary
flag to use this approach to suppressing warnings, so I couldn't do this
as two PRs cleanly. =/ That's why I've bundled both the Bazel (and
protobuf) updates with the warning strategy change.
Last but not least, I've fixed several unused parameters in Carbon's
code that our warnings now catch.
Followup for #2490
The switch of DiagnosticMessage to have DiagnosticMessage means we don't need to use unique_ptr. This means that copy constructors are implicit again and don't need to be avoided, but per discussion still keeping with moves. Comments on HandleDiagnostic try to capture the use of moves there.
I'm keeping DiagnosticLevel at the top-level, and adding some checking that notes are actually Notes.
Also, adding MakeMessage to unify some of the logic (this has particularly been bugging me around format_fn, and I ran into it here because of the Diagnostic -> DiagnosticMessage change). The addition of NoTypeDeduction is intended to avoid some duplicative comments that'd been piling up.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This is currently used once for PreviousDefinition in semantics.
This PR doesn't just add a builder, it also adds support to the emitter itself to collect notes attached to a diagnostic, and to the consumers and emitters to print all of them.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
In theory we're doing a central registry so that we can ensure there's at least one test for each. This isn't doing that, but I'm trying to validate that the central registry isn't leading to duplicates or abandoned checks (and catching a couple of each).
When binding a name, add it to name lookup. On NameReference nodes, use name lookup.
- Switches from "identifiers" to the more generic "strings". Not strictly necessary here, but it's the overall direction I think we've agreed upon and wanted to do it while building more support out.
- Starts doing deduplication of strings.
- On BindName, registers names with name lookup.
- Does name lookup based on the deduplicated string.
- Per discussion with zygoloid, design is intended to be constant-time lookup regardless of the number of parent scopes.
- Adds scopes so that we can track names which will be deregistered from lookup.
First change towards re-implementing interfaces using the new parser. I kept it small to make sure we are on the same page regarding stack states and how the parse tree should look like.
Co-authored-by: ergawy <kareem.ergawy@guardsquare.com>
This is a first pass at what semantic type checking might look like. Types propagate along nodes, we use an InvalidType object when there's an error, and once there's an InvalidType we stop doing so much type checking.
This adds some RealLiteral handling in order to get type mismatches. I'm cautious about creating some real value for SemanticsIR (since the tokenized buffer version is a bit constrained), so I'm not doing that yet. But I will probably need to in order to maintain SemanticsIR having hermetic copies of its data, without a parse tree dependency.
Summary:
Adds parsing support for the `package` directive as specified by the
`Code and name organization` design doc.
Co-authored-by: ergawy <kareem.ergawy@guardsquare.com>
Jon Ross-PerkinsandRichard Smith
Farzana Ahmed SiddiqueandFarzana Ahmed Siddique
Geoff RomerandRichard Smith
Saeed Ahmed
Kareem Ergawy
