A lot of this is more boring "remove unused header", plus some other
minor cleanups. I think the most significant changes were:
- yaml_test_helpers.cpp is doing a switch on an unsigned int, comparing
to enum values.
-
[bugprone-switch-missing-default-case](https://clang.llvm.org/extra/clang-tidy/checks/bugprone/switch-missing-default-case.html)
is unhappy with EnumBase, but correctly identified
yaml_test_helpers.cpp, so I'm opting to address it rather than disabling
it even though it needs NOLINT in several locations as a result, in
addition to what I think are some low-value `default` cases. I'd be fine
going the other way with this too and disabling it globally (I could see
it being noisier in the explorer).
- MarkInitializerFor swaps the argument names between the .h and .cpp. I
think the .cpp had the order as intended.
- There's a new-ish
[performance-enum-size](https://clang.llvm.org/extra/clang-tidy/checks/performance/enum-size.html)
which I'm basically treating as "add int8_t to enums".
My main motivation here is to just clean up as many of these as I can so
that I stop seeing them in vscode.
Building on #3010, the handling of incomplete lines seems like it can be
straightened out. Doing this separately because it seemed better to
demonstrate tests aren't affected by the change.
The goal of this change is to start refactoring the monolithic file into separate files that will hopefully pose fewer conflicts for developers, and make it easier to skip to handling of specific functionality. It additionally addresses a scaling issue with parser.cpp where the file would continue to get larger as more features are added.
Switches Parser to a ParserContext, moves handlers to be free functions, and moves the controller logic into ParseTree. parser_handle_states.h does the declarations for handlers and little else; handlers are split out to individual files based on prefix (which is deliberately authored to cluster).
A couple things I'm avoiding based on historical discussion are:
- Having a subdirectory for all the handlers, such as `toolchain/parser/handlers/call_expression.cpp`
- Putting handlers in a namespace, such as `Carbon::ParserHandler::CallExpression`.
- The name of `Carbon::ParserHandlerCallExpression` is then necessary to minimize the chance of conflicts with semantics and lowering, where everything can be expected to be named similarly.
I'm globbing handlers because it seems hard to see missed ones under this approach -- names are too boilerplate.
I think this current setup could be split into target-per-file, but I'm not sure that's needed, so I'd delay until it becomes a build-time issue.
I'm suspicious this is responsible for some OOMs in fuzzing... Specifically, it dumps a really big parse tree, then the auto fuzzing system OOMs trying to cache the entire output in memory.
Adjusts handling of class constants (`static const`) to use CamelCase. This probably better reflects how we use it in C++ code, treating as appropriate for CamelCase instead of under_score.
Fixes adding_children to be preorder in caller (not sure why this wasn't automated).
No automated changes.
There were some notes about default constructors being required in parse_tree.h, but they don't seem to be. Removing the default constructor forces the explicit `::Invalid` where there's no value immediately being assigned, which works fine for existing code. I think this actually reduces the chance of accidents more than the prior default-construct-as-invalid approach.
Fundamentally this `.Print()` is wrong for debug output at present because `.fixed_spelling()` can be empty. It's also inconsistent with other enums to use it. We frequently print tokens for debugging, and it's easy to forget to specify `.name()` there.
Diagnostics use formatv, so we can provide a format_provider and address it in one spot that way. It also makes it harder to just forget to do the right thing.
Initially I'd added this to lexer, this includes parser and semantics. Also adds ComparableIndexBase to unify a few common cases where <> comparisons are supported.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
LLVM's bazel build has changed a bit, so this updates the tree for that.
LLVM is also moving `llvm::Optional` to match the standard API, but it seemed simpler to just switch to `std::optional`.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
- Finishes remaining "todo" parse nodes.
- Improving error recovery for invalid designators and structs, so that the parse tree still looks similar to a valid parse tree.
- Call expressions now have the thing being called as a child (of the start) instead of a sibling.
- Use of Start is replacing use of End in several parse nodes, like structs and call expressions.
- Adjusting documentation of parse node structures in an attempt to make it more consistent and understandable.
- The current state for interfaces and if/else is mostly being documented, not altered.
This switches the Verify method to walk postorder so that we can see how much subtree_size is really used, and shift towards removing it. It also starts calling Verify.
Also, I think I'd lost the reserve/size check during Parser refactoring, so I'm putting that back in as part of Verify.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This is a first pass at what semantic type checking might look like. Types propagate along nodes, we use an InvalidType object when there's an error, and once there's an InvalidType we stop doing so much type checking.
This adds some RealLiteral handling in order to get type mismatches. I'm cautious about creating some real value for SemanticsIR (since the tokenized buffer version is a bit constrained), so I'm not doing that yet. But I will probably need to in order to maintain SemanticsIR having hermetic copies of its data, without a parse tree dependency.
Adds remaining expression support, and switches the default to Parser2.
Note, this doesn't delete the current Parser yet. I'll just do that in its own PR.
The ParseTree comments say that preorder is "easier to visualize and read". The problem is, both the ParseTree and Semantics need to operate on the postorder traversal: the ParseTree during construction, and the Semantics during processing. As a consequence, understanding the postorder traversal is important, but it's also very hard to decipher when presented preorder. This PR provides a way to see the postorder, with helpful indents to show subtrees.
This retains the preorder printing as an option for people who prefer that. I'm pretty sure it'll be easier to debug tests if we can see the postorder, so I'm making that the default.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
The intent of this approach is to eliminate recursion limits as a barrier for the parser. While it may not be urgent to address, I want to avoid pouring effort into a parser approach that we don't think will be usable long-term.
Right now this is passing a minor set of tests. It's intended to be enough to show how I'm thinking about flow control for the parser. I'm manually switching back and forth because it seemed like the easiest approach that avoids duplicating tests.
Working on toolchain semantics:
- SemanticsIR is set up as a container for the semantic tree.
- SemanticsIRFactory builds the tree, with separate transformations for each ParseNodeKind.
- ParseSubtreeConsumer is a helper for transforming a ParseTree::Node's children, managing size/nodes to prevent errors.
- The nodes subdirectory contains SemanticIR nodes.
- MetaNode is used to represent nodes which have "sub-classes": Statements, Declarations, and Expressions.
- MetaNodeBlock is used to represent nodes which exist together in a block with name lookup: Statements and Declarations (not Expressions).
This is traversing children first in order to address the RPO format of ParseTree. This means that when lists are formed, they're reversed to be in code-order (`FixReverseOrdering`).
This is still very much incomplete -- the main intent at present is to demonstrate structure.
I'm doing this to avoid macro name conflicts, following https://google.github.io/styleguide/cppguide.html#Preprocessor_Macros: "If you do export a macro from a header, it must have a globally unique name. To achieve this, it must be named with a prefix consisting of your project's namespace name (but upper case)."
Commands run:
```
sed -i 's/\(DCHECK\|CHECK\|FATAL\|MAKE_UNIQUE_NAME\|MAKE_UNIQUE_NAME_IMPL\|RAW_EXITING_STREAM\|RETURN_IF_ERROR\|RETURN_IF_ERROR_IMPL\|ASSIGN_OR_RETURN\|ASSIGN_OR_RETURN_IMPL\|DIAGNOSTIC_KIND\|RETURN_IF_STACK_LIMITED\)(/CARBON_\1(/g' $(git ls-files *.cpp *.h *.lpp *.ypp *.def ':!third_party')
sed -i 's/#undef DIAGNOSTIC_KIND/#undef CARBON_DIAGNOSTIC_KIND/' toolchain/diagnostics/diagnostic_registry.def
```
Note this isn't *quite* everything, but it's intended to be a large pass at everything:
```
╚╡git grep '#define ' *.cpp *.h *.lpp *.ypp *.def ':!third_party' | grep -v '#define CARBON' | grep -v _H_
explorer/syntax/lexer.lpp: #define YY_USER_ACTION \
explorer/syntax/lexer.lpp: #define SIMPLE_TOKEN(name) \
explorer/syntax/lexer.lpp: #define ARG_TOKEN(name, arg) \
explorer/syntax/parse_and_lex_context.h:#define YY_DECL \
migrate_cpp/cpp_refactoring/var_decl.cpp:#define ABSTRACT_TYPE(Class, Base)
migrate_cpp/cpp_refactoring/var_decl.cpp:#define TYPE(Class, Base) \
```
We may in particular want to do a pass to clean up #ifdef guards and make them be CARBON_ rooted.
There are some declaration order changes, and a few test classes switched from `struct` to `class`. However, this PR is mostly adopting `_` naming of private member variables due to the shift in naming style. None of what's here should have behavior impacts, it should just be style.
Note, there are a lot of things that *look* like they could be accessor-named, but I'm not doing that in this change. Happy to do it separately if you want me to do another PR focused on it.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This moves over to the vanilla upstream GoogleTest pulled in the more
expected manner with Bazel. It also adds Abseil and Google Benchmark
libraries in the same fashion (there are cross dependencies here).
As part of this, also introduce a dependency check test that can enforce
basic layering of dependencies. For example, this lets us ensure that
non-test Carbon code only depends on LLVM and Clang despite having other
libraries available. There remains some cleanup to improve the way these
dependency tests work, but this at least ensures we don't regress.
I've also provided workarounds to allow both Carbon code and LLVM code
to freely be used with GoogleTest (and other `std::ostream` based
output code). This is done by extending the code in
`//common/ostream.h`. One downside is that it requires opening the
`llvm` namespace and adding an ADL_found overload there. I think on
balance this is still a win and doesn't make me too nervous.
The new version of GoogleTest requires printing more often from matchers
and so I've also added several printing routines to types that
previously didn't require them. Otherwise, most of the updates are just
using the more conventional upstream style of including the headers and
adding `ostream.h` where it is needed.
I did consider moving code over to use `std::ostream` instead of LLVM's
`raw_ostream`, but the advantages of not doing virtual dispatch still
seem significant, and it also seems good to retain access to LLVM's
formatting utilities built around `raw_ostream` given that we can't pull
arbitrary dependencies into Carbon code outside of test code.
All of this was slightly motivated by requests for newer features in
GoogleTest, but much more-so by my desire to have access to Google
Benchmark and Abseil when writing benchmarks. For example, using
Abseil's random number generator seems extremely helpful when generating
inputs for benchmarks. The growing dependencies between these packages
further motivated me to just pull them all in and ensure they worked
well.
This should clean up our top level directory and the build patterns.
No non-mechanical edits here. Just injecting `toolchain/` and
`TOOLCHAIN_` and then running formatting tools.
Jon Ross-Perkins
Richard Smith
Chandler CarruthandJon Ross-Perkins
Jon MeowandRichard Smith