This was bugging me after I saw all the strings; it feels like this is why we have EnumBase on the toolchain side.
I've included the move of EnumBase to //common because I figured it's reasonable to evaluate together; if we don't want EnumBase in this case, it doesn't make sense to move.
The goal here is to (significantly) reduce the boilerplate needed when defining classes that wrap enums, especially those managed with the `.def`-file style X-macros that are common in the toolchain.
This should also provide both better and more consistent functionality to those classes once ported over to it.
Initially, only `ParserState`, `SemanticsNodeKind`, and `SemanticsBuiltinKind` are ported as these were also the three that JonMeow ported in his original pull/2453 "option 5". This is heavily based on that version of the code.
Goals I was considering that influenced the design:
- Keep the individual enum-wrapping classes as simple and easy to read as possible. Especially important is keeping the `.def` files that are often filled with really important documentation clean and easy to maintain over time.
- Don't rely on computed `#include`s as that is an especially dark corner of the preprocessor and breaks some build systems.
- Have a really good API of the enum-wrapping class, including nice constant names for the values, easy printing, and even easy debugger-callable methods to get the name (as opposed to the integer value).
- Keep the API that users interact with in the base class as clean and easy to read as possible.
- Reduce the boiler plate for each instance of these as much as possible.
- Avoid excessive inline generated code or constants that would result in steady growth in object file sizes and linker effort doing deduplication.
These goals aren't always compatible, so we end up needing to pick a compromise between them when in tension. I think this version is a pretty good compromise.
The original version I started with already pull most of the API into a CRTP-style base class. This version pulls *all* of the common API. This is the main tool for getting consistency and avoiding duplication. However, connecting this base class to the individual enum wrappers is still difficult. Some specific changes here that try to do as much as possible there:
- Use a slightly fancier macro pattern to reduce the boilerplate of defining the raw `enum class` prior to the wrapper class.
- Use a macro to simplify naming the base class.
- Move the name table to a `.cpp` file to avoid every inclusion generating a complete copy of the strings (that the linker has to deduplicate). This is done with some care to sharply reduce the boilerplate needed in that `.cpp` file.
- Sink the name _API_ fully into the CRTP base class. This requires some significant complexity in the implementation, but all of that is hidden behind a single implementation detail macro, and the API itself is simple and readable. This also makes it much more reasonable to test the entire system a single time next to the base class.
This version also moves from constant factory functions to normal constants. This requires two batches -- first a declaration, and then a definition -- but the API result is significantly better and similar to the original option, the macro structure reduces the cost of these. Unfortunately that makes the adoption a bit noisy, but I think its worth the churn.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This switches the Verify method to walk postorder so that we can see how much subtree_size is really used, and shift towards removing it. It also starts calling Verify.
Also, I think I'd lost the reserve/size check during Parser refactoring, so I'm putting that back in as part of Verify.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Also does `if` support, discussed refactorings, like PopState/PushState instead of edits.
With these changes, I'm to where I can start talking about what's still missing:
```
//toolchain/parser/testdata:basics/fail_invalid_designators.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:basics/fail_paren_match_regression.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:basics/function_call.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:basics/package.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:basics/structs.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:basics/tuples.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:basics/var.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:for/fail_colon_instead_of_in.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:for/fail_missing_in.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:for/fail_missing_var.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:for/nested.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:for/simple.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:function/definition/with_params.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:operators/associative.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:operators/fail_missing_precedence_and_or.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:operators/fail_missing_precedence_or_and.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:operators/fail_variety.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:operators/fixity.carbon.test FAILED in 0.9s
//toolchain/parser/testdata:operators/missing_precedence_not.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:operators/postfix_unary.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:operators/prefix_unary.carbon.test FAILED in 0.7s
//toolchain/parser/testdata:while/basic.carbon.test FAILED in 0.6s
//toolchain/parser/testdata:while/fail_unbraced.carbon.test FAILED in 0.6s
```
This does modify a couple `if` tests to not test so much expression syntax -- that just seems like unrelated syntax.
The intent of this approach is to eliminate recursion limits as a barrier for the parser. While it may not be urgent to address, I want to avoid pouring effort into a parser approach that we don't think will be usable long-term.
Right now this is passing a minor set of tests. It's intended to be enough to show how I'm thinking about flow control for the parser. I'm manually switching back and forth because it seemed like the easiest approach that avoids duplicating tests.
This switch is being done in order to make it easier to update tests when the parse tree structure changes. I've been spending a lot of time doing such updates as part of refactoring the parse tree, and I expect more, so that's really at the root of all the refactoring I've been doing to lit updates.
Working on toolchain semantics:
- SemanticsIR is set up as a container for the semantic tree.
- SemanticsIRFactory builds the tree, with separate transformations for each ParseNodeKind.
- ParseSubtreeConsumer is a helper for transforming a ParseTree::Node's children, managing size/nodes to prevent errors.
- The nodes subdirectory contains SemanticIR nodes.
- MetaNode is used to represent nodes which have "sub-classes": Statements, Declarations, and Expressions.
- MetaNodeBlock is used to represent nodes which exist together in a block with name lookup: Statements and Declarations (not Expressions).
This is traversing children first in order to address the RPO format of ParseTree. This means that when lists are formed, they're reversed to be in code-order (`FixReverseOrdering`).
This is still very much incomplete -- the main intent at present is to demonstrate structure.
The small size is for the 1m vs 5m time limit -- all these tests _should_ be fast so a lower limit seems consistent, and the 5m timeout was getting in my way when trying to debug *actual* timeouts.
The Carbon::Testing bit is for convenience -- test libraries are generally using it, it seems like the tests should too. Note this reduces the need for `using`.
This does push NodeMatchers into Carbon::Testing -- I don't think this was benefiting from having its own namespace; `using namespace` is discouraged [under style](https://google.github.io/styleguide/cppguide.html#Namespaces), we wouldn't support an equivalent in Carbon, and it feels like it's not helping to avoid name collisions. (also tidy was bugging about it, and while I could NOLINT that, this felt like the better approach)
This moves over to the vanilla upstream GoogleTest pulled in the more
expected manner with Bazel. It also adds Abseil and Google Benchmark
libraries in the same fashion (there are cross dependencies here).
As part of this, also introduce a dependency check test that can enforce
basic layering of dependencies. For example, this lets us ensure that
non-test Carbon code only depends on LLVM and Clang despite having other
libraries available. There remains some cleanup to improve the way these
dependency tests work, but this at least ensures we don't regress.
I've also provided workarounds to allow both Carbon code and LLVM code
to freely be used with GoogleTest (and other `std::ostream` based
output code). This is done by extending the code in
`//common/ostream.h`. One downside is that it requires opening the
`llvm` namespace and adding an ADL_found overload there. I think on
balance this is still a win and doesn't make me too nervous.
The new version of GoogleTest requires printing more often from matchers
and so I've also added several printing routines to types that
previously didn't require them. Otherwise, most of the updates are just
using the more conventional upstream style of including the headers and
adding `ostream.h` where it is needed.
I did consider moving code over to use `std::ostream` instead of LLVM's
`raw_ostream`, but the advantages of not doing virtual dispatch still
seem significant, and it also seems good to retain access to LLVM's
formatting utilities built around `raw_ostream` given that we can't pull
arbitrary dependencies into Carbon code outside of test code.
All of this was slightly motivated by requests for newer features in
GoogleTest, but much more-so by my desire to have access to Google
Benchmark and Abseil when writing benchmarks. For example, using
Abseil's random number generator seems extremely helpful when generating
inputs for benchmarks. The growing dependencies between these packages
further motivated me to just pull them all in and ensure they worked
well.
This supports both struct type literals, `{.x: i32, .y: i32}`, and struct
value literals, `{.x = 3, .y = 4}`. The degenerate case of `{}` is
treated as a struct value literal, with the expectation that an empty
struct value has the same type/value duality as an empty tuple value.
Per https://github.com/bazelbuild/rules_cc this still isn't necessary. There's no build-time enforcement, so usage is inconsistent/incorrect. Rather than letting this linger, remove it pending Bazel tooling enforcing it.
This should clean up our top level directory and the build patterns.
No non-mechanical edits here. Just injecting `toolchain/` and
`TOOLCHAIN_` and then running formatting tools.
Jon Ross-Perkins
Chandler CarruthandJon Ross-Perkins
pk19604014
Jon MeowandGeoff Romer
Richard Smith