The goal of this change is to start refactoring the monolithic file into separate files that will hopefully pose fewer conflicts for developers, and make it easier to skip to handling of specific functionality. It additionally addresses a scaling issue with parser.cpp where the file would continue to get larger as more features are added.
Switches Parser to a ParserContext, moves handlers to be free functions, and moves the controller logic into ParseTree. parser_handle_states.h does the declarations for handlers and little else; handlers are split out to individual files based on prefix (which is deliberately authored to cluster).
A couple things I'm avoiding based on historical discussion are:
- Having a subdirectory for all the handlers, such as `toolchain/parser/handlers/call_expression.cpp`
- Putting handlers in a namespace, such as `Carbon::ParserHandler::CallExpression`.
- The name of `Carbon::ParserHandlerCallExpression` is then necessary to minimize the chance of conflicts with semantics and lowering, where everything can be expected to be named similarly.
I'm globbing handlers because it seems hard to see missed ones under this approach -- names are too boilerplate.
I think this current setup could be split into target-per-file, but I'm not sure that's needed, so I'd delay until it becomes a build-time issue.
This builds on #2814 by adding the test framework to the toolchain. On Linux, this is 7s -> 4s for me. #2811 has more detailed timing for the explorer, which also had more dramatic changes because it's about 3x more tests run in 2x as many ways (6x total).
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This is really part of #2811, but is extracted out to allow a little review in parallelism because #2811 expects #2813. Getting this in will allow migration of toolchain tests, whereas #2811 is focused on explorer tests. For explorer test timing information, see #2811.
The syntax being used for matching deliberately mirrors the `FileCheck` setup, partly for compatibility if something changes, partly so there's nothing new to learn, partly so that we don't need to build more test updating.
Individual tests look like:
```
[ RUN ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon
To test this file alone, run:
bazel test //explorer/parse_and_execute:file_test.subset --test_arg=explorer/parse_and_execute/testdata/assert/convert.carbon
[ OK ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon (202 ms)
```
The printed command line is intended to assist developers in debugging a single test, particularly when sharding the main test. The use of a single `.subset` target means the total number of targets is constant even as the number of test files increases, which may be important for some `bazel` execution environments. I plan to make similar changes to the `glob_sh_run` implementation so that we have consistent setups, i.e. that we no longer create target-per-file scaling risks.
This uses `native_test` to share the test binary, avoiding re-linking if files are individually run.
Investigation did reveal a mistake where STDOUT/STDERR wasn't prefixed on empty output lines; this PR fixes that mistake, so that output is fully covered.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Currently ParseAndExecute-style logic is done in main.cpp and a test. #2799 is adding another test that needs it, as is #2811.
Also more clearly marks fuzzing as testing.
This is being extracted out of #2799 in order to try unblocking progress while review continues.
Provide a clear definition of our goals for the 0.1 Carbon language, and a
concrete feature-set that is expected to satisfy these goals.
The high level goal proposed for the 0.1 language is to reach an evaluation-MVP
(Minimal Viable Product): it should be sufficiently complete to evaluate its
suitability specifically with respect to fitness as a C++ successor language.
The features proposed for 0.1 language in turn focus on C++ interoperability and
a minimal subset of foundational aspects of the language.
Beyond the language itself, the other project features and milestones proposed
focus on enabling evaluation of the language design and interoperating with C++
in practice.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: jonmeow <jperkins@google.com>
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
protobuf deps have become complexly structured, but AFAICT this is still the preferred dep. Also fixes some the fuzzer include checking, but that's a trivial misconfig.
I couldn't figure out a way to actually hit a reasonable out-of-memory case once I add the maximum interpreter step count. However, the step count limit seems more important.
I've moved the todo stack limit out of function calls because there are plenty of ways to build up the todo stack without any function calls.
Fixes#2791
adds trailing comma to single element tuple
example:
```carbon
package ExplorerTest api;
fn Main() -> i32
{
var a: (i32,) = (1, 2);
return 0;
}
```
change in error message:
```diff
-type error in initializer of variable: '(i32, i32)' is not implicitly convertible to '(i32)'
+type error in initializer of variable: '(i32, i32)' is not implicitly convertible to '(i32,)'
```
I'm partly doing this because the current setup would be difficult to share with the toolchain. e.g., ProtoToCarbon isn't explorer-specific, but the only way to run it via CLI is the explorer's fuzzverter. I want a separate tool.
This change:
- Adds a //common/fuzzing:proto_to_carbon tool.
- The rest of fuzzverter is now just //explorer/fuzzing:ast_to_proto.
- The change simplifies overall handling and removes a LLVM CLI dependency.
- Stops allowing unknown fields in the proto.
- This has mostly led to forgetting to remove fuzzer inputs that were for removed features.
- Moves more non-explorer-specific bits to //common/fuzzing.
- Cleans up remaining pieces in //explorer/fuzzing
- Merges the //explorer/fuzzing proto tests, which deduplicates AstToString copies.
- These tests also had duplicate dependencies, etc -- and all complete in ~6s.
- Updates and fixes regen_corpus which was previously broken by other changes.
- Updates the README to reflect changes.
- Removes obsolete proto-fuzzer build configuration (AFAICT this is no longer needed).
This also clusters the various `var` tests. This tests similar patterns for a few syntaxes, but `fn f(x: i32, i32) {}` and local variables `var (x: i32, i32);` and `var i32;` were the crashers I found.
In `fn`, the `x: i32,` is helpful to convince the parser that this is valid syntax.
Fixes#2778
Removes `__continuation`, `__await`, and `__run`.
In part here, the discussion was that while the feature had been useful for validating the early explorer design, it's no longer needed for that role as the explorer is now quite robust. Continuations have been experimental and, at this point, don't have an owner pushing to a proposal.
The triggering factor is that, as we push to address fuzzer issues, I ran into a crash bug in this code; basically, `fn Main() -> i32 { __await; return 0; }`. When I mentioned this, the reaction seemed to trend towards removal of the feature.
Right now, running the fuzzer in fuzzing mode is pretty noisy due to this output; it makes it hard to see the fuzzer's own progress output. It's also not really needed: users can run explorer on inputs more directly, without using this tool (albeit with fuzzverter), if they want to see detailed results.
running clang-tidy locally got a bunch of warning.
I have ignored `bugprone-unchecked-optional-access` for now, because there are large number of occurrences.
At present, `var *;` is a crash because it has errors that weren't being properly stored/handled. This changes the handling to give more to track invalid parses in patterns.
This is adding more validation of intrinsics. It addresses a bug in rand where CHECK-fails would occur for bad range inputs, instead of a runtime error. I'm also addressing what I think was an int32 range issue in the handling by running the generator with int64.
Note the dest_class case didn't actually check dyn_cast success until after using the pointer. That's the case I noticed first, but also cleaning up a couple other small related things.
This case previously crashed. Trying to handle allow_implicit_conversion better per discussion; I've split out the for loop for each path because I think it's now clearer this way.
This proposal replaces the termination algorithm for `impl` selection. The previous algorithm relied on a recursion limit, which is counter to [our goal for predictability](/docs/design/generics/goals.md#predictability).
The replacement is to terminate if any `impl` declaration depends transitively on itself with a "strict superset" type as one of its parameters.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Satakshi Garg <56358125+satakshigarg@users.noreply.github.com>
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Move concrete type check for Variable Definition (Local variable) from interpreter to type checker using the `ExpectConcreteType` function created in #2748
Jon Ross-Perkins
Amr Hesham
Richard Smith
Manmeet Singh
Lucile Rose Nihlen
masbuz
Poorna Gunathilaka
Adrien Leravat