When a template has an argument that involves a generic parameter, we're supposed to delay instantiation until we know the concrete value, but explorer is not set up to do that yet, so for now we instead instantiate the template with the symbolic argument. When that happens, bring the constraints on the generic parameter into scope so they can be used inside the template instantiation.
This requires adding a new search over a value for the generic parameters that appear within it; a `VisitNestedValues` visitor is added to visit all the `Value`s nested with a value, and also convert an existing place where we were doing the same thing in a way that was incorrect (but harmlessly incorrect for now) to use it.
This is needed by #2881, which needs implementations of `ImplicitAs` for nested types when instantiating a builtin impl of `ImplicitAs` for an aggregate type.
Co-authored-by: Geoff Romer <gromer@google.com>
In explorer, we already support parsing a string_view, so use that. In toolchain, we need to build support, probably using vfs, so that's a todo.
bazel test //explorer:file_test --runs_per_test=5
- branch: Stats over 250 runs: max = 18.3s, min = 5.2s, avg = 11.2s, dev = 2.9s
- trunk: Stats over 250 runs: max = 22.3s, min = 5.9s, avg = 12.1s, dev = 2.8s
Not a dramatic improvement, but maybe more effective long-term, and this'd been requested on #2876
By implementing these improvements, users will have the ability to choose specific parts of the trace output.
Currently, when executing a file using the explorer with the --trace_file=- or --trace_file=filename.txt flag, the resulting output is an extensive and verbose log containing all the information.
In this PR, I have introduced the `ProgramPhase` enum class, which have distinct phases encountered during the compilation of a program in the explorer. Each member of this enum class corresponds to a specific phase, signifying the relevant information to be included in the trace output.
The phases covered by the `ProgramPhase` enum class are as follows:
1. Printing the source program
2. Name resolution
3. Control flow resolution
4. Type checking
5. Unformed variable resolution
6. Printing declarations
7. Printing the timings
8. Printing whole output.
These phases can be selected by passing the following compiler flags along with `--trace_file=-`.
`-trace_source_program`, `-trace_name_resolution`, `-trace_control_flow_resolution`, `-trace_type_checking`, `-trace_unformed_variables_resolution`, `-trace_declarations`, `-trace_execution`, `-trace_timing` and `-trace_all`. If none of these flags is passed only execution trace will be added to the output.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Fix misidentification of a potential cycle in the case where the inner match is missing labels from the outer match, and the inner match is strictly more complex when considering only its labels. We previously ignored labels in the outer match that are absent in the inner match, but the existence of any such label should cause us to treat the inner match as not being strictly more complex.
I'm looking at this as I start thinking about handling `import`. Syntax is based on llvm's `split-file` tool.
The `std::vector` -> `llvm::SmallVector` switch is minor, I'm doing it here because I had to touch everything anyways and I think for tests I'll lean slightly more towards the toolchain's way of doing things versus explorer's.
Per #257, we should be treating unformedness as all-or-nothing, rather than being a per-field or per-array-element property. Previously we initialized an array with no explicit initializer as containing a sequence of uninitialized values, but that led to crashes when attempting to access those values, as the checks for reading an uninitialized value only expected values to be uninitialized at the top level.
Also, we had existing tests that attempt to store to an element of an uninitialized array. We now detect that and treat it as UB during evaluation, rather than crashing due to trying to perform field access into an uninitialized value.
Finally, many of these problems can be detected statically, but the resolve_unformed pass wasn't catching them because it missed a few expression and declaration forms. Support for those cases has been added too. This causes the pass to recurse more often, and in particular our existing recursion test started hitting a stack overflow after this, so resolve_unformed now uses `RunWithExtraStack`. In passing, remove the need to explicitly tell `RunWithExtraStack` the return type, and infer it as the return type of the callable instead.
The old version wasn't compatible with ASan's instrumentation. Instead,
use a builtin when available, and even in the fallback disable inlining
and sanitizing as they can't do anything useful in this routine.
This fixes a confusing internal ASan error seen by some folks since the
stack detection was added.
There was a warning "Unchecked access to optional value" in this piece of code inside `action.cpp`
```cpp
if (this->scope().has_value()) {
out << " " << *this->scope();a
}
```
Fixed it by accessing `scope_` directly rather than using `this->scope()`.
Adds stack space handling similar to Clang's approach, but with more support for forwarding return values. Refactors ParseAndExecute for better sharing, and for centralization of the InitStackSpace handling.
Fixes#2795
The intent of this change is that instead of paths looking like `explorer/testdata/foo/bar.carbon` (repo-relative), they're now just `bar.carbon` (local). The consequence is that paths should be a bit more durable in various environments, and just paths should be shorter and easier to read.
The explorer's prelude is an exception to this since it comes from data, rather than being the test target. Due to the change in approaches, it needs the regex again.
Uses #2829
- Moves most parts to //testing/lit_test to be consistent with //testing/file_test.
- Separates the autoupdate script out because it's shared between lit_test and file_test now, not lit-specific.
- Renames scripts to autoupdate_testdata (or autoupdate_lit_testdata for explorer's extra) to be more consistent with the non-lit-specific setup.
- Switches from execv to subprocess.call to head off a subtle issue regarding execution of multiple scripts, which we're likely to want in the future. Mostly in this PR because everything was already being touched.
- Removes autoupdate's dependency on merge_output in order to (a) better support the division of lit and non-lit logic and (b) remove a subprocess, for reasons similar to file_test's removal of subprocesses.
Isolating test execution time (no build time included):
- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s
In GH actions:
- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
- test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
- test / test (macos-12, opt) (pull_request_target) Successful in 9m
I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
This is really part of #2811, but is extracted out to allow a little review in parallelism because #2811 expects #2813. Getting this in will allow migration of toolchain tests, whereas #2811 is focused on explorer tests. For explorer test timing information, see #2811.
The syntax being used for matching deliberately mirrors the `FileCheck` setup, partly for compatibility if something changes, partly so there's nothing new to learn, partly so that we don't need to build more test updating.
Individual tests look like:
```
[ RUN ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon
To test this file alone, run:
bazel test //explorer/parse_and_execute:file_test.subset --test_arg=explorer/parse_and_execute/testdata/assert/convert.carbon
[ OK ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon (202 ms)
```
The printed command line is intended to assist developers in debugging a single test, particularly when sharding the main test. The use of a single `.subset` target means the total number of targets is constant even as the number of test files increases, which may be important for some `bazel` execution environments. I plan to make similar changes to the `glob_sh_run` implementation so that we have consistent setups, i.e. that we no longer create target-per-file scaling risks.
This uses `native_test` to share the test binary, avoiding re-linking if files are individually run.
Investigation did reveal a mistake where STDOUT/STDERR wasn't prefixed on empty output lines; this PR fixes that mistake, so that output is fully covered.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Currently ParseAndExecute-style logic is done in main.cpp and a test. #2799 is adding another test that needs it, as is #2811.
Also more clearly marks fuzzing as testing.
This is being extracted out of #2799 in order to try unblocking progress while review continues.
I couldn't figure out a way to actually hit a reasonable out-of-memory case once I add the maximum interpreter step count. However, the step count limit seems more important.
I've moved the todo stack limit out of function calls because there are plenty of ways to build up the todo stack without any function calls.
Fixes#2791
adds trailing comma to single element tuple
example:
```carbon
package ExplorerTest api;
fn Main() -> i32
{
var a: (i32,) = (1, 2);
return 0;
}
```
change in error message:
```diff
-type error in initializer of variable: '(i32, i32)' is not implicitly convertible to '(i32)'
+type error in initializer of variable: '(i32, i32)' is not implicitly convertible to '(i32,)'
```
I'm partly doing this because the current setup would be difficult to share with the toolchain. e.g., ProtoToCarbon isn't explorer-specific, but the only way to run it via CLI is the explorer's fuzzverter. I want a separate tool.
This change:
- Adds a //common/fuzzing:proto_to_carbon tool.
- The rest of fuzzverter is now just //explorer/fuzzing:ast_to_proto.
- The change simplifies overall handling and removes a LLVM CLI dependency.
- Stops allowing unknown fields in the proto.
- This has mostly led to forgetting to remove fuzzer inputs that were for removed features.
- Moves more non-explorer-specific bits to //common/fuzzing.
- Cleans up remaining pieces in //explorer/fuzzing
- Merges the //explorer/fuzzing proto tests, which deduplicates AstToString copies.
- These tests also had duplicate dependencies, etc -- and all complete in ~6s.
- Updates and fixes regen_corpus which was previously broken by other changes.
- Updates the README to reflect changes.
- Removes obsolete proto-fuzzer build configuration (AFAICT this is no longer needed).
This also clusters the various `var` tests. This tests similar patterns for a few syntaxes, but `fn f(x: i32, i32) {}` and local variables `var (x: i32, i32);` and `var i32;` were the crashers I found.
In `fn`, the `x: i32,` is helpful to convince the parser that this is valid syntax.
Fixes#2778
Removes `__continuation`, `__await`, and `__run`.
In part here, the discussion was that while the feature had been useful for validating the early explorer design, it's no longer needed for that role as the explorer is now quite robust. Continuations have been experimental and, at this point, don't have an owner pushing to a proposal.
The triggering factor is that, as we push to address fuzzer issues, I ran into a crash bug in this code; basically, `fn Main() -> i32 { __await; return 0; }`. When I mentioned this, the reaction seemed to trend towards removal of the feature.
Right now, running the fuzzer in fuzzing mode is pretty noisy due to this output; it makes it hard to see the fuzzer's own progress output. It's also not really needed: users can run explorer on inputs more directly, without using this tool (albeit with fuzzverter), if they want to see detailed results.
running clang-tidy locally got a bunch of warning.
I have ignored `bugprone-unchecked-optional-access` for now, because there are large number of occurrences.
This is adding more validation of intrinsics. It addresses a bug in rand where CHECK-fails would occur for bad range inputs, instead of a runtime error. I'm also addressing what I think was an int32 range issue in the handling by running the generator with int64.
Note the dest_class case didn't actually check dyn_cast success until after using the pointer. That's the case I noticed first, but also cleaning up a couple other small related things.
This case previously crashed. Trying to handle allow_implicit_conversion better per discussion; I've split out the for loop for each path because I think it's now clearer this way.