Adding `name` to `glob_lit_tests` will make it conform with other
implementations of `glob_lit_tests` out there. If someone uses this repo
while providing a different version of `glob_lit_tests` that requires a
name, those build rules will become invalid.
To filter based on file contexts, you can use `-trace_file_context=...`
flag along with `--trace_file=...` flag. These are the following options
you can pass to the `-trace_file_context=...` flag (you can also pass
them as list by separating them with comma).
1. `main`: Include trace output for file containing the main function.
2. `prelude`: Include trace output for prelude.
3. `import`: Include trace output for imports.
4. `include`: Include trace output for all.
If the flag isn't used or non of the options are passed, by default only
main file context is traced.
File contexts are distinguished based upon the source location by
passing the source location to `TraceStream::is_enabled(..)` as
arguments. If no arguments are passed, file context is
`FileContext::Unknown` and for now it will add the trace from unknown
file context.
Note: `import` option currently doesn't work as imports aren't supported
yet.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
For `-trace_phase` flag, makes the selection of multiple phases possible
by separating them by comma.
**Example Usage:**
```
bazel run //explorer -- <program_location> --trace_file=... -trace_phase=source_program,execution
```
## Description
For filtering the trace output, we currently pass flags like
`-trace_type_checking`, `-trace_execution` etc individually and these
can be passed in a combination.
This PR introduces `-trace_phase` flag that takes a list of the
following options:
- `name_resolution`: Include trace output for the name resolution phase.
- `control_flow_resolution`: Include trace output for the control flow
resolution phase.
- `type_checking`: Include trace output for the type checking phase.
- `unformed_variables_resolution`: Include trace output for the unformed
variables resolution phase.
- `declarations`: Include trace output for printing declarations.
- `execution`: Include trace output for program execution.
- `timing`: Include timing logs indicating the time taken by each phase.
- `all`: Include trace output for all phases.
### Example usage:
```
bazel run //explorer -- <program_location> --trace_file=... -trace_phase=type_checking,execution,declarations
```
Overall, cleaning up remaining lit uses.
#2851 had removed FileCheck invocations from some of the explorer tests; this starts as just restoring that. But, now that we have far fewer `lit` tests, it seems best to refine `lit.cfg.py` to focus on providing fewer commands (not all were even used).
Also, adding testing of an error to explorer (which I noticed due to a change that would've broken that) made me notice that autoupdate_lit_test's for_lit logic didn't actually work, so I'm just cutting it and going to manual updates. Really, we might want to just remove lit autoupdate support altogether since it's only a couple tests using it, but I'm not ready to make that change right now.
Fixes#2912
By implementing these improvements, users will have the ability to choose specific parts of the trace output.
Currently, when executing a file using the explorer with the --trace_file=- or --trace_file=filename.txt flag, the resulting output is an extensive and verbose log containing all the information.
In this PR, I have introduced the `ProgramPhase` enum class, which have distinct phases encountered during the compilation of a program in the explorer. Each member of this enum class corresponds to a specific phase, signifying the relevant information to be included in the trace output.
The phases covered by the `ProgramPhase` enum class are as follows:
1. Printing the source program
2. Name resolution
3. Control flow resolution
4. Type checking
5. Unformed variable resolution
6. Printing declarations
7. Printing the timings
8. Printing whole output.
These phases can be selected by passing the following compiler flags along with `--trace_file=-`.
`-trace_source_program`, `-trace_name_resolution`, `-trace_control_flow_resolution`, `-trace_type_checking`, `-trace_unformed_variables_resolution`, `-trace_declarations`, `-trace_execution`, `-trace_timing` and `-trace_all`. If none of these flags is passed only execution trace will be added to the output.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
- Moves most parts to //testing/lit_test to be consistent with //testing/file_test.
- Separates the autoupdate script out because it's shared between lit_test and file_test now, not lit-specific.
- Renames scripts to autoupdate_testdata (or autoupdate_lit_testdata for explorer's extra) to be more consistent with the non-lit-specific setup.
- Switches from execv to subprocess.call to head off a subtle issue regarding execution of multiple scripts, which we're likely to want in the future. Mostly in this PR because everything was already being touched.
- Removes autoupdate's dependency on merge_output in order to (a) better support the division of lit and non-lit logic and (b) remove a subprocess, for reasons similar to file_test's removal of subprocesses.
Isolating test execution time (no build time included):
- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s
In GH actions:
- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
- test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
- test / test (macos-12, opt) (pull_request_target) Successful in 9m
I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).