The intent of this change is that instead of paths looking like `explorer/testdata/foo/bar.carbon` (repo-relative), they're now just `bar.carbon` (local). The consequence is that paths should be a bit more durable in various environments, and just paths should be shorter and easier to read.
The explorer's prelude is an exception to this since it comes from data, rather than being the test target. Due to the change in approaches, it needs the regex again.
Uses #2829
Isolating test execution time (no build time included):
- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s
In GH actions:
- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
- test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
- test / test (macos-12, opt) (pull_request_target) Successful in 9m
I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
Adds a simple script to merge stdout/stderr and put on labels. This is hidden to the RUN line using lit.cfg.py.
This is my solution to addressing how errors printed by the toolchain break sorting of stdout output; just put stdout first. We could also have toggles for interleaving output or such, which might help test whether we do it properly.
This also moves some previous-distributed replacement logic into lit_autoupdate_base.py: I think having that adjacent to lit.cfg.py is probably the better choice, and it reduces duplication in toolchain scripts. It happens here because I need to change the resulting commands to include the merge.
On #2224 @zygoloid pointed out we needed --implicit-check-not to ensure we were correctly matching output. This is the standard way we're writing explorer tests, so I was looking at unifying our lit approaches.
This is one take on it, making more use of substitutions to bring various testing into alignment, as well as symlinks to avoid config skew (maybe I'll eventually figure out a better solution than symlinks).
Makes a couple small fixes in explorer to remove end-of-line whitespace on output.
When checking for control flow falling off a function after a `match`, determine whether it's possible for no case to have matched. Using the same implementation, also detect whether `case`s in a `match` are unreachable.
For now, the implementation never considers a match against specific values for any type other than tuples, alternatives, and `bool` to be exhaustive. In particular, matching against the sole value `{}` of type `{}` is not considered exhaustive. This is probably best left until explorer supports matching on struct and maybe class types more generally.
This implementation closely follows the algorithm described in the paper [Warnings for pattern matching](http://moscova.inria.fr/~maranget/papers/warn/warn.pdf) by Luc Maranget. Various optimizations are possible, such as reducing the amount of copying done, but for the purposes of explorer, comprehensibility is being favored over efficiency.
The problem is, perhaps surprisingly, co-NP-hard (by reduction to the tautology problem for disjunctive normal form, which is in turn dual to the satisfaction problem for conjunctive normal form, which is well-known to be NP-hard). The algorithm is therefore exponential-time in the worst case, but seems to be well-studied and performs well enough on non-pathological examples. Nonetheless, a depth limit has been imposed to prevent pathological examples such as those generated by a fuzzer from causing long runtimes.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Jon Ross-Perkins
Richard SmithandJon Ross-Perkins