Commit Graph
2 Commits
Author SHA1 Message Date
Jon Ross-Perkins e2b1511a0d Clean up lit tests and config to reflect current uses. (#2913)
Overall, cleaning up remaining lit uses.

#2851 had removed FileCheck invocations from some of the explorer tests; this starts as just restoring that. But, now that we have far fewer `lit` tests, it seems best to refine `lit.cfg.py` to focus on providing fewer commands (not all were even used).

Also, adding testing of an error to explorer (which I noticed due to a change that would've broken that) made me notice that autoupdate_lit_test's for_lit logic didn't actually work, so I'm just cutting it and going to manual updates. Really, we might want to just remove lit autoupdate support altogether since it's only a couple tests using it, but I'm not ready to make that change right now.

Fixes #2912
2023-06-16 11:52:11 -07:00
Jon Ross-Perkins 82f33ff384 Use new test framework in explorer to reduce per-test overhead (#2811)
Isolating test execution time (no build time included):

- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s

In GH actions:

- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
  - test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
  - test / test (macos-12, opt) (pull_request_target) Successful in 9m

I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
2023-05-15 16:23:59 -07:00