Commit Graph
5 Commits
Author SHA1 Message Date
Jon Ross-Perkins 82f33ff384 Use new test framework in explorer to reduce per-test overhead (#2811)
Isolating test execution time (no build time included):

- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s

In GH actions:

- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
  - test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
  - test / test (macos-12, opt) (pull_request_target) Successful in 9m

I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
2023-05-15 16:23:59 -07:00
Jon Ross-Perkins d7978deb3a Start doing more validation of intrinsics: assert, new, delete, rand (#2766)
This is adding more validation of intrinsics. It addresses a bug in rand where CHECK-fails would occur for bad range inputs, instead of a runtime error. I'm also addressing what I think was an int32 range issue in the handling by running the generator with int64.
2023-04-13 12:53:34 -07:00
Richard Smith 4b2254a61f Fix some comments after #2612. (#2614) 2023-02-21 09:58:31 -08:00
Jon Ross-PerkinsandRichard Smith 73869e9438 Modify lit_autoupdate so that it can handle errors in prelude.carbon (#2612)
The intent here is that changes to prelude.carbon shouldn't break every test that expects some error from prelude.carbon; that would be too fragile. As a consequence, this effectively ignores the line number in prelude.carbon.

This is a little complex because we don't know which line in the original source file is actually causing the error, just that there is an error. Also, the previous look-behind approach required a fixed-with prefix, whereas we want a little more than that in order to capture the filename for comparison.

This would be hard to do with extra_check_replacement because the path to bazel.runfiles is complex to calculate. As a consequence, this is basically all new code.

Co-authored-by: Richard Smith <richard@metafoo.co.uk>
2023-02-17 09:31:48 -08:00
Jon Ross-Perkins eac7c2bda4 Automate the addition of RUN and simplify RUN lines (#2292)
This was an offshoot of the discussion about how much boilerplate we could remove. lit requires RUN lines be there, everything else is optional.
2022-10-17 13:52:37 -07:00