Add framework for replacing lit with cc_test (#2814)

This is really part of #2811, but is extracted out to allow a little review in parallelism because #2811 expects #2813. Getting this in will allow migration of toolchain tests, whereas #2811 is focused on explorer tests. For explorer test timing information, see #2811.

The syntax being used for matching deliberately mirrors the `FileCheck` setup, partly for compatibility if something changes, partly so there's nothing new to learn, partly so that we don't need to build more test updating.

Individual tests look like:

```
[ RUN      ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon

To test this file alone, run:
  bazel test //explorer/parse_and_execute:file_test.subset --test_arg=explorer/parse_and_execute/testdata/assert/convert.carbon

[       OK ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon (202 ms)
```

The printed command line is intended to assist developers in debugging a single test, particularly when sharding the main test. The use of a single `.subset` target means the total number of targets is constant even as the number of test files increases, which may be important for some `bazel` execution environments. I plan to make similar changes to the `glob_sh_run` implementation so that we have consistent setups, i.e. that we no longer create target-per-file scaling risks.

This uses `native_test` to share the test binary, avoiding re-linking if files are individually run.

Investigation did reveal a mistake where STDOUT/STDERR wasn't prefixed on empty output lines; this PR fixes that mistake, so that output is fully covered.

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This commit is contained in:
Jon Ross-Perkins
2023-05-12 10:05:08 -07:00
committed by GitHub
co-authored by Chandler Carruth
parent b1ee45c1e4
commit 941e60ade6
17 changed files with 465 additions and 29 deletions
+1 -1
View File
@@ -10,7 +10,7 @@ package ExplorerTest api;
fn Main() -> i32 {
// CHECK:STDERR: SYNTAX ERROR: {{.*}}/explorer/testdata/string/fail_newline.carbon:[[@LINE+2]]: missing closing quote in single-line string: "new
// CHECK-EMPTY:
// CHECK:STDERR:
Print("new
line");
return 0;
@@ -10,7 +10,7 @@ package ExplorerTest api;
fn CompareStr(s: String) -> i32 {
// CHECK:STDERR: SYNTAX ERROR: {{.*}}/explorer/testdata/string/fail_raw_more_hash_tags_on_left.carbon:[[@LINE+2]]: missing closing quote in single-line string: ##"str"#) {
// CHECK-EMPTY:
// CHECK:STDERR:
if (s == ##"str"#) {
return 0;
}