carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-05 22:02:55 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Use new test framework in explorer to reduce per-test overhead (#2811)

Browse Source 
Isolating test execution time (no build time included):

- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s

In GH actions:

- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
  - test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
  - test / test (macos-12, opt) (pull_request_target) Successful in 9m

I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
This commit is contained in:
Jon Ross-Perkins
2023-05-15 16:23:59 -07:00
committed by GitHub
parent ee9f883ef3
commit 82f33ff384
763 changed files with 556 additions and 1899 deletions
Download Patch File Download Diff File Expand all files Collapse all files
explorer/testdata/array/element_convertible_to_type.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 3
package ExplorerTest api;
explorer/testdata/array/fail_index.carbon
Vendored
+1 -3
View File
@@ -3,13 +3,11 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
fn Main() -> i32 {
var x: [i32; 2] = (0, 1);
// CHECK:STDERR: RUNTIME ERROR: {{.*}}/explorer/testdata/array/fail_index.carbon:[[@LINE+1]]: index 2 out of range in (0, 1)
// CHECK:STDERR: RUNTIME ERROR: explorer/testdata/array/fail_index.carbon:[[@LINE+1]]: index 2 out of range in (0, 1)
return x[2];
}
explorer/testdata/array/fail_negative_size.carbon
Vendored
+1 -3
View File
@@ -3,13 +3,11 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
fn Main() -> i32 {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/array/fail_negative_size.carbon:[[@LINE+1]]: Array size cannot be negative
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/array/fail_negative_size.carbon:[[@LINE+1]]: Array size cannot be negative
var x: [i32; -1] = ();
return x[0];
}
explorer/testdata/array/fail_print_uninitalized_array_element.carbon
Vendored
+1 -3
View File
@@ -3,14 +3,12 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest impl;
fn Main() -> i32 {
var my_array : [i32; 1];
// CHECK:STDERR: RUNTIME ERROR: {{.*}}/explorer/testdata/array/fail_print_uninitalized_array_element.carbon:[[@LINE+1]]: Printing uninitialized value
// CHECK:STDERR: RUNTIME ERROR: explorer/testdata/array/fail_print_uninitalized_array_element.carbon:[[@LINE+1]]: Printing uninitialized value
Print("{0}", my_array[0]);
return 0;
}
explorer/testdata/array/fail_size_mismatch.carbon
Vendored
+1 -3
View File
@@ -3,13 +3,11 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
fn Main() -> i32 {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/array/fail_size_mismatch.carbon:[[@LINE+1]]: type error in initializer of variable: '(i32, i32, i32)' is not implicitly convertible to '[i32; 2]'
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/array/fail_size_mismatch.carbon:[[@LINE+1]]: type error in initializer of variable: '(i32, i32, i32)' is not implicitly convertible to '[i32; 2]'
var x: [i32; 2] = (0, 1, 2);
return x[0];
}
explorer/testdata/array/index.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/array/nested.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/array/uninitialized_global_array_access.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: 100
// CHECK:STDOUT: result: 0
explorer/testdata/array/uninitialized_local_array_access.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: 100
// CHECK:STDOUT: result: 0