carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-06 07:14:42 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Use new test framework in explorer to reduce per-test overhead (#2811)

Browse Source 
Isolating test execution time (no build time included):

- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s

In GH actions:

- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
  - test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
  - test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
  - test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
  - test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
  - test / test (macos-12, opt) (pull_request_target) Successful in 9m

I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
This commit is contained in:
Jon Ross-Perkins
2023-05-15 16:23:59 -07:00
committed by GitHub
parent ee9f883ef3
commit 82f33ff384
763 changed files with 556 additions and 1899 deletions
Download Patch File Download Diff File Expand all files Collapse all files
explorer/testdata/generic_function/apply.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: -2
package ExplorerTest api;
explorer/testdata/generic_function/call_at_compile_time.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/generic_function/fail_implicit_conversion_extra_field.carbon
Vendored
+1 -3
View File
@@ -3,15 +3,13 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
fn Bad[T:! type](x: {.a: i32, .b: T}) {}
fn Main() -> i32 {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_implicit_conversion_extra_field.carbon:[[@LINE+1]]: mismatch in field names, source field `c` not in destination type `{.a: i32, .b: T}`
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_implicit_conversion_extra_field.carbon:[[@LINE+1]]: mismatch in field names, source field `c` not in destination type `{.a: i32, .b: T}`
Bad({.b = 5, .a = 7, .c = 2});
return 0;
}
explorer/testdata/generic_function/fail_implicit_conversion_missing_field.carbon
Vendored
+1 -3
View File
@@ -3,15 +3,13 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
fn Bad[T:! type](x: {.a: i32, .b: T}) {}
fn Main() -> i32 {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_implicit_conversion_missing_field.carbon:[[@LINE+1]]: mismatch in field names, destination field `b` not in source type `{.a: i32}`
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_implicit_conversion_missing_field.carbon:[[@LINE+1]]: mismatch in field names, destination field `b` not in source type `{.a: i32}`
Bad({.a = 5});
return 0;
}
explorer/testdata/generic_function/fail_missing_exclam.carbon
Vendored
+1 -3
View File
@@ -3,12 +3,10 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
// CHECK:STDERR: SYNTAX ERROR: {{.*}}/explorer/testdata/generic_function/fail_missing_exclam.carbon:[[@LINE+1]]: illegal binding pattern in implicit parameter list
// CHECK:STDERR: SYNTAX ERROR: explorer/testdata/generic_function/fail_missing_exclam.carbon:[[@LINE+1]]: illegal binding pattern in implicit parameter list
fn F[T: type]();
fn Main() -> i32 {
explorer/testdata/generic_function/fail_not_addable.carbon
Vendored
+1 -3
View File
@@ -3,13 +3,11 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
fn id[T:! type](x: T) -> T {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_not_addable.carbon:[[@LINE+2]]: type error in `+`:
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_not_addable.carbon:[[@LINE+2]]: type error in `+`:
// CHECK:STDERR: could not find implementation of interface AddWith(U = i32) for T
return x + 0;
}
explorer/testdata/generic_function/fail_not_type.carbon
Vendored
+1 -3
View File
@@ -3,12 +3,10 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_not_type.carbon:[[@LINE+1]]: type error in type expression: 'i32' is not implicitly convertible to 'type'
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_not_type.carbon:[[@LINE+1]]: type error in type expression: 'i32' is not implicitly convertible to 'type'
fn F[a:! 42]();
fn Main() -> i32 {
explorer/testdata/generic_function/fail_type_deduction_mismatch.carbon
Vendored
+1 -3
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
@@ -13,7 +11,7 @@ fn fst[T:! type](x: T, y: T) -> T {
}
fn Main() -> i32 {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_type_deduction_mismatch.carbon:[[@LINE+3]]: deduced multiple different values for T:! type:
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_type_deduction_mismatch.carbon:[[@LINE+3]]: deduced multiple different values for T:! type:
// CHECK:STDERR: i32
// CHECK:STDERR: bool
return fst(0, true);
explorer/testdata/generic_function/fail_type_deduction_unused.carbon
Vendored
+1 -3
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
@@ -13,6 +11,6 @@ fn id[T:! type](x: i32) -> i32 {
}
fn Main() -> i32 {
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_type_deduction_unused.carbon:[[@LINE+1]]: could not deduce type argument for type parameter T in call
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_type_deduction_unused.carbon:[[@LINE+1]]: could not deduce type argument for type parameter T in call
return id(0);
}
explorer/testdata/generic_function/fail_type_only.carbon
Vendored
+1 -3
View File
@@ -3,14 +3,12 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package ExplorerTest api;
class C {}
// CHECK:STDERR: SYNTAX ERROR: {{.*}}/explorer/testdata/generic_function/fail_type_only.carbon:[[@LINE+1]]: syntax error, unexpected RIGHT_SQUARE_BRACKET, expecting COLON
// CHECK:STDERR: SYNTAX ERROR: explorer/testdata/generic_function/fail_type_only.carbon:[[@LINE+1]]: syntax error, unexpected RIGHT_SQUARE_BRACKET, expecting COLON
fn f[x:! i32, addr C]() {}
fn Main() -> i32 {
explorer/testdata/generic_function/fail_wrong_variable_substituation.carbon
Vendored
+1 -3
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{not} %{explorer-run}
// RUN: %{not} %{explorer-run-trace}
package Foo api;
@@ -15,7 +13,7 @@ fn Get[T:! ()](n: T) -> i32 { return 2; }
fn Main() -> i32 {
var x : X = {};
// CHECK:STDERR: COMPILATION ERROR: {{.*}}/explorer/testdata/generic_function/fail_wrong_variable_substituation.carbon:[[@LINE+1]]: cannot convert deduced value class X for T to parameter type ()
// CHECK:STDERR: COMPILATION ERROR: explorer/testdata/generic_function/fail_wrong_variable_substituation.carbon:[[@LINE+1]]: cannot convert deduced value class X for T to parameter type ()
Get(x);
return 0;
}
explorer/testdata/generic_function/generic_method.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/generic_function/implicit_conversion.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 4213
package ExplorerTest api;
explorer/testdata/generic_function/instantiate_deduced.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/generic_function/non_generic_param.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 1
package ExplorerTest api;
explorer/testdata/generic_function/nondeduced_generic_param.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 1
package ExplorerTest api;
explorer/testdata/generic_function/return_val.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/generic_function/swap.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;
explorer/testdata/generic_function/tuple_map.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 1
package ExplorerTest api;
explorer/testdata/generic_function/type_matching.carbon
Vendored
-2
View File
@@ -3,8 +3,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// AUTOUPDATE
// RUN: %{explorer-run}
// RUN: %{explorer-run-trace}
// CHECK:STDOUT: result: 0
package ExplorerTest api;