autoupdate: mention in the output tests that would still fail after the update (#7877)

If a test would still fail post-autoupdate due to either mismatched
`fail_` prefix or due to being NOAUTOUPDATE, the test is reported as `?`
instead of `.` in the incremental progress output, and a message is
listed after the progress output describing the problem. For example:

```console
Problems that require manual fixes:
  - `toolchain/check/testdata/basics/zz_prefix_probe.carbon` split `bad_missing_prefix.carbon` failed; if failure is expected, add the `fail_` prefix.
  - `toolchain/driver/testdata/fail_config.carbon` succeeded; if success is expected, remove the `fail_` prefix.
```

This change caused the `NOAUTOUPDATE` test
`toolchain/driver/testdata/fail_config.carbon` to fail, because when run
under autoupdate (or with `bazel run`), the install manifest would be
present, and the test is checking that it's not. Fixed by using the VFS
to look for the manifest, which then required fixing
toolchain/driver:driver_test to set up a real filesystem VFS so that it
*could* find the manifest.

With this change, there should no longer be a reason to manually run
file_test in addition to running autoupdate.

Assisted-by: Claude via Antigravity
This commit is contained in:
Richard Smith
2026-10-01 19:40:30 +00:00
committed by GitHub
parent 3dadff7d01
commit d9106700c6
5 changed files with 136 additions and 63 deletions
+16 -2
View File
@@ -893,8 +893,10 @@ static auto ProcessFileContent(llvm::StringRef filename,
return Success();
}
auto ProcessTestFile(llvm::StringRef test_name, bool running_autoupdate)
-> ErrorOr<TestFile> {
// Implementation of `ProcessTestFile`, without special handling for
// NOAUTOUPDATE files during autoupdate.
static auto ProcessTestFileImpl(llvm::StringRef test_name,
bool running_autoupdate) -> ErrorOr<TestFile> {
TestFile test_file;
// Store the original content, to avoid a read when autoupdating.
@@ -976,4 +978,16 @@ auto ProcessTestFile(llvm::StringRef test_name, bool running_autoupdate)
return std::move(test_file);
}
auto ProcessTestFile(llvm::StringRef test_name, bool running_autoupdate)
-> ErrorOr<TestFile> {
auto result = ProcessTestFileImpl(test_name, running_autoupdate);
// Autoupdate won't modify a NOAUTOUPDATE file, so process it as normal. This
// builds the expectations, which lets us report mismatches to the user.
if (running_autoupdate && result.ok() && !result->autoupdate_line_number) {
return ProcessTestFileImpl(test_name, /*running_autoupdate=*/false);
}
return result;
}
} // namespace Carbon::Testing