carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-05 22:02:55 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Use a thread pool when building runtimes (#6133)

Browse Source 
This parallelizes the compilations and dramatically reduces the time to
build runtimes.

As part of this, teach the driver infrastructure to have an option to
control the use of threads and to build the relevant thread pool and
thread it into the various APIs.

However, it requires our `ClangRunner` to become thread-safe and to
invoke Clang in a way that is thread-safe. This is somewhat challenging
as the code in `clang_main` is distinctly _not_ thread-safe.

To address this, the relevant logic of `clang_main`, especially the CC1
execution, is extracted into our runner and cleaned up to be much more
appropriate in a multithreaded context. Much of this code should
eventually be factored back into Clang, but that will be a follow-up
patch to upstream.

Last but not least, this rearranges the `ClangRunner` API to make a bit
more sense out of the different options for building runtimes, and have
a clean model for which things need to be passed in at which points.

---------

Co-authored-by: Dana Jansens <danakj@orodu.net>
This commit is contained in:
Chandler Carruth
2025-09-30 12:54:51 +00:00
committed by GitHub
co-authored by Dana Jansens
parent a6bb11f1cf
commit 35fb000536
10 changed files with 366 additions and 150 deletions
Download Patch File Download Diff File Expand all files Collapse all files
toolchain/driver/lld_runner_test.cpp
+9 -18
View File
@@ -85,22 +85,17 @@ static auto CompileTwoSources(const InstallPaths& install_paths,
// First compile the two source files to `.o` files with Clang.
RawStringOstream verbose_out;
auto vfs = llvm::vfs::getRealFileSystem();
ClangRunner clang(&install_paths, /*on_demand_runtimes_cache=*/nullptr, vfs,
&verbose_out);
ClangRunner clang(&install_paths, vfs, &verbose_out);
std::string target_arg = llvm::formatv("--target={0}", target).str();
std::string out;
std::string err;
CARBON_CHECK(Testing::CallWithCapturedOutput(
out, err,
[&] {
auto run_result = clang.Run({target_arg, "-fPIE", "-c",
test_a_file.string(), "-o",
test_a_output.string()});
if (!run_result.ok()) {
err = run_result.error().message();
return false;
}
return *run_result;
auto run_result = clang.RunWithNoRuntimes(
{target_arg, "-fPIE", "-c", test_a_file.string(), "-o",
test_a_output.string()});
return run_result;
}),
"Verbose output from runner:\n{0}\nStderr:\n{1}\n",
verbose_out.TakeStr(), err);
@@ -109,14 +104,10 @@ static auto CompileTwoSources(const InstallPaths& install_paths,
CARBON_CHECK(Testing::CallWithCapturedOutput(
out, err,
[&] {
auto run_result = clang.Run({target_arg, "-fPIE", "-c",
test_b_file.string(), "-o",
test_b_output.string()});
if (!run_result.ok()) {
err = run_result.error().message();
return false;
}
return *run_result;
auto run_result = clang.RunWithNoRuntimes(
{target_arg, "-fPIE", "-c", test_b_file.string(), "-o",
test_b_output.string()});
return run_result;
}),
"Verbose output from runner:\n{0}\nStderr:\n{1}\n",
verbose_out.TakeStr(), err);