Use a thread pool when building runtimes (#6133)

This parallelizes the compilations and dramatically reduces the time to
build runtimes.

As part of this, teach the driver infrastructure to have an option to
control the use of threads and to build the relevant thread pool and
thread it into the various APIs.

However, it requires our `ClangRunner` to become thread-safe and to
invoke Clang in a way that is thread-safe. This is somewhat challenging
as the code in `clang_main` is distinctly _not_ thread-safe.

To address this, the relevant logic of `clang_main`, especially the CC1
execution, is extracted into our runner and cleaned up to be much more
appropriate in a multithreaded context. Much of this code should
eventually be factored back into Clang, but that will be a follow-up
patch to upstream.

Last but not least, this rearranges the `ClangRunner` API to make a bit
more sense out of the different options for building runtimes, and have
a clean model for which things need to be passed in at which points.

---------

Co-authored-by: Dana Jansens <danakj@orodu.net>
This commit is contained in:
Chandler Carruth
2025-09-30 12:54:51 +00:00
committed by GitHub
co-authored by Dana Jansens
parent a6bb11f1cf
commit 35fb000536
10 changed files with 366 additions and 150 deletions
+4 -3
View File
@@ -118,9 +118,10 @@ auto LinkSubcommand::Run(DriverEnv& driver_env) -> DriverResult {
clang_args.append(options_.object_filenames.begin(),
options_.object_filenames.end());
ClangRunner runner(driver_env.installation, &driver_env.runtimes_cache,
driver_env.fs, driver_env.vlog_stream);
ErrorOr<bool> run_result = runner.Run(clang_args);
ClangRunner runner(driver_env.installation, driver_env.fs,
driver_env.vlog_stream);
ErrorOr<bool> run_result = runner.Run(clang_args, driver_env.runtimes_cache,
*driver_env.thread_pool);
if (!run_result.ok()) {
// This is not a Clang failure, but a failure to even run Clang, so we need
// to diagnose it here.