mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-10-04 12:21:07 +01:00
This is the first step to having Clang's runtime libraries fully available for the Carbon toolchain. This PR focuses on the lowest level runtimes, the CRT files and the builtins library. The goal is to intercept Clang runs where it needs these target-dependent pieces to be available, and build them on demand using our Clang-running infrastructure. This avoids most of the subprocess overhead, but there is still some due to missing features in Clang. This requires exporting the sources for these runtimes from the Bazel build, and installing them in our target-independent resource directory. We then build a simplified "build" of these sources within the `ClangRunner` itself to produce the specific artifacts and layout expected by Clang. It also required fixing our use of Clang on macOS to have a default system root in order to successfully compile or link. It also required cleaning up how the `ClangRunner` used target information more generally -- instead of taking the target as a constructor parameter, it manages its target internally and relies on the Clang target-specifying command line flags. I looked at whether we could split this into another layer separate from the `ClangRunner`, but that proved frustratingly difficult to manage. While we support building these on-demand as part of a detected link, that doesn't seem feasible as we don't have the necessary separation between compilation runs of Clang and link runs of Clang. However, I have tried to factor the internals to provide as clear of separation as I could across these. I have also created a stand-alone subcommand to directly build the runtimes which allows for easy testing. It also supports building them into a specific directory, and that directory can in turn be passed to a Clang invocation. This is designed to work both at the API level with `ClangRunner` and at the subcommand level. Currently, the only part of the commandline that is detected and forwarded to the runtimes build is the target. Eventually, the plan is to expand this so that we can build a maximally tailored set of runtimes for a given compilation. The other big TODO here is to actually implement caching storage of these runtimes so they aren't built on every execution. Right now, this uses a somewhat hack-y build of a temporary directory, but this isn't expected to be suitable long-term. Building these runtimes on *every* link makes those commands take approximately 15 seconds with an ASan build like our default development build, and just over 2 seconds in an optimized build. Because of this, I've kept all of this disabled by default for now. The goal is that once caching and some other improvements land, we can enable this by default. --------- Co-authored-by: Jon Ross-Perkins <jperkins@google.com> Co-authored-by: Richard Smith <richard@metafoo.co.uk>
170 lines
7.1 KiB
C++
170 lines
7.1 KiB
C++
// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
|
|
// Exceptions. See /LICENSE for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
|
|
#ifndef CARBON_TOOLCHAIN_INSTALL_INSTALL_PATHS_H_
|
|
#define CARBON_TOOLCHAIN_INSTALL_INSTALL_PATHS_H_
|
|
|
|
#include <filesystem>
|
|
|
|
#include "common/error.h"
|
|
#include "common/filesystem.h"
|
|
#include "llvm/ADT/SmallString.h"
|
|
#include "llvm/ADT/StringRef.h"
|
|
#include "llvm/ADT/Twine.h"
|
|
#include "toolchain/base/llvm_tools.h"
|
|
|
|
namespace Carbon {
|
|
|
|
// Locates the toolchain installation and provides paths to various components.
|
|
//
|
|
// The Carbon toolchain expects to be installed into some install prefix; see
|
|
// `prefix_` for details. When locating an install, we verify it with
|
|
// `CheckMarkerFile`. When errors occur, `SetError` makes `error()`
|
|
// available for diagnostics and clears the install prefix (leaving things
|
|
// minimally functional).
|
|
//
|
|
// The factory methods locate the install prefix based on their use-case:
|
|
//
|
|
// - `MakeExeRelative` for command line tools in an install.
|
|
// - `MakeForBazelRunfiles` for locating through Bazel's runfile tree.
|
|
// - `Make` for an explicit path, for example in tests.
|
|
//
|
|
// An instance of this class provides methods that query for specific paths
|
|
// within the install. Note that we want to abstract away any platform
|
|
// differences in the installation layout. When a specific part of the install
|
|
// is needed, a dedicated accessor should be added that computes the path for
|
|
// that component.
|
|
//
|
|
// TODO: Need to check the installation structure of LLVM on Windows and figure
|
|
// out what Carbon's should be within a Windows prefix and how much of the
|
|
// structure we can share with the Unix-y layout of the prefix.
|
|
//
|
|
// TODO: InstallPaths is typically called from places using a VFS (both tests
|
|
// and the Driver), but does not use a VFS itself. It currently only supports
|
|
// using the real filesystem, but should probably support a VFS.
|
|
class InstallPaths {
|
|
public:
|
|
// Provide the current executable's path to detect the correct installation
|
|
// prefix path. This assumes the toolchain to be in its installed layout.
|
|
//
|
|
// If detection fails, this reverts to using the current working directory as
|
|
// the install prefix, and the error detected can be checked with `errors()`.
|
|
static auto MakeExeRelative(llvm::StringRef exe_path) -> InstallPaths;
|
|
|
|
// Provide the current executable's path, and use that to detect a Bazel or
|
|
// Bazel-compatible runfiles install prefix path. This should only be used
|
|
// where it is reasonable to rely on this rather than a fixed install location
|
|
// such as for internal development purposes or other Bazel users of the
|
|
// Carbon library.
|
|
//
|
|
// This method of construction also ensures the result is valid. If detection
|
|
// fails for any reason, it will `CARBON_CHECK` fail with the error message.
|
|
static auto MakeForBazelRunfiles(llvm::StringRef exe_path) -> InstallPaths;
|
|
|
|
// Provide an explicit install paths prefix, which must be absolute. This is
|
|
// useful for testing or for using Carbon in an environment with an unusual
|
|
// path to the installed files.
|
|
static auto Make(llvm::StringRef install_prefix) -> InstallPaths;
|
|
|
|
// Returns the contents of the prelude manifest file. This is the list of
|
|
// files that define the prelude, and will always be non-empty on success.
|
|
auto ReadPreludeManifest() const -> ErrorOr<llvm::SmallVector<std::string>>;
|
|
|
|
// Returns the contents of the clang builtin headers manifest file. This is
|
|
// the list of header files that are installed as part of the clang compiler,
|
|
// and will always be non-empty on success.
|
|
auto ReadClangHeadersManifest() const
|
|
-> ErrorOr<llvm::SmallVector<std::string>>;
|
|
|
|
// Check for an error detecting the install paths correctly.
|
|
//
|
|
// A nullopt return means no errors encountered and the paths should work
|
|
// correctly.
|
|
//
|
|
// A string return means there was an error, and details of the error are
|
|
// in the `StringRef` for inclusion in any user report.
|
|
[[nodiscard]] auto error() const -> std::optional<llvm::StringRef> {
|
|
return error_;
|
|
}
|
|
|
|
// The directory containing the `Core` package. Computed on demand.
|
|
auto core_package() const -> std::filesystem::path;
|
|
|
|
// The directory containing LLVM install binaries. Computed on demand.
|
|
auto llvm_install_bin() const -> std::filesystem::path;
|
|
|
|
// The path to `clang`.
|
|
auto clang_path() const -> std::filesystem::path;
|
|
|
|
// The path to `lld' and various aliases of `lld`.
|
|
auto lld_path() const -> std::filesystem::path;
|
|
auto ld_lld_path() const -> std::filesystem::path;
|
|
auto ld64_lld_path() const -> std::filesystem::path;
|
|
|
|
// The path to any of the LLVM tools.
|
|
auto llvm_tool_path(LLVMTool tool) const -> std::filesystem::path;
|
|
|
|
// The path to the Clang resources.
|
|
auto clang_resource_path() const -> std::filesystem::path;
|
|
|
|
// The path to the root of LLVM runtime sources.
|
|
auto llvm_runtime_srcs() const -> std::filesystem::path;
|
|
|
|
private:
|
|
friend class InstallPathsTestPeer;
|
|
|
|
InstallPaths() { SetError("No prefix provided!"); }
|
|
explicit InstallPaths(std::filesystem::path prefix)
|
|
: prefix_(std::move(prefix)) {}
|
|
|
|
static auto MakeFromFile(std::filesystem::path file) -> InstallPaths;
|
|
|
|
// Set an error message on the install paths and reset the prefix to empty,
|
|
// which should use the current working directory.
|
|
auto SetError(llvm::Twine message) -> void;
|
|
|
|
// Check that the install paths have a marker file at
|
|
// `prefix()/lib/carbon/carbon_install.txt". If not, calls `SetError` with the
|
|
// relevant error message.
|
|
auto CheckMarkerFile() -> void;
|
|
|
|
// Read a manifest file.
|
|
auto ReadManifest(std::filesystem::path manifest_path,
|
|
std::filesystem::path manifest_file) const
|
|
-> ErrorOr<llvm::SmallVector<std::string>>;
|
|
|
|
// The computed installation prefix. This will be an absolute path. We keep an
|
|
// absolute path for when the command line uses a relative path
|
|
// (`./bin/carbon`) and the working directory changes after initialization
|
|
// (for example, to Bazel's working directory). In the event of an error, this
|
|
// will be the empty string.
|
|
//
|
|
// When run from bazel (for example, in unit tests or development binaries)
|
|
// this will look like:
|
|
// `bazel-bin/some/bazel/target.runfiles/_main/toolchain/install/prefix_root`
|
|
//
|
|
// When installed, it's expected to be similar to the CMake install prefix:
|
|
//
|
|
// - `C:/Program Files/Carbon` or similar on Windows.
|
|
// - `/usr` or `/usr/local` on Linux and most BSDs.
|
|
// - `/opt/homebrew` or similar on macOS with Homebrew.
|
|
//
|
|
// See https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html
|
|
// for more details. While we don't build the toolchain with CMake, we expect
|
|
// our installation to behave in a similar and compatible way.
|
|
//
|
|
// The hierarchy of files beneath the install prefix can be found in the
|
|
// BUILD's `install_dirs`.
|
|
std::filesystem::path prefix_;
|
|
|
|
// The opened prefix directory, suitable for relative path access.
|
|
Filesystem::Dir prefix_dir_;
|
|
|
|
std::optional<std::string> error_;
|
|
};
|
|
|
|
} // namespace Carbon
|
|
|
|
#endif // CARBON_TOOLCHAIN_INSTALL_INSTALL_PATHS_H_
|