Currently, these lookup the token in its `ValueStore`, and then access
the kind and flag from that. This is near-optimal in the lexer while
_writing_ the tokens, but ends up being considerably slower in parsing
when the overhead of digging the token out of the value store dominates
the actual use case.
This PR does an struct-of-arrays reorganization of the kinds and
whitespace flags, but only after lexing. This lets lexing write a linear
series of info structs, and only afterward re-organize into a
query-optimized representation that is a struct of arrays.
However, to make this a net performance gain, we have to create the
arrays _very_ efficiently. We can avoid hand-written assembly here by
very carefully and precisely writing the inner loop to build these
arrays so that LLVM will vectorize it into high-quality SIMD code for us
across both x86 and Arm. We also need the arrays to have a particular
shape (for example, using a `SmallVector<bool>` rather than a
`BitVector`) in order to preserve this vectorized creation. It burns a
tiny amount of memory, but is key to the whole thing being a net win of
5% to 10% across parsing.
Benchmark results (./scripts/bench_runner.py --runs=12,
-march=x86-64-v3):
- BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Parse>/16384: CPU
Time: -9.045%
- BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Parse>/65536: CPU
Time: -6.526%
- BM_CompileImplFile<Lang::Carbon, Phase::Parse>/16384: CPU Time:
-9.275%
- BM_CompileImplFile<Lang::Carbon, Phase::Parse>/65536: CPU Time:
-8.382%
Assisted-by: Antigravity with both Gemini and Claude
A number of stores have been added to File without updating
CollectMemUsage. Add all the stores that were missed, ordering
everything by the order the fields appear in the class. Also include the
filename_ string, which requires a new overload of Collect, since
strings are like vectors of chars.
For stores with untagged ID types, `Apply` and `Remove` still did (some)
of the work, despite the fact that it could never be needed. Because
both depend on runtime parts of the class, we need to explicitly check
whether the store's IDs are tagged at compile time with `if constexpr`.
This is important because the hottest value stores aren't tagged, and we
get significant performance uplift in lexing and parsing.
Benchmark results (./scripts/bench_runner.py --runs=12,
-march=x86-64-v3):
- BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Lex>/65536: CPU Time:
-4.713%
- BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Parse>/65536: CPU
Time: -7.251%
- BM_CompileImplFile<Lang::Carbon, Phase::Lex>/65536: CPU Time: -8.016%
- BM_CompileImplFile<Lang::Carbon, Phase::Parse>/65536: CPU Time:
-9.477%
Assisted-by: Antigravity with various models
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Extract the allocation path (which is inherently cold due to growth).
Making the outlining effective requires both blocking further inlining
and using `preserve_most` so that the existence of the call doesn't
disrupt the register allocation.
We also mark the insertion statements with callbacks as `always_inline`
so it being cold doesn't cause it to be left out-of-line and further
disrupt register allocation. It's a bit strange to force inlining of a
cold-path function, but the register impact of the call extends back
into the hot-path.
Agy's benchmarks (`./scripts/bench_runner.py --runs=12`, `-c opt
--copt=-march=x86-64-v3`, vs. `trunk`):
- `BM_ValidIdentifiers<3, 5, true>`: -6.277% CPU time
- `BM_ValidIdentifiers<3, 16, true>`: -4.525% CPU time
- `BM_CompileApiFileDenseDecls<Lang::Carbon, Phase::Lex>/262144`:
-2.012% CPU time (47.5 ms ± 6.358% -> 46.54 ms ± 1.773%, p=0.0167)
Assisted-by: Antigravity with Gemini
Add support to the driver for CLI flags that select which CPU to target,
which CPU to tune performance for, and which CPU features to enable.
- `--target-cpu` selects the architecture of CPU to target, similar to
`-march` on x86. This combines `-march` with `armv*` strings and `-mcpu`
on Arm for specific CPU names. Similarly for RISC-V (if/when we support
and test it), etc. Notably, `--target-cpu=native` targets the current
native CPU.
- `--target-cpu-tune` maps to `-mtune`. This is expected to be rarely if
ever desirable, and I'm open to removing it, but if folks are relying on
this for performance I didn't want to hide the option.
- `--target-cpu-features` allows a "standard" (for LLVM) comma-separated
list of LLVM subtarget features, including `+feature` and `-feature`
syntax.
These flags, when provided, establish the default baseline for CPU
targeting and control how on-demand runtimes are built. They can be
changed or overridden by additional Clang flags when using
`--clang-arg`.
A critical part of this is building runtimes on demand using these
features. For that, we need to do some amount of canonicalization
(re-using Clang's canonicalization to ensure consistent behavior), and
extend the runtimes cache features to recognize these customizations.
Note that only the top-level Carbon flags impact runtimes, any
particular Clang arguments are only used by the specific compiles that
receive those arguments.
There is some nascent RISC-V support here, but it is incomplete and only
minimally tested. Still, it seemed better to record the parts we could
test than remove them.
Assisted-by: Antigravity with Gemini
Noticed these issues when trying to query raw semir with
[yq](https://github.com/mikefarah/yq)
- Removed additional "}" within PrintClassFields
- Removed additional "}" after is_frozen_period_self
- Surround `int(value: 123)` in quotes to avoid issues parsing the
additional ':'
- Updated DeclaredFacetType::Print to have valid YAML structure
- Constraints are added to inline list (i.e. wrapping in `[ ... ]`)
- Switched Outer seperator to Comma.
The last two may be up for debate, maybe there's no intention for raw
semir to be true valid yaml but I think its quite useful, for example I
can now do the following:
```sh
carbon compile examples/sieve.carbon --phase=check --dump-raw-sem-ir | yq '
.sem_ir |
select(.names as $n | .classes[] | .name as $k | $n[$k] == "Sieve") |
. as $ir |
$ir.classes[] |
select(.name as $k | $ir.names[$k] == "Sieve" and .body_block_id != "inst_block_empty") |
. as $c |
($ir.inst_blocks[$c.body_block_id][] as $i | $ir.insts[$i])
'
```
outputs:
```yaml
{kind: ImplDecl, arg0: impl41000000, arg1: inst_block41000008}
{kind: ImplSelfWitness, arg0: inst41000014, arg1: specific_interface41000000, type: type(inst(WitnessType))}
{kind: FunctionDecl, arg0: function41000000, arg1: inst_block41000013, type: type(inst41000030)}
{kind: FunctionDecl, arg0: function41000001, arg1: inst_block41000023, type: type(inst4100005B)}
{kind: FieldDecl, arg0: field41000000, arg1: region41000002, type: type(inst41000065)}
{kind: CompleteTypeWitness, arg0: inst41000067, type: type(inst(WitnessType))}
```
This change introduces a new inst kind `CppFunctionPointerType`, which
represents an imported C++ function pointer that can be invoked from
Carbon (support for forming such a pointer from a Carbon function is in
a follow-up PR). This is implemented by treating the operation of
invoking a function pointer in C++ as if it were a call to an `__invoke`
method on the function pointer type, and extending the existing
function-import logic to support importing this fictitious method.
---------
Co-authored-by: Nicholas Bishop <nbishop@nbishop.net>
Check that every use of a non-constant instruction is dominated by a
definition of that instruction. Remove the fake (instruction creation
order based) dominance checks in convert; these start spuriously failing
during template instantiation of initializers.
Assisted-by: Claude Opus 5 via Antigravity
The `Support/LLVMDriver.h` header was renamed to `Driver.h`, and the API
of `llvm::ToolContext` changed such that a constructor call is needed
instead of aggregate initialization.
A couple new bazel deps were added (libxml2 and xz), and rules_cc
required an updated version. The addition of libxml2 and xz required
some additions to check_non_test_cc_deps.py. libxml2 also required
allow-listing an additional URL in CI jobs.
Dropped `0011_Temporarily_remove_reference_to_hermetic_toolchain.patch`,
no longer needed.
Added `0012_Drop_dependencies_on_linux_uapi_and_pyyaml.patch` to remove
a couple dependencies from llvm that Carbon doesn't currently require.
These were causing bazel queries in check_deps and
forbid_llvm_googletest.py to fail.
The StringRef represents a bounded region of a string, but using data()
drops the end bound, and makes LLVM construct a new StringRef starting
in the same position and going until a nul terminator. If this worked
correctly before, it was because the StringRef was always pointing to a
full `std::string` or the tail of one.
- Move formatted printing to `hashing.cpp` instead of `hashing.h`
- Separate APInt and APFloat hashing specializations into a new
`hashing_llvm.h`
- Update toolchain/base dependencies and include sites that hash LLVM
data types to include `hashing_llvm.h`
Combined, this reduces the transitive includes caused by `hashing.h`.
Assisted-by: Antigravity with Gemini
Addresses part of issue raised in #7159 by implementing float.negate
builtin for FloatLiteralValues
The following code now compiles:
```
let a: f64 = -1.0;
```
To achieve this I switch the mantissa from being unsigned to signed.
Lexed literals within source file will still always be unsigned, however
it is now possible to create negative FloatLiteralValue constants. Main
non-local changes this causes is:
- All llvm::APInt parsing / printing calls flipped isSigned param
- Zero extension replaced with sign extension
- getActiveBits() replaced with getSignificantBits() for min bit width
calculation
This isolates the C++ imports in different Carbon files from each other
in `--share-cpp-ast` mode, so that a Carbon file can only see the
portions of the shared Clang `ASTContext` that it actually imported.
Assisted-by: Gemini via Antigravity
Refactors the link driver to automatically compile and cache the carbon
prelude for use in linking.
Implements a `carbon_library` rule for compiling the Core library
dependencies in the examples.
Replace the llvm::map_range based anonymous ranges returned by values()
and enumerate() with custom range and iterator types. This avoids
exposing complex template return types and improves encapsulation. But
most importantly, it is *much* cheaper to compile.
This change:
- Defines ValueStoreIterator, ValueStoreEnumerateIterator, and
ValueStoreRange templates in the Internal namespace.
- Exposes them via aliases in ValueStore: Iterator, ConstIterator,
Range, MutableRange, and EnumerateRange.
- Supports safe implicit conversion from Iterator to ConstIterator.
- Supports operator-> even when the value type is returned by value
(e.g., llvm::StringRef) by conditionally const-qualifying the pointer
type.
- Ensures C++20 comparison consistency with custom operator<=> and
operator== definitions.
- Restricts construction of these iterators and ranges to ValueStore
methods by using private constructors and friend declarations.
Assisted-by: Antigravity with Gemini
One API fix: BumpPtrAllocator no longer tracks the amount of memory it's
handed out separately from the amount of memory it has allocated from
the system.
Assisted-by: Gemini via Antigravity
Adds `toolchain/benchmarking/prelude_benchmark.cpp`, which measures the
time to compile the Core prelude and reports the most interesting SemIR
memory statistics as benchmark counters.
The prelude is compiled by checking a file that imports it: the implicit
prelude import causes the check phase to lex, parse, and check the full
set of prelude files, so this is a direct measure of prelude compilation
cost. Four input variations exercise increasing amounts of the prelude:
an empty file, a minimal single-type use, an operator-heavy file that
hits many impls, and a compact file that pulls in a wide swath of the
prelude.
Memory usage is queried directly: `Driver::set_mem_usage` takes a
`MemUsage` that a compile merges each file's usage into; the benchmark
passes one, compiles, and sums the entries by label. A compilation unit
collects into its own `MemUsage` whenever usage is dumped or a sink is
provided (decided in `SetMultiUnitCache`); after a file is done it dumps
that `MemUsage` per-file as before and, if a sink was provided, merges
into it via a new `MemUsage::Add(const MemUsage&)` overload. `MemUsage`
also exposes its entries via a public `Entry` type and an `entries()`
accessor.
Also extends `scripts/bench_runner.py` to (1) treat Mem-prefixed
counters as cost metrics (smaller is better) and (2) tolerate metrics
that aren't reported by every benchmark in a binary.
Assisted-by: Claude Code
---------
Co-authored-by: Christopher Di Bella <cjdb.ns@gmail.com>
These were defined inline in headers reached by most of the toolchain,
and as non-template functions their bodies -- including some very
expensive template instantiations -- are compiled in every including TU
even though they are only cold debug/dump paths:
- BundleStore's YAML output entry points instantiate IdAndKind::Dispatch
over all ~50 ID kinds (two ~365ms instantiations per TU) plus
Yaml::OutputScalar/std::function machinery; moved to a new bundle.cpp.
- TypeStore, ConstantValueStore, NameStoreWrapper, and SharedValueStores
OutputYaml bodies wrap capturing lambdas in std::function via
Yaml::OutputMapping; moved to their existing .cpp files.
This change appears to be worth another 10% compile time reduction.
Assisted-by: Claude
As part of this, move functions that seem reasonable to make out-of-line
to a separate `_impl.h` header file that is only included where the
explicit instantiation _definition_ is provided.
By using explicit instantiation we can make these templates behave more
like non-template classes in terms of supporting out-of-line definitions
that don't need to be compiled by every translation unit. The set of
eventual instantiations here is fundamentally known, and there tend to
be headers that define a canonical "leaf" type where it makes sense to
trigger the explicit instantiation.
Where we already had a `.cpp` file to put the explicit instantiation
definition, use it. But in some places we didn't have such a `.cpp` file
so this PR adds those.
This also requires that we have precise constraints on APIs that _can't_
be instantiated for specific argument types, as now we don't do this
lazily.
Combined, this appears to reduce the sum of object file sizes in the
`check` directory by almost 40% (122mb -> 74mb) in my measurement.
My actual goal was to improve compile times, but so far I don't have a
great methodology for measuring these... But the object file size
reduction seems to confirm this is a net win and likely represents a
non-trivial improvement in compile time.
Assisted-by: Antigravity with Gemini
This enables searching facets like `T:! Z where .Z1 impls Y` for queries
like `T.Z1 as Y`, etc.
Any facet in the query self or the query target type may provide a
witness for the requirements of the impl lookup, so search through them
all.
We accept partially identified facet types in the query self, since
`Self` can be used inside a named constraint before it's fully
identified. But any use of `Self` being converted would put it in the
query self, not the query target, which would be some generic parameter
facet type that a type involving `Self` is being converted to.
TypeIterator is expanded to support this use case, by giving it the
ability to walk through FacetType's extend/impls constraints. We return
the facet values that we find in the iterator instead of a type id. And
we also include FacetValue instructions as an iterator step for clients
that want them, while also recursing through them.
This gets us back to being mostly clang-tidy clean. This turns out to be
important for agentic coding agents, which otherwise sometimes try to
"fix" these false-positive lints.
Assisted-by: Gemini via Antigravity
This also switches to a more Bazel-based install layout, skipping the
FHS-based synthetic layout. The FHS-based layout is still reconstructed
explicitly when building an installable tar-ball.
The biggest change is to configure the just-built install as a Bazel
toolchain, including allowing it to build its own runtime libraries as
native Bazel libraries. This removes the need for a monolithic runtimes
build, all of that code logic is removed.
This should also pave the way to using the just-built toolchain for
doing a full 3-stage bootstrap. Building the 2nd stage is included here
as it was a particularly effective way to test that the Bazel
integration was fully working. Adding a 3rd-stage check for stability is
future work, but should be pretty easy.
There is a down-side: this uses the busybox to do the runtimes
compilation, which means they will be re-built after ~any change to
Carbon. However, the integration with Bazel should largely pay for this,
and we can continue to factor the tests away from depending on built
runtimes in most cases.
Now that we're building and testing the runtimes more directly, this
surfaced a problem with the layout of runtimes on macOS that is fixed
here. All of the Darwin OSes use a custom layout for their resource
directory compared to other targets. We now model this in both the C++
built runtimes and the Bazel built runtimes.
Assisted-by: Gemini via Antigravity
## Summary
`ValueStore::GetRawIndex` formatted the first `CARBON_DCHECK` with
`index` before the local `index` is declared. Use `id.index` so the
diagnostic matches the condition being checked.
## Test plan
- `bazelisk build //toolchain/base:base` (or `//toolchain/...` as
appropriate)
Use parens to delay macro expansion to address the comma separator case,
allowing reuse in AnyBindingOrExportDecl. Also add
CARBON_INST_CATEGORY_ANY_EXPAND to reduce some boilerplate.
Assisted-by: Google Antigravity with Gemini
This shifts the Bazel toolchain configuration of our installation to
build all of the Clang runtimes Carbon uses on-demand natively in Bazel.
We export the information about how to build into a generated Starlark
file, and emit BUILD files and Starlark logic into the installation to
orchestrate the build.
This requires some complex management of Bazel toolchains -- we need to
first set-up a "runtimes toolchain" that doesn't have runtimes of its
own, but can be used to _build_ runtimes. Then we build the runtimes
using that toolchain, and assemble them into the standard layout for a
Carbon runtimes tree. Finally we configure the _actual_ toolchain with
this built tree.
Currently, this is only setup for the installed toolchain, but I plan to
factor this runtimes build into one that can be used directly as well to
break up the monolithic runtimes build step into Bazel-integrated build
of the runtimes. This will also serve as the foundation for adding
bootstrapping support directly to our Bazel build.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
This moves the most complex of the logic fully into Starlark: both the
many different platform sources list, and the overriding of generic
files with architecture specific files.
This also fixes significant bugs in the AArch64 build where we were
skipping numerous files: all of the outlined atomics and `emupac.cpp`.
This PR forcibly disables `emupac.cpp` as fixing that will require a
more significant change.
The Carbon style guide prefers `const` to be on the left wherever
possible, and also has a de-facto standard for specifier order. Since
the order of specifiers and qualifiers tends to become a part of
muscle-memory, deferring the checking of this to tooling should lift a
small burden on both contributors and reviewers.
---------
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
This uses the `CARBON_KIND_ANY(AnyImportRef, auto import_ref):` syntax
that seemed to be favored [on
Discord](https://discord.com/channels/655572317891461132/655578254970716160/1478486848207720478).
This converted uses in the `sem_ir` directory to show it works
initially, then added `check` for full coverage plus validating the
`SemIR::` namespace discard.
Note in inst_namer.cpp, AnyBindingPattern includes FormBindingPattern
which wasn't previously handled.
I'm disabling clang-format because I think it formats with readability
issues, e.g.:
```
#define CARBON_KIND_ANY_EXPAND_AnyBinding(X, SEP) \
X(::Carbon::SemIR::AliasBinding) \
SEP X(::Carbon::SemIR::FormBinding) SEP X(::Carbon::SemIR::RefBinding) \
SEP X(::Carbon::SemIR::SymbolicBinding) \
SEP X(::Carbon::SemIR::ValueBinding)
```
Since `SEP` is typically a comma, it's also a nuisance to treat as an
argument to `X` (which could get better results).
Assisted-by: Google Antigravity with Gemini 3 Flash
This isn't as interesting as others, as it only involves compile
options.
It also adds a missing flag of `-fno-lto` as these objects can't be
LTO-ed.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The goal here is to be able to construct a build of the runtimes
directly in Bazel, or by emitting `BUILD` files, or by emitting into C++
code and using that on-demand. For that, we want a single source of
truth, and that source in Starlark.
This should also make the information more generally useful, and so I'm
moving as much as I can into the LLVM Bazel build. Apologies as that
makes the diffs extra annoying.
I do plan on upstreaming the Bazel parts of this, but would like to get
everything working in Carbon and stabilized first.
While here, I've also made a change suggested for the future in the
initial review by lifting the C++ template out of a string literal in
the `.bzl` file, and into an actual separate C++ file.
This only moves libc++, libc++abi, and libunwind. I want to get those
three working end-to-end before I work on the builtins or `crtbegin` and
`crtend`, as those have a bunch of additional complexity.
This also only uses the info in the C++ on-demand build. It seemed like
a reasonable increment to start code review, and my plan is to work on
other build strategies in a follow-up PR. If that doesn't work, let me
know and I'll come back once I have at least a second use of the info
here.
Turn a few section options on by default in Clang's options, and
propagate the setting from Clang to Carbon. These settings can't be
different between the two sides of the compilation, so merging the
behavior of Carbon's defaults and Clang's flags seems best.
Mainly because "sorting_diagnostic_consumer" is legacy, since
`SortingDiagnosticConsumer` became `SortingConsumer`. Also better
reflecting contents of these files.
Where I'm not renaming, I'm less positive about dropping "diagnostics"
from "file_diagnostics" and "null_diagnostics" (which contain both a
consumer and emitter, and "null.h" seems like poor naming), so not doing
that here. Also "diagnostic.h" contains `struct Diagnostic`, so is a
decent fit.
Assisted-by: Google Antigravity with Gemini 3 Flash
Most versions are through `pre-commit autoupdate --freeze`, clang-format
was manually updated to the latest at
https://github.com/ssciwr/clang-format-wheel
My read of the style changes here are that they seem fine, none of them
look like regressions (which has caused me to delay/adjust updates in
the past).
This makes it easy to wire up build systems like Bazel that need to know
the actual include paths used. It also gives us a convenient place to
export any other information that build systems or integrations need,
and to get debugging info from users.
Most of the complexity is computing the Clang header search paths, but
I couldn't see a direct way to get closer to the source-of-truth than
this, and it doesn't seem _too_ unreasonable.
Depends on #6636 - start review at commit
[643fdab1](6637/commits/643fdab1)
This brings some fixes:
- The handling of `zlib` and `zstd` are much cleaner
- Three of our patches are no longer needed
This also includes the fixes from #6562
It also moves us from `zlib` to `zlib-ng` which is a much better basis
for what we want, and likely makes our toolchain faster when generating
debug info at least.
It fixes another API change in terms of which headers provide the
`createInvocation` we use.
Lastly, it cleans up the deps test to correctly recognize the wrappers
for `zlib-ng` and `zstd`, as well as improving the documentation for why
we allow dependencies on them.
- Distinguish attached vs. unattached constants.
- Add some missing value stores to the top-level output.
- Add missing fields to various Print methods.
Background:
https://docs.google.com/document/d/1wi85FRiWh4X9A-gCYMVGKR40-q5fM6-3JaSpePk-XCY/edit?usp=sharing
And specifically this work is essentially an alternative to #5543
Clang's code generation is implemented through an ASTListener
(clang::CodeGenerator) that is attached throughout Clang's
parsing/sema/code
generation phases and acts on Clang AST incrementally throughout that
process.
Prior to this patch, Carbon has only created the CodeGenerator during
Carbon's
`lower` phase, missing out on key callbacks that would be made by Clang
during
`check`. Some of these issues were addressed by #6237 and #6483 - but
there were
still remaining cases where the delayed processing lead to missing
functionality.
With #6483 much of the Clang code that made multithreaded complexity of
#5543 is
no longer present, and we have access to the point of ASTListener
registration
so we can register the CodeGenerator there and consume its resulting
llvm::Module during lower.
Examples of some of the bugs this addresses are seen in the linked doc,
and
checked in as tests in this change in
`clang_code_generator_callbacks.carbon`
An indicental bug that's also fixed, and caused all the other test case
churn,
is that the `CodeGenerator` created during `lower` wasn't getting passed
the
Clang `CodeGenOpts` and was creating its own default - so, most notably,
optimization flags were not respected. This meant that the LLVM IR from
Clang
was always -O0 style IR (optnone, no inlinehint, no TBAA, etc). With
this
change, now the Clang IRGen gets the real `CodeGenOpts` and respects
optimization/other flags specified there.
This is only meant to be a rough proof of concept - I'm totally open to
reworking this in any way (even quite substantially) if folks have ideas
about
how this should be implemented most generally/elegantly/etc.
The IdTag knows the type of the Id its tagging and the type of the Id
being used as the tag. This prevents mixing up tagged and untagged ids,
and avoids having to work with untyped integers.
Adds an Untagged marker struct that's used as the tag type in IdTag when
no tag is desired.
The complexity of ConstantIds and TypeIds became a bit visible: TypeIds
are concrete ConstantIds. And ConstantIds have two different tagging
schemes, one for concrete and one for symbolic ids. And ConstantIds are
actually re-cast InstIds with the same index. The LoweredTypeStore needs
to work with tagged TypeIds, but the tags actually come from an InstId
store in ConstantValueStore. Now this is expressed in the type system by
getting the tags for TypeIds from the ConstantValueStore.
ValueStores without an TagId type parameter are now visibly untagged.
IdTag is now only default constructible when it does not have a tag,
which means ValueStore is only default constructible when the TagId is
untagged. This forces tagged value stores to be constructed correctly
with a tag at compile time, and untagged ones to be constructed without.
FixedSizeValueStore has overloads for dealing with tagged and untagged
Ids, since it can't default-construct ValueStore for tagged ids, and no
longer requires passing in default-constructed tags when there is no tag
in the ids.
Previously, we used the FHS "prefix" concept as the basis of the
install, but this makes it hard to integrate an installed toolchain with
Bazel (or similar) build system where it wants the "root" of the
toolchain to have some specific files (`MODULES.bazel` or
`BUILD.bazel`), and cannot reference anything outside that directory
tree.
An easy solution is to make the `lib/carbon` directory the root of the
install and never walking up from it. Then we simply have a `bin/carbon`
symlink to the busybox that is useful for getting the command into the
PATH, but isn't used for anything else. The FHS-constrained install
paths surround a root we fully control the layout and files within.
While initially motivated by trying to make a single toolchain structure
that works both for installation and for Bazel, it actually makes the
paths we end up using in the toolchain much simpler. We no longer have
awkward `.../lib/carbon/../../lib/carbon/...` sequences in the toolchain
which is cleaner and even a (trivial) efficiency gain.
As I was doing this I noticed several out-of-date comments that I tried
to fix, and I tried to improve some code reuse rather than re-computing
paths.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Also consolidate on using `//bazel/cc_rules:defs.bzl` where appropriate.
Also update a couple of Bazel modules deps of `@rules_cc` to the latest
versions.
This enables on-demand building of runtimes by default, and enables
their header files for all of the Clang invocations. This also switches
the default flags to use the LLVM-provided runtimes (compiler-rt,
libunwind, and libcxx).
This also switches even `llvm_symlinks_test` to use the Bazel prebuilt
runtimes, which requires having a way to pass a Carbon flag even when
invoking the busybox as `clang` or `clang++`. This uses the pattern that
has worked for other Clang wrappers of spelling flags:
`-X<tool-name>=--flag=value`
Last but not least, this updates the Carbon Bazel rules to use our
installed and the Bazel prebuilt runtimes. With that, we make the C++
interop hello-world be enabled by default as this should pass reliably
on both Linux and macOS now.
This unifies the default Clang arguments between the `clang` subcommand,
the `link` subcommand, and the `ClangInvocation` built for C++ interop.
This sets the stage to integrate either pre-built or on-demand runtimes
flags for both of these. However, this PR should have very little
practical difference. The biggest functional change is wrapping the
default arguments in flags to allow unused flags so that we can build a
collection of flags viable across compile and link.