Dana JansensandJon Ross-Perkins
53c98a8619
Add a new instruction called ImplSymbolicWitness which represents a
search for an impl declaration given a self type and an interface to
find implemented for the self type. The self type is stored as a
constant instruction id, rather than as a ConstantId, as instructions
don't currently support holding ConstantId. The interface is stored as a
SpecificInterface but we can't fit all of it directly into the
instruction. So we add a new id to refer to the SpecificInterface as
follows.
Add a new SpecificInterfaceId which indexes into a canonical value store
on SemIR::File. This tracks all `SpecificInterface`s stored in an
instruction - specifically the ImplSymbolicWitness instruction.
The SpecificInterface on Impl is still stored there as a value, not as
an id, and no id is eagerly constructed for it. We wait until an id is
needed to make one. Since they are canonical, a new id is only create
when a new SpecificInterface value is seen.
When doing impl lookup, and the query is not concrete, and the impl is
not effectively final, the query needs to consider future impls that may
specialize either the self type or the constaint to make a more precise
match and replace the found impl declaration. Instead of returning the
ImplWitness instruction from the found impl, we generate a
ImplSymbolicWitness instruction, storing the query so that it can be
replayed later. This instruction is added to the generic eval block and
thus will be re-evaluated later with a SpecificId that may make the
query more concrete. When evaluating the instruction and replaying the
query, the lookup has the same conditions and if it does not decide to
use the found impl concretely, then the same instruction is returned
from eval, leaving it as symbolic.
--- Impl lookup changes ---
Impl lookup gets a little more interesting now. It continues to look in
the facet value for a witness if the self type is a facet value. Then
falls back to looking for an impl declaration. This step is no longer
done directly. Instead, we construct a ImplSymbolicWitness instruction
and evaluate it immediately for each interface that are in the query
facet type.
The ImplSymbolicWitness instruction, when evaluated, calls back to the
impl lookup code, with a query specific interface. There we resume back
into the same code path as from before, finding a witness in an impl
declaration. But we may return "found a non-final impl" instead of a
concrete witness. If eval receives this back, it evaluates to the
current ImplSymbolicWitness instruction as the resulting constant value.
To pass lookup failures back through eval, a result of InstId::None from
the second step of impl lookup will result in a non-constant value,
which is used as a signal back up the stack to the original impl lookup
function that the lookup failed. Using a non-constant value here would
break evaluation of the generic eval block if impl lookup could fail
there, however we know it will not since we only leave behind an
ImplSymbolicWitness instruction in the eval block if we found at least
one matching impl already, and we just want to look for a better match
with a more specific query.
We must take care to not store a reference into any value store across
computation in impl lookup, since impl lookup can recurse into itself
invalidate those stores. That includes the SpecificInterface obtained
from a SpecificInterfaceId, which impl lookup also inserts into the
store.
--- The long tail ---
Adding a new instruction and a new id type requires a myriad of changes
to support them:
We add Dump() support for SpecificInterfaceId. And fix a crash in Dump
for SpecificId::None. We also add MakeSpecificInterfaceId() for dumping
arbitrary ids.
The type of ImplSymbolicWitness is a new singleton builtin type
instruction called WitnessSymbolicType (like WitnessType is the type for
an ImplWitness).
Both ImplSymbolicWitness and WitnessSymbolicType are given `Value` as
their expression category as they are builtin constant values. And
BuildInfo() in TypeCompleter is taught about them both, returning a
`ValueRepr::Copy`.
WitnessSymbolicType is added to the set of SingletonInstKinds, so that
it can have a singleton instrution id as a static member.
Lower's BuildTypeForInst() is taught to make an empty struct for
WitnessSymbolicType, similar to WitnessType.
Instruction formatter (FormatterImpl) grows support for printing a
SpecificInterfaceId so that it can print both arguments of
ImplSymbolicWitness on the RHS when printing the SemIR instruction. To
print a SpecificInterfaceId, it prints both the interface id and the
specific id (if there is one). For example, for a query on a generic
interface `Z` with one parameter, the RHS includes the query, interface,
and specific:
```
%Z.impl_symbolic_witness: <symbolic witness> = impl_symbolic_witness %U, @Z, @Z(%U.as_type) [symbolic]
```
IdKind is extended to include SpecificInterfaceId.
InstFingerprinter is taught to look through SpecificInterfaceId and use
the interface and specific ids in the fingerprint.
InstNamer is taught about SpecificInterfaceId, counting the interfaces
when building an index. It is also tought about ImplSymbolicWitness,
using the name of the interface within and the `.impl_symbolic_witness`
suffix. For example, here the LHS is named after the interface in the
query:
```
%Z.impl_symbolic_witness: <symbolic witness> = impl_symbolic_witness %U, @Z, @Z(%U.as_type) [symbolic]
```
StringifyTypeExpr is taught about WitnessSymbolicType, which uses its IR
name since it's a singleton. And about ImplSymbolicWitness which uses
its constant value. The handling of ImplWitnessAccess also needed to be
adjusted, since it assumed that ImplWitnessAccess::witness_id would
always be a FacetAccessWitness, but it can now also be an
ImplSymbolicWitness. (It seems that the witness_id is also assigned
ImplWitness instructions, but those ImplWitnessAccess instructions don't
ever seem to get stringified in a diagnostic at this time.) At the
moment the ImplWitnessAccess with a symbolic witness is just stringified
as "<symbolic>", such as in:
```
x.carbon:1:2: error: cannot implicitly convert value of type `()` to `<symbolic>` [ConversionFailure]
let a: C(D).(Z.X) = ();
^~
```
There is a TODO left behind to include more information there.
The TypeStructure builder is made to handle WitnessSymbolicType and
WitnessType. These come up now in deduce where a generic impl will have
a ImplSymbolicWitness in a FacetValue for a generic self type. The query
may have a concrete ImplWitness in the same position. Since deduce tries
to deduce through the FacetValue, it tries to convert ImplWitness to
ImplSymbolicWitness, tries to do an impl lookup for `impl ImplWitness as
ImplicitAs(ImplSymbolicWitness)` and causes us to build type structures
with each of these.
Subst is updated to handle pushing and popping SpecificInterfaceId.
Without this, when finishing a generic's eval block, we would walk into
the ImplSymbolicWitness instruction, and its arguments, and fail to
recurse down into the SpecificInterfaceId. Then any specifics inside
would be left as "orphaned" without any generic id attached to them, and
we would never update the instructions in the SpecificInterface's
instructions (inside its own SpecificId) with new constant values when
evaluating the generic eval block against a specific. To do this we push
the specific_id inside the SpecificInterface, and when popping we pop
the specific_id then construct a new canonical SpecificInterface with it
and return that id.
We add support for importing ImplSymbolicWitness by importing its self
constant instruction and specific interface id. However we also had to
add import support for SpecificImplFunction, which can now appear in the
generic eval block for a generic impl declaration, and thus must be
imported with the declaration. This is done very similarly to
SpecificFunction, except the `type_id` is a singleton value.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Carbon Language:
An experimental successor to C++
Why? | Goals | Status | Getting started | Join us
See our announcement video from CppNorth. Note that Carbon is not ready for use.
Fast and works with C++
- Performance matching C++ using LLVM, with low-level access to bits and addresses
- Interoperate with your existing C++ code, from inheritance to templates
- Fast and scalable builds that work with your existing C++ build systems
Modern and evolving
- Solid language foundations that are easy to learn, especially if you have used C++
- Easy, tool-based upgrades between Carbon versions
- Safer fundamentals, and an incremental path towards a memory-safe subset
Welcoming open-source community
- Clear goals and priorities with robust governance
- Community that works to be welcoming, inclusive, and friendly
- Batteries-included approach: compiler, libraries, docs, tools, package manager, and more
Why build Carbon?
C++ remains the dominant programming language for performance-critical software, with massive and growing codebases and investments. However, it is struggling to improve and meet developers' needs, as outlined above, in no small part due to accumulating decades of technical debt. Incrementally improving C++ is extremely difficult, both due to the technical debt itself and challenges with its evolution process. The best way to address these problems is to avoid inheriting the legacy of C or C++ directly, and instead start with solid language foundations like modern generics system, modular code organization, and consistent, simple syntax.
Existing modern languages already provide an excellent developer experience: Go, Swift, Kotlin, Rust, and many more. Developers that can use one of these existing languages should. Unfortunately, the designs of these languages present significant barriers to adoption and migration from C++. These barriers range from changes in the idiomatic design of software to performance overhead.
Carbon is fundamentally a successor language approach, rather than an attempt to incrementally evolve C++. It is designed around interoperability with C++ as well as large-scale adoption and migration for existing C++ codebases and developers. A successor language for C++ requires:
- Performance matching C++, an essential property for our developers.
- Seamless, bidirectional interoperability with C++, such that a library anywhere in an existing C++ stack can adopt Carbon without porting the rest.
- A gentle learning curve with reasonable familiarity for C++ developers.
- Comparable expressivity and support for existing software's design and architecture.
- Scalable migration, with some level of source-to-source translation for idiomatic C++ code.
With this approach, we can build on top of C++'s existing ecosystem, and bring along existing investments, codebases, and developer populations. There are a few languages that have followed this model for other ecosystems, and Carbon aims to fill an analogous role for C++:
- JavaScript → TypeScript
- Java → Kotlin
- C++ → Carbon
Language Goals
We are designing Carbon to support:
- Performance-critical software
- Software and language evolution
- Code that is easy to read, understand, and write
- Practical safety and testing mechanisms
- Fast and scalable development
- Modern OS platforms, hardware architectures, and environments
- Interoperability with and migration from existing C++ code
While many languages share subsets of these goals, what distinguishes Carbon is their combination.
We also have explicit non-goals for Carbon, notably including:
- A stable application binary interface (ABI) for the entire language and library
- Perfect backwards or forwards compatibility
Our detailed goals document fleshes out these ideas and provides a deeper view into our goals for the Carbon project and language.
Project status
Carbon Language is currently an experimental project. We are hard at work on a toolchain implementation with compiler and linker. You can try out the current state at compiler-explorer.com.
We want to better understand whether we can build a language that meets our successor language criteria, and whether the resulting language can gather a critical mass of interest within the larger C++ industry and community.
Currently, we have fleshed out several core aspects of both Carbon the project and the language:
- The strategy of the Carbon Language and project.
- An open-source project structure, governance model, and evolution process.
- Critical and foundational aspects of the language design informed by our
experience with C++ and the most difficult challenges we anticipate. This
includes designs for:
- Generics
- Class types
- Inheritance
- Operator overloading
- Lexical and syntactic structure
- Code organization and modular structure
- A prototype interpreter demo that can both run isolated examples and gives a detailed analysis of the specific semantic model and abstract machine of Carbon. We call this the Carbon Explorer.
- An under-development compiler and toolchain that will compile Carbon (and eventually C++ code as well) into standard executable code. This is where most of our current implementation efforts are directed.
If you're interested in contributing, we're currently focused on developing the Carbon toolchain until it can support Carbon ↔ C++ interop. Beyond that, we plan to continue developing the design and toolchain until we can ship the 0.1 language and support evaluating Carbon in more detail.
You can see our full roadmap for more details.
Carbon and C++
If you're already a C++ developer, Carbon should have a gentle learning curve. It is built out of a consistent set of language constructs that should feel familiar and be easy to read and understand.
C++ code like this:
corresponds to this Carbon code:
You can call Carbon from C++ without overhead and the other way around. This means you migrate a single C++ library to Carbon within an application, or write new Carbon on top of your existing C++ investment. For example:
Read more about C++ interop in Carbon.
Beyond interoperability between Carbon and C++, we're also planning to support migration tools that will mechanically translate idiomatic C++ code into Carbon code to help you switch an existing C++ codebase to Carbon.
Generics
Carbon provides a modern generics system with checked definitions, while still supporting opt-in templates for seamless C++ interop. Checked generics provide several advantages compared to C++ templates:
- Generic definitions are fully type-checked, removing the need to
instantiate to check for errors and giving greater confidence in code.
- Avoids the compile-time cost of re-checking the definition for every instantiation.
- When using a definition-checked generic, usage error messages are clearer, directly showing which requirements are not met.
- Enables automatic, opt-in type erasure and dynamic dispatch without a separate implementation. This can reduce the binary size and enables constructs like heterogeneous containers.
- Strong, checked interfaces mean fewer accidental dependencies on implementation details and a clearer contract for consumers.
Without sacrificing these advantages, Carbon generics support specialization, ensuring it can fully address performance-critical use cases of C++ templates. For more details about Carbon's generics, see their design.
In addition to easy and powerful interop with C++, Carbon templates can be constrained and incrementally migrated to checked generics at a fine granularity and with a smooth evolutionary path.
Memory safety
Safety, and especially memory safety, remains a key challenge for C++ and something a successor language needs to address. Our initial priority and focus is on immediately addressing important, low-hanging fruit in the safety space:
- Tracking uninitialized states better, increased enforcement of initialization, and systematically providing hardening against initialization bugs when desired.
- Designing fundamental APIs and idioms to support dynamic bounds checks in debug and hardened builds.
- Having a default debug build mode that is both cheaper and more comprehensive than existing C++ build modes even when combined with Address Sanitizer.
Once we can migrate code into Carbon, we will have a simplified language with room in the design space to add any necessary annotations or features, and infrastructure like generics to support safer design patterns. Longer term, we will build on this to introduce a safe Carbon subset. This will be a large and complex undertaking, and won't be in the 0.1 design. Meanwhile, we are closely watching and learning from efforts to add memory safe semantics onto C++ such as Rust-inspired lifetime annotations.
Getting started
To try out Carbon immediately in your browser, you can use the toolchain at: carbon.compiler-explorer.com.
We are developing a traditional toolchain for Carbon that can compile and link
programs. However, Carbon is still an early, experimental project, and so we
only have very experimental nightly releases of the Carbon toolchain available
to download, and only on limited platforms. If you are using a recent Ubuntu
Linux or similar (Debian, WSL, etc.), you can try these out by going to our
releases page and
download the latest nightly toolchain tar file:
carbon_toolchain-0.0.0-0.nightly.YYYY.MM.DD.tar.gz. Then you can try it out:
# A variable with the nightly version from yesterday:
VERSION="$(date -d yesterday +0.0.0-0.nightly.%Y.%m.%d)"
# Get the release
wget https://github.com/carbon-language/carbon-lang/releases/download/v${VERSION}/carbon_toolchain-${VERSION}.tar.gz
# Unpack the toolchain:
tar -xvf carbon_toolchain-${VERSION}.tar.gz
# Create a simple Carbon source file:
echo "import Core library \"io\"; fn Run() { Core.Print(42); }" > forty_two.carbon
# Compile to an object file:
./carbon_toolchain-${VERSION}/bin/carbon compile \
--output=forty_two.o forty_two.carbon
# Install minimal system libraries used for linking. Note that installing `gcc`
# or `g++` for compiling C/C++ code with GCC will also be sufficient, these are
# just the specific system libraries Carbon linking still uses.
sudo apt install libgcc-11-dev
# Link to an executable:
./carbon_toolchain-${VERSION}/bin/carbon link \
--output=forty_two forty_two.o
# Run it:
./forty_two
As a reminder, the toolchain is still very early and many things don't yet work. Please hold off on filing lots of bugs: we know many parts of this don't work yet or may not work on all systems. We expect to have releases that are much more robust and reliable that you can try out when we reach our 0.1 milestone.
If you want to build Carbon's toolchain yourself or are thinking about contributing fixes or improvements to Carbon, you'll need to install our build dependencies (Clang, LLD, libc++) and check out the Carbon repository. For example, on Debian or Ubuntu:
# Update apt.
sudo apt update
# Install tools.
sudo apt install \
clang \
libc++-dev \
libc++abi-dev \
lld
# Download Carbon's code.
$ git clone https://github.com/carbon-language/carbon-lang
$ cd carbon-lang
Then you can try out our toolchain which has a very early-stage compiler for Carbon:
# Build and run the toolchain's help to get documentation on the command line.
$ ./scripts/run_bazelisk.py run //toolchain -- help
For complete instructions, including installing dependencies on various different platforms, see our contribution tools documentation.
Learn more about the Carbon project:
Conference talks
Carbon focused talks from the community:
2024
- Generic implementation strategies in Carbon and Clang, LLVM Developers' Meeting (video, slides)
- The Carbon Language: Road to 0.1, NDC {TechTown} (video, slides)
- How designing Carbon with C++ interop taught me about C++ variadics and overloads, CppNorth (video, slides)
- Generic Arity: Definition-Checked Variadics in Carbon, C++Now (video, slides)
- Carbon: An experiment in different tradeoffs, panel session, EuroLLVM (video, slides)
- Carbon's high-level semantic IR lightning talk, EuroLLVM (video)
2023
- Carbon’s Successor Strategy: From C++ interop to memory safety, C++Now (video, slides)
- Definition-Checked Generics, C++Now
- Modernizing Compiler Design for Carbon’s Toolchain, C++Now (video, slides)
2022
- Carbon Language: Syntax and trade-offs, Core C++ (video, slides)
- Carbon Language: An experimental successor to C++, CppNorth (video, slides)
Join us
We'd love to have folks join us and contribute to the project. Carbon is committed to a welcoming and inclusive environment where everyone can contribute.
- Most of Carbon's design discussions occur on Discord.
- To watch for major release announcements, subscribe to our Carbon release post on GitHub and star carbon-lang.
- See our code of conduct and contributing guidelines for information about the Carbon development community.
Contributing
You can also directly:
- Contribute to the language design: feedback on design, new design proposal
- Contribute to the language implementation
- Carbon Toolchain, and project infrastructure
You can check out some
"good first issues",
or join the #contributing-help channel on
Discord. See our full
CONTRIBUTING documentation for more details.