This allows two C++ callers to the same Carbon function, written in
different Carbon files, to generate thunks that won't collide/duplicate
symbol at link time - instead the thunks get deduplicated silently by
the linker.
Carbon gives the C++ declaration of an exported Carbon thunk an asm
label that
includes the target's global symbol prefix (`_` on Mach-O). Clang names
a
declaration with an asm label `\01<label>` in the IR, so that LLVM uses
the
label verbatim as the symbol name, while Carbon lowered the thunk
definition
under its unprefixed name. Both name the same symbol, but they were
distinct
globals in the IR, so LLVM couldn't see that the C++ calls refer to the
Carbon
definition. That only works while the definition is never discarded.
Mark the Carbon thunk as a `CppThunk` (as described in
`toolchain/docs/check/cpp/thunks.md`), record its C++ declaration in
`clang_decls`, and when lowering the thunk, ask Clang for its function
(creating it if necessary) and define the thunk in that function. This
means
that the thunk and the C++ calls to it always use the same IR global,
with
Clang's name and signature.
The thunk takes all of its parameters by reference, so Clang lowers its
parameters the same way Carbon does, but some C++ ABIs (such as Apple
arm64)
return `this` from constructors. In that case, return the parameter that
Clang
marks `returned`.
On targets without a global prefix, the IR names already matched, so the
only
change is that the thunk definitions now carry Clang's parameter and
function
attributes.
This allows a generic class to inherit from a virtual C++ class as long
as the subclass doesn't override any methods. See
toolchain/lower/testdata/interop/cpp/class/import/inherit.carbon for a
working example.
The TODO in `CheckCompleteClassType` is now in
`CarbonExternalASTSource::CompleteType`, and only triggers if virtual
function thunks need to be defined.
A fix in `FileContext::BuildVtable` is also needed: when looking up the
clang decl associated with the vtable, if the class is generic then the
`ClassTemplateSpecializationDecl` associated with the specific is
needed. This required adding `ConstantStore::Get`, since only `const`
access is allowed during lowering.
Adds code to track default values for function parameters in C++ export,
and to export them as global variables to C++ that are attached to
default value expressions in the clang AST.
Modifes lower to export global variables from another block, and adds
the variable initialization to the `global_init` block so their value
gets set at load time.
Diagnostic rendering was part of the data model: `Loc::FormatLocation`
and
`Loc::FormatSnippet` wrote plain text to a stream, so no capability,
color, or
terminal width could reach them.
A diagnostic is now one message with labels, contexts, and location
information
attached, and `Diagnostics::Renderer` draws it into a
`Terminal::Buffer`. The
design is in `toolchain/docs/diagnostics_rendering.md` (#7668),
authoring in
`toolchain/docs/diagnostics.md`.
Before:
```
example.carbon:6:3: error: 1 argument passed to function expecting 0 arguments
Run0(1);
^~~~~~~
example.carbon:3:1: note: calling function declared here
fn Run0() {}
^~~~~~~~~~~
```
After:
```
error: 1 argument passed to function expecting 0 arguments
╭─┤ example.carbon:6:3
┆
3 │ fn Run0() {}
· ─────┬─────
· ╰──┤ calling function declared here
┆
-> 6 │ Run0(1);
· ───────
╰────────────
```
`--no-diagnostic-snippets` gives a compact form, `example.carbon:6:3-9`.
File
tests use it; the rendering tests under `toolchain/diagnostics/testdata`
ask for
snippets.
C++ interop diagnostics are data now rather than text Clang drew:
`SemIR::ClangSourceLocId` names a `clang::SourceRange`, and a Clang
location is
marked as the point it names.
`StreamConsumer` moves to a `:renderer` target that `:emitter` does not
depend
on, keeping `common/terminal` out of `consumer.h`. Capabilities come
from the
file behind the error stream, so tests get plain ASCII.
`renderer_fuzzer` draws diagnostics built from fuzzer bytes with every
capability combination, since a shape the renderer doesn't expect must
never be
why the compiler dies while reporting a problem.
TODOs mark the follow-ups: line weights and crossing lines, wrapping
that keeps
a quoted type whole, and ordering diagnostics that share a byte offset.
Also: `IncorrectExtension`'s hint moves into its message, since its note
had no
location of its own, and `strip_noise` in `check_diagnostics.py` blanks
multi-line `#define`s instead of deleting them.
Assisted-by: Claude Code
A function with a value return form, such as `->? form(val C)`, is
lowered to return the value representation of `C`: a pointer for types
with a pointer value representation. However, lowering of `ReturnExpr`
re-derived the return convention independently of the function's
signature, and assumed that any returned value expression was a
`returned var`, taking the initializing-return path. For types with an
in-place initializing representation, that tried to initialize a
nonexistent return slot and crashed. (This only appeared to work for
types like `i32`, whose by-copy initializing representation happens to
match the value representation.)
Fix this by recording how the function returns its result -- `void`, by
copy, by reference, in place via an `sret` parameter, or as the entry
point's implicit `i32` -- as a `ReturnKind` in `FunctionTypeInfo` and
`FunctionInfo` when the signature is lowered, and having the `Return`
and `ReturnExpr` handlers switch on it. This keeps the signature and the
`return` lowering in agreement by construction, and replaces the
`IsEntryPoint` check in `Return` lowering.
Support for `fn` expressions with the same feature set as `fn`
declarations. In particular, there is no support for captures,
positional parameters, `=>` bodies, or `auto` return types.
Note that this results in two parse nodes for each `=>` token. That's a
bug, but it matches the behavior of function declarations, and will be
fixed in a follow-up change.
Assisted-by: Gemini via Antigravity
This change introduces a new inst kind `CppFunctionPointerType`, which
represents an imported C++ function pointer that can be invoked from
Carbon (support for forming such a pointer from a Carbon function is in
a follow-up PR). This is implemented by treating the operation of
invoking a function pointer in C++ as if it were a call to an `__invoke`
method on the function pointer type, and extending the existing
function-import logic to support importing this fictitious method.
---------
Co-authored-by: Nicholas Bishop <nbishop@nbishop.net>
If a Carbon class overrides virtual functions from a C++ base class but
is never referenced from C++, it is never exported to Clang. During
lowering, `BuildVtable` then fails to find a `CXXRecordDecl` and crashes
when attempting to get the vtable from Clang's code generator.
Ensure dynamic classes with foreign vtables are exported to Clang when
completing the class definition in `CheckCompleteClassType`, and look up
`first_decl_id()` in `BuildVtable`.
Fixes#7721
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Use a single `SemIR::Function` per `Core` interface method, whether it's
generated locally or imported. This prevents generating duplicate
functions, which lead to different types when the witness appears in a
`FacetValue` as part of a specific for a class.
We use a `CanonicalValueStore` of `GeneratedFunction` objects that allow
finding an existing FunctionId for a `Generated` special function before
(re-)generating it. Mangling for `Generated` functions is also moved to
use the values from the `GeneratedFunction`'s canonicalization key, so
that we have a consistent source of truth for the unique ID of a
`Generated` function across all files.
New tests are in
`toolchain/check/testdata/impl/custom_witness/destroy.carbon`.
### Description
When lowering symbols (functions, global variables, and vtables), a new
`SemIR::Mangler` was being created on each call. Each `Mangler` creates
its own `InstFingerprinter` with a fresh store, preventing any
fingerprint computations from being reused across manglings.
This PR caches a single `SemIR::Mangler` instance in `FileContext` so
that the underlying fingerprint cache is preserved and reused across all
symbol manglings within the file.
Assisted-by: Google Deepmind Antigravity
Instead of creating a CodeGenerator per CppDomain, and then crashing in
lowering when we try to consume the same llvm Module multiple times,
create a CodeGenerator for each CppFile within the domain.
For now, we mulitplex all of Clang's ASTConsumer output to all code
generators, which means that any strong external definitions within a
Carbon file (for example, in an inline `Cpp` fragment) will be emitted
to all output files in the same `CppDomain`, resulting in link errors
due to symbol redefinitions. This will be addressed later. But this
should be sufficient for Carbon compilations in which such symbols are
not defined.
We also don't yet attempt to classify which compilations will need C++
code generation, and instead create a clang `CodeGenerator` for every
Carbon file that has C++ imports. For `carbom compile`, only one Carbon
file will need code generation, and yet we still build multiple
`CodeGenerator` objects in general. Fixing this requires more plumbing
from the driver, and this will also be handled in a follow-up.
Assisted-by: Gemini via Antigravity
We can end up emitting the same thunk from multiple compilations in some
cases -- in particular, when the thunk is wrapping a function that is
either synthesized by the compiler or imported from C++. When this
happens, we will have multiple-definition link errors unless we allow
redefinitions across multiple files.
It'd be nice to detect when we need to do this and when we don't, but
that's a bit tricky to do in practice. Ideally, in fact, we would use a
different strategy, and emit the thunks as discardable definitions in
each compilation that *uses* them. But for now emitting them with
weak_odr linkage seems like a good way to make progress.
This helps us move away from the clone-with-modifications approach to
thunking, which gets unwieldy as signatures get more complex.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Change `Lookup` by InstId to return a ClangDecl pointer. All callers
were immediately calling `Get` anyway, so this makes call sites a little
shorter. The other `Lookup` method, by ClangDeclKey, is sometimes called
without calling `Get`, so left that as-is, but renamed to `LookupId`.
Also add a `decl` method to ClangDecl so that the commonly repeated
`clang_decl->key.decl` can be written `clang_decl->decl()`.
Replace all uses of CppGlobalVarStore store with ClangDeclStore.
Adding a VarStorage->VarDecl mapping to ClangDeclStore is now done with
the `AddVar` method, which takes an extra `pattern_id` arg. While the
corresponding `ClangDecl` is unchanged from before, the reverse mapping
in `inst_id_to_clang_decl_id_` now uses the `pattern_id` as the key.
This is necessary because in some places the original VarStorage
instructions gets replaced (e.g. by a call to `Convert`). The
`pattern_id` remains stable in those cases.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Removing the clang_decl_id on SemIR::Function - using only the
clang_decls map to create the association between SemIR::Function and
clang::FunctionDecls.
This adds an `is_external` flag to ClangDecl to indicate whether the
entity originated from Carbon or was imported from another language.
(I'm open to names - I guess for now we mostly use "is this from C++" to
be more specific than "is this external" - eg: NameScope::is_cpp_scope)
Add `ExportVarToCpp`. This checks the `clang_decls` mapping and returns
an existing decl if found. Otherwise, it creates a new `VarDecl` and
adds it to the `clang_decls` mapping.
When lowering, in `FileContext::BuildGlobalVariableDecl`, the
`clang_decls` mapping is used to lookup an existing
`llvm::GlobalVariable` for the instruction. If found, use that rather
than creating a new one to avoid an unwanted second definition in the
llvm IR.
The intent is to add visibility into how the fingerprint is computed, so
that fingerprinting issues and mangling collisions can be more readily
understood and fixed.
Assisted-by: Gemini via Antigravity
This adds the `static` token to the lexer and parses it as a modifier.
In check, `FullPatternStack::Kind::FieldDecl` is now used for both
static and non-static vars. Static vars get treated basically the same
as `NameBindingDecl`s.
Global initialization is used for static var initializers. To make the
necessary stack information available to `pattern_match.cpp`, the
`full_pattern_stack` and `decl_introducer_state_stack` are now popped
later in `handle_let_and_var.cpp`.
In lowering, each class's body is checked for `VarStorage` insts and
lowered the same as global vars.
This correctly renders the vtable in SemIR, including allowing overrides
in
Carbon-derived-from-C++ classes.
It doesn't work in lowering because clang walks the methods of the
CXXRecordDecl - and we currently don't export anything into the
CXXRecordDecl's methods (we do export the fields) - so that's next.
This also doesn't teach Clang to affirmatively emit the vtable
regardless of the types use in C++ code - or to have Carbon use the
vtable in an object's initialization.
https://carbon.compiler-explorer.com/z/88K9Kh5Wo shows the program
exiting with a garbage value copied from uninitialized memory.
This PR modifies `lower` to detect if the function lowered is the
entry point and doesn't specify a return type. If so, it emits
different LLVM IR to return int32 0, and modifies the lowered
function signature to match the int32 return type.
Instead of treating all C++ code as coming from a single synthetic
`CheckIRId`, track the `SemIR::File` associated with each C++ location.
This is necessary since each `SemIR::File` has a distinct `CppFile` and
therefore distinct `SourceLocation`s and `ClangSourceLocId`s.
Assisted-by: Gemini via Antigravity
ImplWitnessAccess can contain a LookupImplWitness instruction as an
operand, which used to be SymbolicOnly but has now become Conditional in
#6915. This opens up the possibility for LookupImplWitness to have a
concrete value, without resolving to a different instruction kind. If
this occurs when it's the operand of an ImplWitnessAccess, and the
access is unable to find a different value to resolve to through the
witness' self type, then the ImplWitnessAccess can also become concrete.
This can happen in particular when:
- You have an ImplWitnessAccess into a `.Self` symbolic in a facet type.
- You convert a concrete type to the facet type.
- The `.Self` is replaced by a concrete type, causing the impl lookup to
be on that concrete type.
- The (now concrete) impl lookup fails to find any impl witness, so it
remains a concrete LookupImplWitness
This results in a diagnostic, as the incoming type does not satisfy the
facet type, but in the meantime we have a concrete ImplWitnessAccess,
which we do not want to crash.
For example in this test:
```carbon
interface I {
let I1:! type;
}
interface J {}
fn F(T:! I where .I1 impls J) {}
fn G() {
class C;
// This identifies `C as (I where .I1 impls J)`, which replaces `.Self.I1`
// with `C.(I.I1)`. This is a concrete lookup since C and I are both concrete,
// but it doesn't find anything as there is no impl.
//
// As such, the call to F fails to deduce a value for T.
F(C);
}
```
This PR splits out the change to ImplWitnessAccess from the larger
change of replacing `.Self` in identified facet types (and comparisons
with identified facet types).
Instead of allowing lower to pick whatever type layout it desires,
compute the layouts of types as part of completing the type, and make
lower build types that match that representation.
For now we assume that all pointers are 64-bit, since we don't have
access to target information. We allow tail padding reuse for structs
and tuple types (and by extension, for classes, since they use structs
as their object representation), but not for arrays.
In order to build matching LLVM types, we create LLVM packed structs
where necessary, and we insert inter-field padding on the end of the
previous field so that GEP indexes still always match Carbon's
ElementIndexes.
We don't yet use the computed alignment much in LLVM IR generation -- in
particular, `alloca`s, `load`s, and `store`s should probably use the
computed type alignment, but don't.
Assisted-by: Gemini via Antigravity
When a function call appears in a generic, and calls another generic
that has a concrete type in its call-site signature, that concrete type
will be completed only in the file that contains the call. The generic
containing the call won't require completeness to be checked again when
forming a specific call, because the type was concrete. This means that
when lowering the call instruction, there is no single file that is
guaranteed to contain complete types for all of the callee's parameters
-- the file containing the specific callee won't necessarily have
completed the concrete parts of the signature, and the files containing
the definition and call won't necessarily have completed the symbolic
parts of the signature.
To handle this, look at both versions of the function when building its
lowered signature -- the version that we saw when forming the `call`
instruction and the version corresponding to the concrete, specific
callee, and combine information from both to form the LLVM function
type.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
This is only fixing the decision about *whether* to produce a witness.
Implementation of the witness is still a TODO, though where a body is
generated, it should also precisely reflect where one _needs_ to be
generated.
Note the tests:
- toolchain/lower/testdata/function/generic/import_core_witness.carbon
- toolchain/lower/testdata/function/generic/import_unused_def.carbon
These tests can probably be produced _without_ Core.Destroy, but I found
the essence of them while trying to build //examples with Core.Destroy
and a simpler minimization wasn't striking me.
Assisted-by: Google Antigravity with Gemini
---------
Co-authored-by: jonmeow <jperkins@google.com>
Assert cleanly if we try to emit a definition or a call of a function
whose signature we were not able to emit exactly. This should make such
issues a lot easier to debug, as we were previously failing in quite
mysterious ways in this case.
Since this requires using the `Mangler` class from `toolchain/check`,
moved it from `toolchain/lower` to `toolchain/sem_ir`.
The mangled name is then attached to the `FunctionDecl` with an
`AsmLabelAttr`.
This enables some nice simplifications, and it's also a step toward a
broader restructuring of binding and parameter patterns.
Assisted-by: Gemini 3.1 Pro via Antigravity
This includes checking and lowering for concrete form literals. Support
for symbolic forms is future work.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This is a step toward removing the index from `InitForm`, so that equal
form values always have equal representations.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This cleans up some logic that was left behind when we stopped emitting
C++ function declarations ourselves. We would ask our mangler for a
mangling for a C++ function declaration and then not use it.
This is needed to model things like the category of `x` in the body of
`fn Foo(F:! Core.Form, x:? F)`, where the category of `x` is determined
by the concrete value of `F` (see #5389 for the design of `:?`
bindings).
This will be used in a follow-up PR.
Don't emit them ourselves. This was leading to our emitted variable
being renamed away from the proper symbol name, leading to link errors.
Fixes#6742.
Add support for compile-time functions. `eval fn` is analogous to C++
`constexpr`, and is evaluated at compile time when it has compile-time
arguments. `musteval fn` is analogous to C++ `consteval`, and requires
that its arguments be available at compile time and is always evaluated
at compile time. For now we require the modifier to match across
redeclarations of the function. The specific modifier syntax here is a
placeholder and not yet part of an approved design.
Limitations: Only very basic support for evaluation is provided. So far
there's no support for mutable state or `if` expressions, but otherwise
control flow and passing and returning values should work. Carbon
evaluation recursion is modeled by C++ recursion for now, so you can
overflow the toolchain stack easily. Functions that use in-place
initialization will generally not work yet, as they are modeled as
passing a non-compile-time-constant reference to a temporary to the
call.
Add missing categorization of `name_binding_decl` as `NotExpr` to match
other similar declaration instructions like `FunctionDecl`, so that we
can uniformly skip over them when they occur within function bodies.
Assisted-by: Gemini 3 Pro and Flash via Antigravity
The primary change in this PR is to split the `Initializing` expression
category into separate `ReprInitializing` and `InPlaceInitializing`
categories, depending on whether initialization uses the types
initializing representation, or is guaranteed to be in place. It also
rationalizes and documents the SemIR-level semantics of those categories
(including where #5545's "ephemeral entire reference" category will
fit), and introduces two new inst kinds to close gaps exposed in the
process.
Some additional secondary changes:
- Consistently format the storage arguments of initializers with `to`,
regardless of whether initialization is in-place, and document the `to`
notation.
- Rename some inst kinds and functions, and restructure some of the
code, for clarity and consistency with the new documentation.
- Resolve a TODO to handle more category conversions in
`CategoryConverter`, in order to make it easier to reason about category
conversions.
See #6588 and the review history of this PR for background.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
When performing an implicit conversion to or from a C++ class type, look
for a C++ implicit conversion, and if that conversion involves a
function call (to a constructor or conversion function), call that
function to perform the conversion.
Note that this is just a first pass at supporting implicit conversions.
There are a lot of other things that can happen in a C++ implicit
conversion, such as aggregate initialization or `std::initializer_list`
initialization that aren't handled here. In addition, we intentionally
leave all standard conversions to Carbon to perform, so that we will
reject conversions such as `i32 -> unsigned` that C++ would select but
Carbon considers to be invalid.
Also support `as` conversions. These are treated analogously, but
perform direct-initialization instead of copy-initialization, so they
also find `explicit` constructors and conversion functions.
In order to give good diagnostics, also track the original C++ source
location for imported C++ functions on the imported version of the
function.
Assisted-by: Gemini 3 Pro via Antigravity
Some module metadata changed - because rather than linking one module
with one module metadata value (eg: PIC Level 0, or unspecified) and one
module with a different one (PIC level 2, in clang) - we use Clang's
Module as-is, no merging required, so Clang's module metadata sticks
rather than being merged with default values from Carbon.
Also tweaked the name we use for Clang's module name so it matches the
carbon file name.
Otherwise the IR changes seem to be just reorderings - C++ interop goes
first, then Carbon, rather than the other way around.
Background:
https://docs.google.com/document/d/1wi85FRiWh4X9A-gCYMVGKR40-q5fM6-3JaSpePk-XCY/edit?usp=sharing
And specifically this work is essentially an alternative to #5543
Clang's code generation is implemented through an ASTListener
(clang::CodeGenerator) that is attached throughout Clang's
parsing/sema/code
generation phases and acts on Clang AST incrementally throughout that
process.
Prior to this patch, Carbon has only created the CodeGenerator during
Carbon's
`lower` phase, missing out on key callbacks that would be made by Clang
during
`check`. Some of these issues were addressed by #6237 and #6483 - but
there were
still remaining cases where the delayed processing lead to missing
functionality.
With #6483 much of the Clang code that made multithreaded complexity of
#5543 is
no longer present, and we have access to the point of ASTListener
registration
so we can register the CodeGenerator there and consume its resulting
llvm::Module during lower.
Examples of some of the bugs this addresses are seen in the linked doc,
and
checked in as tests in this change in
`clang_code_generator_callbacks.carbon`
An indicental bug that's also fixed, and caused all the other test case
churn,
is that the `CodeGenerator` created during `lower` wasn't getting passed
the
Clang `CodeGenOpts` and was creating its own default - so, most notably,
optimization flags were not respected. This meant that the LLVM IR from
Clang
was always -O0 style IR (optnone, no inlinehint, no TBAA, etc). With
this
change, now the Clang IRGen gets the real `CodeGenOpts` and respects
optimization/other flags specified there.
This is only meant to be a rough proof of concept - I'm totally open to
reworking this in any way (even quite substantially) if folks have ideas
about
how this should be implemented most generally/elegantly/etc.
The IdTag knows the type of the Id its tagging and the type of the Id
being used as the tag. This prevents mixing up tagged and untagged ids,
and avoids having to work with untyped integers.
Adds an Untagged marker struct that's used as the tag type in IdTag when
no tag is desired.
The complexity of ConstantIds and TypeIds became a bit visible: TypeIds
are concrete ConstantIds. And ConstantIds have two different tagging
schemes, one for concrete and one for symbolic ids. And ConstantIds are
actually re-cast InstIds with the same index. The LoweredTypeStore needs
to work with tagged TypeIds, but the tags actually come from an InstId
store in ConstantValueStore. Now this is expressed in the type system by
getting the tags for TypeIds from the ConstantValueStore.
ValueStores without an TagId type parameter are now visibly untagged.
IdTag is now only default constructible when it does not have a tag,
which means ValueStore is only default constructible when the TagId is
untagged. This forces tagged value stores to be constructed correctly
with a tag at compile time, and untagged ones to be constructed without.
FixedSizeValueStore has overloads for dealing with tagged and untagged
Ids, since it can't default-construct ValueStore for tagged ids, and no
longer requires passing in default-constructed tags when there is no tag
in the ids.
David Blaikie
Nicholas Bishop
Lucile Rose Nihlen
Chandler Carruth
Richard Smith
Geoff RomerandNicholas Bishop
ATHARVAandDana Jansens
Lucile Rose Nihlen
Jon Ross-Perkinsandjonmeow
