Fix a few API issues. There's also a newly-added file in compiler-rt
that is not supposed to be built by default but is not being excluded
properly by a glob. Added a patch to exclude that and sent
https://github.com/llvm/llvm-project/pull/208861 upstream.
Instead of manually creating a map from symbolic types to concrete
types, create a Specific and look up parameter types via that Specific.
This allows C++ to call a Carbon function like `fn F[T: type](unused t:
T*) {}`. See generic_pointer.carbon.
This allows C++ to call Carbon functions with generic type parameters,
with some conditions. Example:
```carbon
interface I {
fn Doit(self);
}
class A {
impl as I { fn Doit(unused self) {} }
}
class B {
impl as I { fn Doit(unused self) {} }
}
fn F[T:! I](t: T) {
t.Doit();
}
inline Cpp '''
void G() {
Carbon::A a;
Carbon::B b;
Carbon::F(a);
Carbon::F(b);
}
''';
```
The initial support is limited; only explicit parameters are handled
currently.
`CarbonExternalASTSource::GetOrExportFunctionToCpp` now generates a
`clang::FunctionTemplateDecl` for generic Carbon functions. If C++ code
attempts to call that templated function,
`CarbonExternalASTSource::LoadExternalSpecializations` will be called
with the template argument types of that call site. Then we can generate
a specialized thunk for those argument types for C++ to call.
Instead of maintaining a stack of pending subpatterns which might or
might not contain expressions, we mark non-nesting regions during
pattern handling that might contain an expression. The implementation
remains largely the same; the difference is that callers are expected to
end a pending expression region as soon as possible, rather than wait
for the end of the subpattern. This makes it possible to emit
non-pattern insts during pattern handling, without the risk that they
will get caught in a pending expression region further up the stack.
The function to set the visible declarations with a given name
overwrites any existing declarations imported from an AST file, so we
need to avoid calling that for declaration contexts whose names are
managed by Clang to avoid clobbering names imported from modules.
Assisted-by: Gemini via Antigravity
Copy MultiplexExternalSemaSource.h and MultiplexExternalSemaSource.cpp
from https://github.com/llvm/llvm-project/pull/204458 into
third_party/llvm, and apply a few minor changes to allow them to compile
and pass precommit checks.
This allows
`0011-Add-empty-constructor-and-GetSources-method-to-Multi.patch` to be
removed, which brings Carbon closer to being able to compile on an
unmodified LLVM toolchain.
If C++ overload resolution selects a builtin operator candidate for an
enum comparison or bitwise operator, provide support for that operator
by generating a corresponding Carbon builtin function. This is
structured to be easily extensible to other C++ builtin overload
candidates if we so choose, but for now the operators defined in the
prelude are doing what we want in most cases.
Bitwise operators on enums produce the same enum type as a result. This
intentionally deviates from C++, where they produce a promoted integral
type.
Assisted-by: Gemini via Antigravity
In generate_ast.cpp, an `CarbonExternalASTSource` is installed that has
a `Check::Context` pointer. During lowering, this `ExternalASTSource` is
still installed, and using it can cause a crash if the now-invalid
pointer is dereferenced.
Fix by adding a new `ReadOnlyASTSource` in sem_ir, and using that during
lowering.
`CarbonExternalASTSource` now inherits from `ReadOnlyASTSource` to avoid
some code duplication.
In generate_ast.cpp, we now always install a multiplex source, even if
there's only one child source. Clang internally keeps pointers to the
top-level `ExternalASTSource` installed via `setExternalSource`, and
those pointers aren't updated if `setExternalSource` is called again. By
using `MultiplexExternalSemaSource`, we can keep the top-level
`ExternalASTSource` pointer the same, and only update its children.
Using `MultiplexExternalSemaSource` this way requires a new constructor
and a method to modify its child sources; added a new LLVM patch adding
those.
Originally landed in #7335, reverted in #7353 due to ASAN errors.
Changes since original:
* Use LLVM RTTI to make `Lower::Context::Finalize` less brittle.
Add LLVM RTTI to `ReadOnlyASTSource` (and `CarbonExternalASTSource`).
Change Finalize so that instead of just deleting the last multiplex
child source, it erases any multiplex child sources that match
`ReadOnlyASTSource`; this includes `CarbonExternalASTSource` since it's
a subclass.
* Fix ASAN error by updating the `MultiplexExternalSemaSource` earlier
in lowering. It is sometimes accessed during PrepareToLower, so update
it in `Context::GetFileContext` rather than `Context::Finalize`.
Fixes https://github.com/carbon-language/carbon-lang/issues/7142
In generate_ast.cpp, an `CarbonExternalASTSource` is installed that has
a `Check::Context` pointer. During lowering, this `ExternalASTSource` is
still installed, and using it can cause a crash if the now-invalid
pointer is dereferenced.
Fix by adding a new `ReadOnlyASTSource` in sem_ir, and using that during
lowering.
`CarbonExternalASTSource` now inherits from `ReadOnlyASTSource` to avoid
some code duplication.
In generate_ast.cpp, we now always install a multiplex source, even if
there's only one child source. Clang internally keeps pointers to the
top-level `ExternalASTSource` installed via `setExternalSource`, and
those pointers aren't updated if `setExternalSource` is called again. By
using `MultiplexExternalSemaSource`, we can keep the top-level
`ExternalASTSource` pointer the same, and only update its children.
Using `MultiplexExternalSemaSource` this way requires a new constructor
and a method to modify its child sources; added a new LLVM patch adding
those.
https://github.com/carbon-language/carbon-lang/issues/7142
Because we now support calling a function with a `self` parameter
directly, we can unconditionally call `operator$(lhs, rhs)` rather than
calling `lhs.operator$(rhs)` if the selected operator function happens
to be a member function.
This makes the logic a bit simpler and the SemIR a bit smaller.
We can't do the same for Carbon operators, unfortunately, as we use the
member access to trigger impl lookup.
Implements proposal #7016: `self` moves from the deduced implicit list
(`fn F[self: Self]()`) to the front of the explicit list. Its type may
be written explicitly (`fn F(self: Self)`) or omitted, in which case it
defaults to `Self` (`fn F(self)`, `fn F(ref self)`); `self` in the
implicit list is rejected.
Throughout checking, `self` is modeled as the first explicit parameter.
Because a method is just a function whose first parameter is `self`, it
can also be called as an ordinary function with the receiver passed
explicitly (`Type.M(obj, ...)`), not only as `obj.M(...)`. A new
`SemIR::CallArgParamPatterns` helper chooses the parameters matched
against the explicit arguments, excluding a leading `self` only when it
is supplied as a method-call receiver; arity checking, conversion, and
generic deduction use it. The resulting SemIR and lowering are
unchanged: `self` is still `call_param0`, and witnesses, thunks, and
vtables are unaffected.
An omitted `self` type is parsed as a `SelfBindingPattern` node with no
type expression; checking synthesizes the `Self` type so it behaves
exactly like `self: Self`. However, the exact spelling used must match
between a forward declaration and a definition, following #3763's rules
around declaration matching.
Generated functions, thunks, and C++ interop import/export build `self`
as the first explicit parameter, and the `self`-type override (e.g.
Derived->Base for a virtual override) applies to the explicit `self`.
Placement is validated by new diagnostics: `SelfInImplicitParamList`,
`SelfNotFirstParam`, and `SelfOutsideParamList`. The benchmark source
generator and the documentation adopt the `(self)` shorthand; the
prelude, the examples, and the test data are migrated in the following
commits.
Assisted-by: Claude Code with Claude Opus 4.7
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
This helps us move away from the clone-with-modifications approach to
thunking, which gets unwieldy as signatures get more complex.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Fundamentally, this uses forward declarations of Clang types to reduce
the overall compile time cost of Clang headers across the codebase.
Tracing and profiling showed ~2s of every check TU's ~8-12s compile time
going just to parsing Clang frontend and AST headers pulled in via a few
sem_ir and check headers that only use the Clang types by pointer or
reference:
- sem_ir/cpp_file.h (reached via sem_ir/file.h by ~150 TUs) included
clang/Frontend/CompilerInstance.h, clang/CodeGen/ModuleBuilder.h,
clang/AST/Mangle.h, and llvm/IR/Module.h. CppFile's accessors move out
of line to a new cpp_file.cpp and the header now forward-declares the
Clang types.
- check/cpp/context.h (reached via check/context.h by ~100 TUs) included
clang/Frontend/FrontendAction.h and clang/Parse/Parser.h, pulling in
clang's Sema.h and ASTUnit.h.
- sem_ir/clang_decl.h included clang/AST/Decl.h; the three small
functions that need complete Clang types move out of line.
- sem_ir/cpp_overload_set.h included clang/Sema/Overload.h solely for
the three-field OverloadCandidateSet::OperatorRewriteInfo, which is now
mirrored as CppOverloadSet::OperatorRewriteInfo, and clang/AST/Decl.h
solely for a pointer.
- sem_ir/name_scope.h's clang/AST/DeclBase.h include was vestigial.
TUs (and more narrowly included headers) that genuinely use the Clang
definitions now include the Clang headers directly.
Representative compile times (fastbuild, aarch64), combined with the
preceding instantiation-cost changes, relative to trunk:
- check/eval.cpp: 11.85s -> 6.94s (-41%)
- check/handle_operator.cpp: 7.71s -> 3.30s (-57%)
- language_server.cpp: 6.68s -> 3.16s (-53%)
- lower/handle.cpp: 6.75s -> 3.66s (-46%)
- sem_ir/file.cpp: 8.60s -> 6.11s (-29%)
- driver.cpp: 6.68s -> 4.78s (-28%)
Measured full-rebuild impact (316 first-party TUs, fastbuild): -689.5s
CPU, -29.9% relative to trunk.
Assisted-by: Claude
Change `Lookup` by InstId to return a ClangDecl pointer. All callers
were immediately calling `Get` anyway, so this makes call sites a little
shorter. The other `Lookup` method, by ClangDeclKey, is sometimes called
without calling `Get`, so left that as-is, but renamed to `LookupId`.
Also add a `decl` method to ClangDecl so that the commonly repeated
`clang_decl->key.decl` can be written `clang_decl->decl()`.
This ensures the clang_decls map is used as a cache - without this,
visiting the same entity twice could cause it to be
re-exported/duplicated. See attached test case.
Replace all uses of CppGlobalVarStore store with ClangDeclStore.
Adding a VarStorage->VarDecl mapping to ClangDeclStore is now done with
the `AddVar` method, which takes an extra `pattern_id` arg. While the
corresponding `ClangDecl` is unchanged from before, the reverse mapping
in `inst_id_to_clang_decl_id_` now uses the `pattern_id` as the key.
This is necessary because in some places the original VarStorage
instructions gets replaced (e.g. by a call to `Convert`). The
`pattern_id` remains stable in those cases.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Removing the clang_decl_id on SemIR::Function - using only the
clang_decls map to create the association between SemIR::Function and
clang::FunctionDecls.
This adds an `is_external` flag to ClangDecl to indicate whether the
entity originated from Carbon or was imported from another language.
(I'm open to names - I guess for now we mostly use "is this from C++" to
be more specific than "is this external" - eg: NameScope::is_cpp_scope)
Add `ExportVarToCpp`. This checks the `clang_decls` mapping and returns
an existing decl if found. Otherwise, it creates a new `VarDecl` and
adds it to the `clang_decls` mapping.
When lowering, in `FileContext::BuildGlobalVariableDecl`, the
`clang_decls` mapping is used to lookup an existing
`llvm::GlobalVariable` for the instruction. If found, use that rather
than creating a new one to avoid an unwanted second definition in the
llvm IR.
This change adds rudimentary support for C++ ranges in Carbon range-for
loops. Since C++ ranges are exposed through `Core.Iterate`, C++ ranges
have the same limitations as Carbon ranges (e.g. can't return
references).
`Iterate.CursorType` now requires `Destroy`, since types that implement
`CppRangeForIterate` can't be used in range-for loops unless their
cursor type can be verifiably destructible.
Since we can't currently iterate over ranges as references, the
`CppUnsafeDeref` interface can't require types that implement it to take
or return references.
For now, disable the use of array types as by-var paramters and by-init
return types when exporting Carbon functions to C++, as C++ does not
support raw arrays being passed or returned by value.
Assisted-by: Gemini via Antigravity
As suggested in [1], replace `FieldInitializerMap` with a `FieldStore`.
The corresponding `FieldId` is now stored in `FieldDecl`. To make room
for the `FieldId`, the `ElementIndex` is now stored in the `Field`,
along with the initializer.
In import_ref.cpp, resolving `FieldDecl` initializers is now supported,
and in convert.cpp `LoadImportRef` is called to do so. The
`field_initializer_import.carbon` test now passes.
Printing a `FieldDecl` instruction now prints the initializer as well,
if present. See field_initializer.carbon for an example.
[1]:
https://github.com/carbon-language/carbon-lang/pull/7238#discussion_r3283217158
Carbon-side thunks (for example the `Copy`/`Destroy` witness thunks
generated for imported C++ types) are mangled by Carbon, and their names
incorporate a fingerprint of the involved types. The instruction
fingerprinter identifies a class only by its name and parent scope,
which is sufficient for Carbon classes but not for imported C++ classes:
different specializations of one class template (and other cases such as
types in anonymous namespaces) share a Carbon name and parent scope. As
a result, the thunks for two distinct specializations could mangle to
the same name, producing a single LLVM function with two definitions and
failing `verifyModule` during lowering.
When fingerprinting a class imported from C++, also include the Clang
mangled name of its type.
Test: toolchain/lower/testdata/interop/cpp/thunks.carbon gains a split
with two specializations of one class template, each requiring a thunk;
their thunks now get distinct mangled names instead of colliding.
Assisted-by: Claude Code
---------
Co-authored-by: Christopher Di Bella <cjdb.ns@gmail.com>
We've talked about adding the title to the filename several times over
the years and it seems really valuable. This requires us to compute a
"slug" for the title spelling that can be part of the filename.
Beyond that, we crossed 7000 recently, and so it seems likely that we
will need to add digits sooner rather than later here, so this goes
ahead and moves us to 6 digits so we don't have to adjust again for a
reasonable length of time.
To implement this and ensure we can sustain it going forward this adds a
tool to our pre-commit that validates (and corrects if needed) the
filename.
In order to update everything and keep links working, there are a _lot_
of changes, but the most interesting for direct review are in
`proposals/scripts`.
Assisted-by: Antigravity with Gemini
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
`CppRangeForIterate` needs to support finding `begin()`/`end()` as a
pair of methods and as a pair of ADL-findable functions. This change
adds the ADL component, which lets us also diagnose types that don't
implement the interface.
Unlike methods, we apparently have support for overloads when using ADL.
Flush diagnostics before destroying Clang. If we see an `inline Cpp` and
`Cpp` initialization failed, recover by skipping the inline code rather
than CHECK-failing.
Provide a source location to the member expression used when a thunk
calls a member function. This ends up being used as the point of
instantiation when the return type triggers a template instantiation;
the absence of this location previously caused assertion failures within
clang.
Assisted-by: Gemini via Antigravity
The intent is to add visibility into how the fingerprint is computed, so
that fingerprinting issues and mangling collisions can be more readily
understood and fixed.
Assisted-by: Gemini via Antigravity
We were incorrectly computing the index of the Clang implicit conversion
corresponding to method arguments. This led to wrong code and a crash in
lowering due to a calling convention mismatch.
Fixes#7224.
Assisted-by: Gemini via Antigravity
This allows Clang to correctly generate the vtable for the exported
class.
There's still something wrong with new virtual functions in the Carbon
type (left a TODO) - I thought it might be related to not flagging
the CXXMethodDecl as virtual, but my initial experiments don't seem to
back that up, so I'll look into it further separately.
There's also a test regression due to an virtual (well, abstract
specifically, but I think it'd happen with a virtual one too) function
in an abstract class taking `self` by value being rejected since
the abstract class can't be instantiated. Not sure if this is a correct
change - the test's behavior could be preserved by using `ref self`
instnead of `self` in this function. Is that reasonable/expected? Should
we not require a type to be complete when passing by value if we can
compute the value representation without such completeness?
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This also fixes passing a value expression to a forwarding reference,
since we currently deduce a `const T&&` parameter in that case.
We were accidentally looking at the type of the thunk parameter (which
is never an rvalue reference) rather than the type of the callee
parameter.
Not every entry in a C++ vtable corresponds to a function that we want
to import. For the holes, leave a `SemIR::InstId::None` in the vtable.
Also mark vtables that extend a C++ vtable as being non-Carbon-native so
we don't try to lower them (and crash on the `None` entries).
In particular, we leave holes for destructors, since we don't have
destructor declarations on the Carbon side that need to override them.
* For Carbon `base class C`, export as a regular C++ class.
* For Carbon `class C`, export with the C++ `final` keyword attribute.
* For Carbon `abstract C`, mark the destructor as pure virtual in cases
where no member function is abstract, or emit an error if the destructor
is not virtual.
To support the final point, mark the destructor of an exported class as
virtual if it overrides a virtual destructor from the base class.
In passing, fix a crash exporting fields if the class has an invalid
base type.
For now, hide `override fn`s from name lookup, so that the base class
version is always used, as the derived-class version does not have its
own vptr entry and so would not do the right thing if a further-derived
class adds a new override. This is implemented via a new access kind of
`Hidden`.
When checking the overriding function, pass in the expected `Self` type
and check the `self` parameter against that; the signature that we
generate for the thunk in the derived class is the base class signature
with the `self` parameter's type changed to the derived class.
When we generate a thunk for a virtual function, the thunk is assigned a
`virtual_index`, and the virtual function itself is not. When the thunk
makes a direct call to the virtual function, recognize this situation by
checking for a `virtual_index`, and perform a non-virtual call if there
isn't one.
Assisted-by: Gemini via Antigravity
Lucile Rose Nihlen
Richard Smith
Nicholas Bishop
Geoff Romer
Christopher Di Bella
David Blaikie
Chandler Carruthandjosh11b
