This proposal defines the concrete technical mechanisms for C++
interoperability. It specifies the precise syntax and semantics for
importing
C++ APIs. This includes the `import Cpp library "..."` and implicitly
importing
C++ built-in entities, and the establishment of the `Cpp` package as the
dedicated namespace for all imported entities.
This PR also includes high level language C++ Interop design and the
basics of importing C++ APIs and function calling.
Leaving plenty of TODOs to make it easier to fill in more details in
followups.
Part of #4666.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
This proposal details the toolchain implementation for calling imported
C++
functions from Carbon. It covers how C++ overload sets are handled, the
process
of overload resolution leveraging Clang, and the generation of "thunks"
(intermediate functions) when necessary to bridge Application Binary
Interface
(ABI) differences between Carbon and C++.
This extends #6364 to allow having:
* `Cpp.unsigned_long` as a distinct type when `unsigned long` is 32
bits.
* `Cpp.long_long` and `Cpp.unsigned_long_long` as distinct types when
`long` and `unsigned long` are 64 bits.
Similarly to #6364, we only support implicit conversions from the
matching literal type (`u32`, `i64` and `u64`).
See #6275 for rationale.
Part of #5263.
This defines `Cpp.void` as a custom type.
`Cpp.void*` is mapped to C++ `void*`.
Not supported yet: Conversions from and to other pointer types.
C++ Interop Demo:
```carbon
// main.carbon
library "Main";
import Core library "io";
import Cpp inline '''
#include <cstdio>
auto GetPointer() -> void* _Nonnull {
static int x = 8;
return &x;
}
auto GetValue(void* _Nonnull ptr) -> int {
return *static_cast<int*>(ptr);
}
''';
fn Run() -> i32 {
let ptr: Cpp.void* = Cpp.GetPointer();
Core.Print(Cpp.GetValue(ptr));
return 0;
}
```
```shell
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link main.o --output=demo
$ ./demo
8
```
Part of #6280.
This proposal defines a direct, zero-cost mapping between C++'s
`std::string_view` and Carbon's `Core.Str` for C++ interoperability.
The goal is to make C++ APIs that use `std::string_view` feel native and
seamless when used from Carbon.
This mapping relies on the two types having an identical memory
representation, a condition that we will work to ensure across all
supported platforms.
This demonstrates that we don't diagnose when trying to call a base
class private function when we refer to the function from a derived
class without qualifications.
Demo: https://godbolt.org/z/vbWs1P915
This would save space for every `EntityName` that is not an imported C++
global variable.
C++ global variables include static data members.
Created `CppGlobalVarId`, `CppGlobalVarKey` and `CppGlobalVar` to allow
having `CanonicalValueStore` that maps `EntityNameId` (which is in
`CppGlobalVarKey` and `CppGlobalVar`) to `ClangDeclId` (which is also in
`CppGlobalVar`).
This is similar to `ClangDeclId`, `ClangDeclKey` and `ClangDecl` .
This demonstrates two issues:
1. It seems like we wrongly treat private static data members the same
way we treat protected and allow access to them from within derived
classes member functions.
2. Calling instance member functions of a base C++ class using a derived
class as self (no implicit upcast) is not yet supported. This isn't
related to access control, but prevents us from testing some access
control use cases.
Part of #5859.
Implemented by generalizing the reference type support for parameters
and return values to other use cases.
The changes to the `method.carbon` test are due to to supporting the
reference types but not supporting the necessary conversions.
C++ Interop Demo:
```c++
// global.h
struct C {
int member = 0;
int& member_ref = member;
};
extern C& global;
```
```c++
// global.cpp
#include "global.h"
static C static_c;
C& global= static_c;
```
```carbon
// main.carbon
library "Main";
import Core library "io";
import Cpp library "global.h";
fn Run() -> i32 {
Core.Print(Cpp.global->member);
++(*Cpp.global->member_ref);
Core.Print(Cpp.global->member);
++(*Cpp.global->member_ref);
Core.Print(Cpp.global->member);
return 0;
}
```
```shell
$ clang++ -stdlib=libc++ -c global.cpp
$ bazel build toolchain:carbon && bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link global.o main.o --output=demo
$ ./demo
0
1
2
```
**Without this change**:
```shell
main.carbon:10:14: error: semantics TODO: `Unsupported: var type: C &`
Core.Print(Cpp.global->member);
^~~~~~~~~~
main.carbon:10:14: note: in `Cpp` name lookup for `global`
Core.Print(Cpp.global->member);
^~~~~~~~~~
```
Part of #6006 and #6186.
The tests are almost identical and test the same logic so basically
duplicated.
The extra coverage that was in `struct.carbon` is added to
`class.carbon`.
One basic test in `struct.carbon` was left just to make sure `struct` is
supported.
Part of #5150.
This requires changing `ReturnSlotPattern` and `OutParamPattern`
definitions to use untyped node id, so they can have any associated
node.
Follow up of #5197.
Part of #5064.
This fixes a bug, which seems to have been introduced in #6108.
In the new test, without this change, we will diagnose with
```
error: semantics TODO: `Unsupported: parameter type: ExplicitObjectParam` [SemanticsTodo]
```
This is a follow up of #6082, which added support for reference types,
but not for return types.
C++ Interop Demo:
```carbon
// main.carbon
library "Main";
import Core library "io";
import Cpp inline '''
struct C {
auto Inc() -> void { ++x; }
int x = 0;
};
auto GetC() -> C& {
static C c;
return c;
}
''';
fn Run() -> i32 {
Core.Print(Cpp.GetC()->x);
Cpp.GetC()->Inc();
Core.Print(Cpp.GetC()->x);
Cpp.GetC()->Inc();
Core.Print(Cpp.GetC()->x);
return 0;
}
```
```shell
$ bazel build toolchain:carbon && bazel-bin/toolchain/carbon compile main.carbon && bazel-bin/toolchain/carbon link main.o --output=demo && ./demo
0
1
2
```
**Without this change**:
```shell
main.carbon:19:14: error: semantics TODO: `Unsupported: return type: C &`
Core.Print(Cpp.GetC()->x);
^~~~~~~~~~
```
Part of #6148.
This allows to find the spaceship `operator<=>` when a comparison
operator is not available, and `operator==` when `operator!=` is not
available.
Support added to both lookup and overload resolution, by adding
`OperatorRewriteInfo` and propagating it in `CppOverloadSet`.
In case overload resolution chooses to use an operator which requires
rewriting, we emit a `TODO` since rewriting is not yet supported.
Part of #6170.
Before this change, we wrongly ignore the decision to generate a thunk
for a function with default args by overriding this decision with the
fact the return type by itself doesn't require a thunk.
This causes not generating a thunk which leads to crashing in lowering.
Add tests that show that now thunk is generated in `check` and it no
longer crashes in `lower`.
Follow up of #6108.
After this change, we correctly increment the offset by the next switch
case type.
Before this change, we accidentally incremented the offset by
`functions()` size instead of `cpp_overload_sets()` size and vice versa.
Also sorted the switch cases according to the order of the enum, for
consistency. This might help prevent a future similar incident.
This fix prevents crashing in the newly introduced test
`multiple_too_few_args_calls`.
This also has the side effect of showing `null name` for
`cpp_overload_set_type` and `cpp_overload_set_value`, instead of having
an arbitrary name.
Examples that demonstrate the old name is arbitrary can easily be seen
in tests like `cpp_namespace.carbon` and `decayed_param.carbon`, but
careful review would show that all old names are arbitrary, though often
luckily almost make sense.
We might want to have a proper name for these, but it's beyond the scope
of this crash fixing change.
See #6156.
Part of #5915.
This adds location information and prevents crashes in some cases of
template instantiation in operator lookup.
Removed `InCppOperatorLookup` note as it is no longer necessary.
Part of #5995.
Instead of calling `PerformCppOverloadResolution()` and use the complex
return value to call `PerformCallToFunction()`, we call
`PerformCallToCppFunction()` which will call both
`PerformCppOverloadResolution()` and `PerformCallToFunction()`.
Followup of #6112.
Part of #5995.
This diagnoses instead of crashing in some cases:
* When one of the operands is an incomplete Carbon type.
* When one of the operands is a C++ class that can't be completed due to
lack of Carbon supported.
The new tests cover these cases.
Part of #5995.
Multiple overloads for the same operator are now resolved using overload
resolution.
This change doesn't try to solve all issues with operator lookup.
Moved the operator lookup logic from `import` to `operators` and changed
it to take the args into account.
Use `Sema::LookupOverloadedBinOp()` (with ADL) when looking up operator
functions to create an overload set.
Verified all demos in #6017, #6020 and #6024 still work.
C++ Interop Demo:
```c++
// my_number.h
class MyNumber {
public:
explicit MyNumber(int value) : value_(value) {}
auto value() const -> int { return value_; }
private:
int value_;
};
class NotMyNumber {};
auto operator+(MyNumber lhs, MyNumber rhs) -> MyNumber;
auto operator+(NotMyNumber lhs, NotMyNumber rhs) -> NotMyNumber;
```
```c++
// my_number.cpp
#include "my_number.h"
auto operator+(MyNumber lhs, MyNumber rhs) -> MyNumber {
return MyNumber(lhs.value() + rhs.value());
}
auto operator+(NotMyNumber lhs, NotMyNumber /*rhs*/) -> NotMyNumber {
return lhs;
}
```
```carbon
// main.carbon
library "Main";
import Core library "io";
import Cpp library "my_number.h";
fn Run() -> i32 {
// Arithmetic
var num1: Cpp.MyNumber = Cpp.MyNumber.MyNumber(14);
var num2: Cpp.MyNumber = Cpp.MyNumber.MyNumber(5);
Core.Print(num1.value());
Core.Print(num2.value());
Core.Print((num1 + num2).value());
return 0;
}
```
**After this change:**
```shell
$ clang -c my_number.cpp
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link my_number.o main.o --output=demo
$ ./demo
14
5
19
```
**Before this change**
```shell
$ bazel-bin/toolchain/carbon compile main.carbon
main.carbon:14:15: error: semantics TODO: `Unsupported: Lookup succeeded but couldn't find a single result; LookupResultKind: 3`
Core.Print((num1 + num2).value());
^~~~~~~~~~~
main.carbon:14:15: note: in `Cpp` operator `AddWith` lookup
Core.Print((num1 + num2).value());
^~~~~~~~~~~
```
Part of https://github.com/carbon-language/carbon-lang/issues/5995.
Only do the lookup when the class is complete.
Before this change we crash in `Sema::LookupQualifiedName()` on
`Declaration context must already be complete!`.
This prevents a null pointer access crash when generating a thunk with
an automatically deduced trivial return type.
In this case, Clang calls `Sema::DeduceFunctionTypeFromReturnExpr()`
which calls `Sema::getReturnTypeLoc()`, which requires this information.
Part of #5514.
Based on #5920.
Added commented out better examples and clarified what is missing to
support them.
Added `tags` support to `carbon_binary` (based on #5967) to set the
`BUILD` rule to manual until we can find `cstdio` in macos.
```shell
$ bazel run examples/interop/cpp:hello_world
...
Hello world!
```
Added tests for different character types.
`char` is currently not in primitives prelude, so had to use full
prelude.
C++ Interop Demo:
```carbon
// main.carbon
library "Main";
import Cpp inline '''
auto output_char(char c) -> void {
printf("%c", c);
}
''';
fn Run() -> i32 {
let msg: array(Core.Char, 13) =
('H', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '!', '\n');
for (c: Core.Char in msg) {
Cpp.output_char(c);
}
return 0;
}
```
```shell
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link main.o --output=demo
$ ./demo
Hello world!
```
Part of https://github.com/carbon-language/carbon-lang/issues/5263.
Use it to dump the AST that includes a generated C++ thunk.
Based on #5917.
Also added printing of the full actual text when check fails to make
debugging easier.
Changed line replacement to allow removing complete lines.
Part of #5514.
When the C++ function has a parameter that is not a pointer and not a
signed integer of 32 or 64 bits, generate a thunk.
Terminology:
* Callee function: The C++ function we actually want to call.
* Thunk function: The C++ function we generated that calls the callee
function.
* A simple ABI type, for now, is one of:
* A pointer
* signed integer with 32 bits
* signed integer with 64 bits
The thunk function is marked `always_inline` and uses the `asm`
attribute to set its mangled to the callee function mangled name
suffixed with `".carbon_thunk"`.
When importing a C++ function, we decide whether calling it requires a
thunk and if so we generate it and import it as well, which is currently
a recursive call.
When calling the thunk function, we initialize a temporary storage for
each non simple ABI parameter type and take its address. This can be
optimized when the variable is already in storage.
Not supported yet:
* Functions with non void return values.
* Member methods.
Moved unsigned int param test from `arithmetic_types_direct.carbon` to
`arithmetic_types_bridged.carbon`, since only signed integers aren't
bridged using a thunk.
C++ Interop Demo:
```c++
// hello_world.h
struct S {
S() {}
S(const S&) { x = 1; }
int x;
};
void hello_world(S s);
```
```c++
// hello_world.cpp
#include "hello_world.h"
#include <cstdio>
void hello_world2(S s) { printf("hello_world2: %d\n", s.x); }
void hello_world(S s) {
printf("hello_world: %d\n", s.x);
hello_world2(s);
}
```
```carbon
// main.carbon
library "Main";
import Cpp library "hello_world.h";
fn Run() -> i32 {
var s : Cpp.S;
Cpp.hello_world(s);
return 0;
}
```
```shell
$ clang -c hello_world.cpp
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link hello_world.o main.o --output=demo
$ ./demo
hello_world: 1
hello_world2: 1
```
Before this change (no thunk - copy constructor not called when calling
`hello_world()`):
```shell
$ ./demo
hello_world: -1219172304
hello_world2: 1
```
Before this change we crash in this case when trying to use the outer
type.
After this change we diagnose a TODO.
Will add the missing support for this in a separate PR.
Part of #5533.
This is instead of having one test file that has different return types
and two test files for `i16` and `i32` param tests, which doesn't seem
consistent.
First commit does file renames for easier review.
Part of #5063.
Added test coverage to demonstrate name lookup following import of a
type of a function parameter.
SemIR changes show that the same decl isn't imported multiple times.
Part of #5533.
When mapping parameter types, we assume that if the `clang::Decl` isn't
mapped, the name wasn't added, so this fixes a bug that triggers a crash
otherwise.
Part of #5533.
