Instead of allowing lower to pick whatever type layout it desires,
compute the layouts of types as part of completing the type, and make
lower build types that match that representation.
For now we assume that all pointers are 64-bit, since we don't have
access to target information. We allow tail padding reuse for structs
and tuple types (and by extension, for classes, since they use structs
as their object representation), but not for arrays.
In order to build matching LLVM types, we create LLVM packed structs
where necessary, and we insert inter-field padding on the end of the
previous field so that GEP indexes still always match Carbon's
ElementIndexes.
We don't yet use the computed alignment much in LLVM IR generation -- in
particular, `alloca`s, `load`s, and `store`s should probably use the
computed type alignment, but don't.
Assisted-by: Gemini via Antigravity
This is only fixing the decision about *whether* to produce a witness.
Implementation of the witness is still a TODO, though where a body is
generated, it should also precisely reflect where one _needs_ to be
generated.
Note the tests:
- toolchain/lower/testdata/function/generic/import_core_witness.carbon
- toolchain/lower/testdata/function/generic/import_unused_def.carbon
These tests can probably be produced _without_ Core.Destroy, but I found
the essence of them while trying to build //examples with Core.Destroy
and a simpler minimization wasn't striking me.
Assisted-by: Google Antigravity with Gemini
---------
Co-authored-by: jonmeow <jperkins@google.com>
This also switches to a more Bazel-based install layout, skipping the
FHS-based synthetic layout. The FHS-based layout is still reconstructed
explicitly when building an installable tar-ball.
The biggest change is to configure the just-built install as a Bazel
toolchain, including allowing it to build its own runtime libraries as
native Bazel libraries. This removes the need for a monolithic runtimes
build, all of that code logic is removed.
This should also pave the way to using the just-built toolchain for
doing a full 3-stage bootstrap. Building the 2nd stage is included here
as it was a particularly effective way to test that the Bazel
integration was fully working. Adding a 3rd-stage check for stability is
future work, but should be pretty easy.
There is a down-side: this uses the busybox to do the runtimes
compilation, which means they will be re-built after ~any change to
Carbon. However, the integration with Bazel should largely pay for this,
and we can continue to factor the tests away from depending on built
runtimes in most cases.
Now that we're building and testing the runtimes more directly, this
surfaced a problem with the layout of runtimes on macOS that is fixed
here. All of the Darwin OSes use a custom layout for their resource
directory compared to other targets. We now model this in both the C++
built runtimes and the Bazel built runtimes.
Assisted-by: Gemini via Antigravity
When a `var` is not explicitly given an initializer, initialize it in
one of two ways:
* If its type implements the new interface `Core.Default`, call
`Core.Default.Op` to initialize it.
* Otherwise, if its type implements `UnformedInit`, leave it in an
unformed state. For now, this is always an uninitialized state, but that
will change in the future.
* If neither of those apply, the `var` declaration is ill-formed.
This is a step towards implementing leads decision #6739 and proposals
#257 and #5913.
Assisted-by: Gemini 3.1 Pro via Antigravity
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Fixes#6895
Note this is just a short-term fix to avoid confusion, as the compile
structure needs to change on the whole.
Assisted-by: Google Antigravity with Gemini
Add an `IntFitsIn` interface with a custom witness, such that `T impls
IntFitsIn(U)` if `T` is an integer type all of whose values fit
losslessly into the integer type `U`. Use it to constrain implicit
conversions between integer types.
So far, this has not been extended to the
`CppCompat.[U]{Long32,LongLong64}` types, only to `Core.Int(N)` and
`Core.UInt(N)`.
Assisted-by: Gemini 3 Pro via Antigravity
Roll LLVM to `6811a83c81500ee373adfc0d9978ff9625a4cf1c`.
This includes https://github.com/llvm/llvm-project/pull/183831 which
moved the functionality of `finish()` on `DiagnosticConsumer`s into the
destructors, and removed the `finish()` method. So, our callers to
`finish()` are migrated to cause the destructor to run at that time
instead.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This currently doesn't include much, but we expect to be generating more
entities, such as `Destroy`, which I'm aiming to get more clearly
categorized here instead of `imports`.
Assisted-by: Google Antigravity with Gemini 3 Flash
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Implementation of unused pattern bindings #2022, continued.
Whereas previous PR #6460 took care of parsing, and PR #6479 prepared
the stage by using _ in some test cases, this PR has the the actual
implementation, using a simple dataflow analysis.
---------
Co-authored-by: Burak Emir <bqe@google.com>
Co-authored-by: jonmeow <jperkins@google.com>
Add an optional additional set of positional parameters that can be
passed to the `link` subcommand for Clang-style (or GCC-style)
`LDFLAGS`. These can _also_ contain object files, etc., and in fact it
is useful to allow them to contain object files in order to integrate
the `carbon link` subcommand into a build system that mixes both link
flags and object files. This at least happens with Bazel, and I suspect
is common.
Eventually, it would be nice to have sufficient semantics to handle all
the varieties of links we want without resorting to this escape hatch,
but that's likely a long way away and so it seems especially useful to
allow falling back to Clang's flags as needed for now.
This does somewhat directly surface the Clang implementation detail in
the command line syntax, but I don't see a lot of good alternatives.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Turn a few section options on by default in Clang's options, and
propagate the setting from Clang to Carbon. These settings can't be
different between the two sides of the compilation, so merging the
behavior of Carbon's defaults and Clang's flags seems best.
This makes it easy to wire up build systems like Bazel that need to know
the actual include paths used. It also gives us a convenient place to
export any other information that build systems or integrations need,
and to get debugging info from users.
Most of the complexity is computing the Clang header search paths, but
I couldn't see a direct way to get closer to the source-of-truth than
this, and it doesn't seem _too_ unreasonable.
Depends on #6636 - start review at commit
[643fdab1](6637/commits/643fdab1)
Previously we only allowed conversions from `void*` to `U*` this way,
requiring casting via `void*` to get from `T*` to `U*`. That seems like
an unnecessary circumlocution.
Some module metadata changed - because rather than linking one module
with one module metadata value (eg: PIC Level 0, or unspecified) and one
module with a different one (PIC level 2, in clang) - we use Clang's
Module as-is, no merging required, so Clang's module metadata sticks
rather than being merged with default values from Carbon.
Also tweaked the name we use for Clang's module name so it matches the
carbon file name.
Otherwise the IR changes seem to be just reorderings - C++ interop goes
first, then Carbon, rather than the other way around.
- Distinguish attached vs. unattached constants.
- Add some missing value stores to the top-level output.
- Add missing fields to various Print methods.
Background:
https://docs.google.com/document/d/1wi85FRiWh4X9A-gCYMVGKR40-q5fM6-3JaSpePk-XCY/edit?usp=sharing
And specifically this work is essentially an alternative to #5543
Clang's code generation is implemented through an ASTListener
(clang::CodeGenerator) that is attached throughout Clang's
parsing/sema/code
generation phases and acts on Clang AST incrementally throughout that
process.
Prior to this patch, Carbon has only created the CodeGenerator during
Carbon's
`lower` phase, missing out on key callbacks that would be made by Clang
during
`check`. Some of these issues were addressed by #6237 and #6483 - but
there were
still remaining cases where the delayed processing lead to missing
functionality.
With #6483 much of the Clang code that made multithreaded complexity of
#5543 is
no longer present, and we have access to the point of ASTListener
registration
so we can register the CodeGenerator there and consume its resulting
llvm::Module during lower.
Examples of some of the bugs this addresses are seen in the linked doc,
and
checked in as tests in this change in
`clang_code_generator_callbacks.carbon`
An indicental bug that's also fixed, and caused all the other test case
churn,
is that the `CodeGenerator` created during `lower` wasn't getting passed
the
Clang `CodeGenOpts` and was creating its own default - so, most notably,
optimization flags were not respected. This meant that the LLVM IR from
Clang
was always -O0 style IR (optnone, no inlinehint, no TBAA, etc). With
this
change, now the Clang IRGen gets the real `CodeGenOpts` and respects
optimization/other flags specified there.
This is only meant to be a rough proof of concept - I'm totally open to
reworking this in any way (even quite substantially) if folks have ideas
about
how this should be implemented most generally/elegantly/etc.
Previously, we used the FHS "prefix" concept as the basis of the
install, but this makes it hard to integrate an installed toolchain with
Bazel (or similar) build system where it wants the "root" of the
toolchain to have some specific files (`MODULES.bazel` or
`BUILD.bazel`), and cannot reference anything outside that directory
tree.
An easy solution is to make the `lib/carbon` directory the root of the
install and never walking up from it. Then we simply have a `bin/carbon`
symlink to the busybox that is useful for getting the command into the
PATH, but isn't used for anything else. The FHS-constrained install
paths surround a root we fully control the layout and files within.
While initially motivated by trying to make a single toolchain structure
that works both for installation and for Bazel, it actually makes the
paths we end up using in the toolchain much simpler. We no longer have
awkward `.../lib/carbon/../../lib/carbon/...` sequences in the toolchain
which is cleaner and even a (trivial) efficiency gain.
As I was doing this I noticed several out-of-date comments that I tried
to fix, and I tried to improve some code reuse rather than re-computing
paths.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Support an implicit conversion from `T*` to `Cpp.void*` and to `const
Cpp.void*`, and an `unsafe as` conversion in the opposite direction.
In order to support C++ calls taking and returning `void*` (which get
mapped to Carbon `Optional(Cpp.void*)`, also support conversions from
`Optional(T)` to `Optional(U)` if there's a conversion from `T` to `U`.
Fix a bug in `OptionalStorage` for `T*` where its `HasValue` was exactly
backwards.
The main changes here are:
- Introducing `InitForm` and `RefForm` to represent initializing and
reference forms (the two return forms currently supported by the
parser).
- Introducing the `FormType` singleton inst to represent their type
(i.e. `Core.Form`).
- Emitting an inst representing a function's declared return form as
part of handling the function signature.
The return form inst is currently ignored. Subsequent PRs will expose it
in `SemIR::Function` and use it to determine the form of call
expressions.
This unifies the default Clang arguments between the `clang` subcommand,
the `link` subcommand, and the `ClangInvocation` built for C++ interop.
This sets the stage to integrate either pre-built or on-demand runtimes
flags for both of these. However, this PR should have very little
practical difference. The biggest functional change is wrapping the
default arguments in flags to allow unused flags so that we can build a
collection of flags viable across compile and link.
### Description
Mangling collisions occur when implementing interfaces with generic
parameters. The mangler does not use the specific id, causing the same
symbol `_C[FunctionName].[PackageName]:[InterfaceName].[PackageName]` to
be generated for all of the implementations below:
```carbon
// Generic interface parameters ignored
impl C as I(A)
impl C as I(B)
// Generic class parameters ignored
impl D(A) as I
impl D(B) as I
// Both ignored
impl D(A) as I(A)
impl D(B) as I(B)
```
### Changes
Updated the mangling logic for `SemIR::ClassDecl` and
`SemIR::InterfaceDecl` to include the specific id. Now the mangling
ensures unique symbols for generic implementations using the format:
`_C[FunctionName].[FunctionSpecificId].[PackageName]:[InterfaceName].[InterfaceSpecificId].[PackageName]`.
Closes#6498
Pursuant to recent decisions on #6124, switch `Destroy` to use a
`CustomWitness` for its implementation. Right now this is manufacturing
no-op implementation functions on each lookup, which obviously isn't
ideal but is intended as a first pass. I'm mostly trying to find the
right balance between updating the approach to reflect new decisions,
while still breaking apart work in a way.
The `CoreInterface` logic is intended to build on `CoreIdentifier`
support. We have a number of additional interfaces that require
specialized logic, and that'll extend pretty far with C++ interop, so it
seemed easiest to have a generic function for it. That's what's
replacing the logic inside C++ interop that was doing string comparisons
(which could have already been moved to `CoreIdentifier`, I just missed
it in my first pass).
This adds `CustomWitness` support because the `Destroy` witnesses can be
imported cross-file. `CustomWitness` was previously only used for C++
types, which don't yet support import, which is why that wasn't
previously an issue. The addition of `query_specific_interface_id` is
similarly needed in order to get correct sorting of witness blocks when
imported.
This PR also removes builtin constraint logic (note this is in a
separate commit to help review; it's not a separate PR because it's
difficult to split apart without tests breaking). This had been made
generic with the expectation that destroy, copy, move, and conversions
would all need related support. Under the new decision, we are not going
to do blanket impls and will instead just manufacture a `CustomWitness`
for everything.
A lot of SemIR fingerprints change, but that's probably because the
addition of `Destroy` on core classes is yielding structural changes.
Instead of naming the root namespace `package` (because it's accessed by
the `package` keyword), change it to use the current package name. Note,
buried in the checksum changes,
`toolchain/check/testdata/package_expr/fail_not_found.carbon`:
```
- // CHECK:STDERR: fail_not_found.carbon:[[@LINE+4]]:16: error: member name `x` not found in `package` [MemberNameNotFoundInInstScope]
+ // CHECK:STDERR: fail_not_found.carbon:[[@LINE+4]]:16: error: member name `x` not found in `Main` [MemberNameNotFoundInInstScope]
```
for:
```
// CHECK:STDERR: var y: i32 = package.x;
// CHECK:STDERR: ^~~~~~~~~
```
I'll leave it to you if you prefer this; the alternative I see is to
just rename `IsCorePackage` to `IsImportedCorePackage`, and/or change it
to a helper that takes a `Context` and does the right thing with
`parse_tree` (which, I need for `Destroy`-related reasons and was my
default approach).
Per discussion, makes all symbolic local bindings a TODO. We should
implement them more correctly before making them operable. Right now
things partially work, but because constants behave mostly right in the
symbolic situations under tests. More broadly, it has incorrect behavior
and crashes, thus the TODO.
This converts most tests using `let` to instead using parameters, but
leaves some behind where a conversion either didn't make sense (e.g. in
`let` tests) or a conversion was unclear to me (multi-layer `let`, which
relies more on planned behavior that seems more bespoke to a local
`let`).
In let's `fail_generic.carbon`, there's a "// TODO: Should this be
valid?" that I'm removing because my understanding is the code in
question should be valid (the file is merged into let's
`generic.carbon`).
Refactoring `HandleAnyBindingPattern` a little because there's a TODO to
make it shorter, and it seemed like a reasonable drive-by change (let me
know if you think there's more I should do, or if I should remove said
TODO even though it's still a bit long).
Fixes#5982
This is a prerequisite for support for interop with C++ template names.
No behavior change here, except that it sadly changes the fingerprinting
for a lot of tests.
This adds just enough debug info for i32/int parameters and return
values, with a path forward for adding DWARF type metadata for other
types.
As it happens, return type information is carried separately from
parameter information:
* Return type information is carried in the `type` of the `DISubprogram`
(as a `DISubroutineType` - which does carry parameter type information
as well, but that's unused when the DWARF is emitted by LLVM)
* Parameter information is carried by `DILocalVariable`s with a non-zero
`arg` value (representing the order of function parameters)
In the absence of locations for the parameters (future work), nothing
would usually keep the `DILocalVariable` live/reachable when emitting
DWARF - so for cases where this can happen (for clang, this happens in
optimized builds where all references to the parameter variable might be
optimized away) the variables can be "retained" in a list on the
`DISubprogram` - achieved by passing `AlwaysPreserve` parameter to
`createParameterVariable` (adds them to a list, then that list gets
attached to the `DISubprogram` when it's finalized later)
For now, any unsupported types are emitted as `void*` (except void
return, which is implemented as void) as a placeholder.
Given this example:
```
import Core library "io";
class MyClass {
}
fn Unsupported(v: MyClass) {
}
fn Ret() -> i32 {
return 42;
}
fn Arg(x: i32) {
Core.Print(x);
}
fn Run() {
}
```
this is the resulting DWARF:
```
DW_TAG_compile_unit
DW_AT_name ("test.carbon")
DW_TAG_subprogram
DW_AT_name ("Unsupported")
DW_TAG_formal_parameter
DW_AT_type (0x00000066 "void *")
DW_TAG_subprogram
DW_AT_name ("Ret")
DW_AT_type (0x00000062 "int")
DW_TAG_subprogram
DW_AT_name ("Arg")
DW_TAG_formal_parameter
DW_AT_type (0x00000062 "int")
DW_TAG_subprogram
DW_AT_name ("Run")
DW_TAG_base_type
DW_AT_name ("int")
DW_TAG_pointer_type
```
And the debugger:
```
(gdb) p Ret()
$1 = 42
(gdb) p Arg(4)
4
$2 = void
```
I'm not sure if there's a way this logic should be merged with the logic
for making the `llvm::Function` type (which the `DISubroutineType`
building code was inspired by/copied from) - since they're done at
different times/places, I don't think there's an easy way to do it in
one pass, but maybe the code can be shared (even if it's run twice) in
some generic `SemIR::Function` type walker.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Right now, the impl lookup can both fail to resolve the specific
definition because it's symbolic, and return a "final" constant because
it's a `final impl`. This is adding an instruction to help ensure the
specific is resolved.
The constant evaluation is fully recursive, but I'm not adding a TODO
since that's a known issue with impl lookup in general.
Following #6357, map C++ `void` to a prelude class type
`Core.CppCompat.VoidBase`, not to a builtin type. This is mostly just
moving logic around, but does notably change `Cpp.void` from being an
incomplete type to being a complete-but-abstract type.
Also change `NullptrT` to be an adapter for `void*` instead of `()*`, to
follow the approved design.
Implicit conversions to `void` and to `void*` are still absent.
Part of #6280.
This helps at least lldb handle calling functions (currently the debug
info describes every function as `void()`, so no parameters or return
values are supported) - seems gdb and lldb both depend on demangling to
varying degrees in C code (marking a function as "prototyped" in C in
DWARF does seem to also address this problem).
Given:
```
fn PrintThree() {
Core.Print(3);
}
```
Before:
```
(lldb) p PrintThree()
error: Couldn't look up symbols:
PrintThree
Hint: The expression tried to call a function that is not present in
the target, perhaps because it was optimized out by the compiler.
```
After:
```
(lldb) p PrintThree()
3
(lldb)
```
Adds support to the `dump` debugger command for named constraint ids,
which are printed as `constraint<number>`. While doing so, we print
whether the `constraint` is complete or not, and add the same to
`interface` to match.
And we noticed that the printing of name and name scope ids, which are
not tagged, are very verbose by adding 7 `0`s to them for no reason. So
make the dump output easier to read by dropping 0 prefixes.
Before:
```
name_scope00000000: {inst: inst0000000E, parent_scope: name_scope<none>, has_error: false, extended_scopes: [], names: {name00000000: inst6000000F, name00000001: inst60000011}} {kind: Namespace, arg0: name_scope00000000, arg1: inst<none>, type: type(inst(NamespaceType))} `package`
```
After:
```
name_scope0: {inst: instE, parent_scope: name_scope<none>, has_error: false, extended_scopes: [], names: {name0: inst6000000F, name1: inst60000011}} {kind: Namespace, arg0: name_scope0, arg1: inst<none>, type: type(inst(NamespaceType))} `package`
```
Adds a flag `--optimize=<mode>` that specifies what to optimize for:
* `--optimize=none` turns off the optimizer as much as possible, but
still respects always_inline.
* `--optimize=debug` aims to be the equivalent of `-Og` / `-O1`, and
provides optimizations that don't affect the ability to debug the
program. This is the default.
* `--optimize=size` optimizes for the size of the produced program, and
aims to be the equivalent of `-Oz`.
* `--optimize=speed` optimizes for the execution time of the produced
program, and aims to be the equivalent of `-O3`.
Following the approach taken by Clang, the optimization level feeds into
both the configuration of the LLVM pass pipeline and the attributes
added to function definitions generated by the frontend.
Optimization is performed in a new phase, `optimize`, which runs between
`lower` and `codegen`.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
`ImplWitnessTablePlaceholder` is the only non-type singleton instruction
(`ErrorInst` is a type; while `ImplWitnessTablePlaceholder` exposes
`TypeInstId`, it's only used as an `InstId`).
In order to allow simpler handling of singleton instructions, replace
`ImplWitnessTablePlaceholder::TypeInstId` uses with
`InstId::ImplWitnessTablePlaceholder`. Since the placeholder instruction
was never evaluated, this has no significant effect on behavior.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
This defines `Cpp.void` as a custom type.
`Cpp.void*` is mapped to C++ `void*`.
Not supported yet: Conversions from and to other pointer types.
C++ Interop Demo:
```carbon
// main.carbon
library "Main";
import Core library "io";
import Cpp inline '''
#include <cstdio>
auto GetPointer() -> void* _Nonnull {
static int x = 8;
return &x;
}
auto GetValue(void* _Nonnull ptr) -> int {
return *static_cast<int*>(ptr);
}
''';
fn Run() -> i32 {
let ptr: Cpp.void* = Cpp.GetPointer();
Core.Print(Cpp.GetValue(ptr));
return 0;
}
```
```shell
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link main.o --output=demo
$ ./demo
8
```
Part of #6280.
This would save space for every `EntityName` that is not an imported C++
global variable.
C++ global variables include static data members.
Created `CppGlobalVarId`, `CppGlobalVarKey` and `CppGlobalVar` to allow
having `CanonicalValueStore` that maps `EntityNameId` (which is in
`CppGlobalVarKey` and `CppGlobalVar`) to `ClangDeclId` (which is also in
`CppGlobalVar`).
This is similar to `ClangDeclId`, `ClangDeclKey` and `ClangDecl` .
This change makes dumping and debugging work again with InstIds that are
now tagged with the CheckIRId. The textual representation of an InstId
is changed from `irN.instM` back to `instM` but the `M` is now a hex
value with the tag as part of it, which is the same number that is
physically in the `InstId::index` field. This prevents any cases where
we would potentially print incorrect values for large InstIds.
We teach the `dump` command in lldb to parse hex values for InstId so
that we can paste these numbers back into the debugger.
This layer allows runtimes to be built on-demand but cached in a
consistent and re-usable location on the system. It handles careful
filesystem operations to ensure consistency even in the face of multiple
versions and build configurations.
This addresses a number of TODOs from the initial runtimes building
on-demand, and sets the stage to scale up to more runtimes.
This doesn't switch on-demand runtimes to be on by default, I wanted to
wait and make that change as a separate step.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
The general strategy here is to force use of a thunk when we want to use
default arguments, and have Clang generate uses of the default arguments
on its side of the thunk.
To support this, change the key type used in `clang_decls` from being
just a `Decl*` to being a pair of `Decl*` and number of parameters in
the case of function decls. Import distinct `SemIR::Function`s for each
number of parameters that's used, and corresponding distinct thunks.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Add a `Core.String` class to the prelude representing a string view, and
rename the `String` keyword to `str` and make it evaluate to
`Core.String`.
`Core.String` is represented as a pair of a pointer to a character
(actually, to the first character of a string, but we don't have a way
of modeling that yet) and a size (which should be pointer-width, but is
currently always a `u64` as we don't have a `usize` equivalent yet).
`Core.String` values are generated directly by the toolchain for string
literal expressions.
This follows the direction established at the recent summit, but the
design implemented here has not been through the proposal process yet.
toolchain/check/testdata/builtins/char/basics.carbon and
toolchain/lower/testdata/builtins/char.carbon are probably the most
interesting tests here. The parse tests is required because this adds a
new node kind, and we need coverage of it; but the attached info is
minor. There's a fair amount of test churn here because I'm adding the
Core.Char and Core.CharLiteral types as new singletons.
My intent here is that `CharId` is always a unicode code point, even
when the type is a `Char` and thus must be a single UTF-8 code unit
(single byte). This mainly means the stored value of a `CharValue` can
be printed internally without knowing the type.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Richard Smith
Jon Ross-Perkinsandjonmeow
Chandler Carruth
Dana JansensandChandler Carruth
David Blaikie
Özgür
Boaz Brickner
