Fixes link failures when referencing a symbol involving a fingerprint
from a different package.
Previously we included the `Namespace`'s `import_id` as part of its
fingerprint, which caused local and imported namespaces to get different
fingerprints. We now store the `import_id` on the `NameScope` instead of
on the `Namespace` inst to avoid this problem.
Also, when we reach a package-level `NameScopeId`, consistently
fingerprint it as a (package name, library name) pair. Previously the
fingerprinting depended on whether it was imported or not, as an
imported `NameScopeId` had a parent scope (the current package). We need
to include the library name here so that private entities with the same
name in different libraries have different fingerprints.
Fixes mangling collisions when two thunks with the same name (eg, `Op`)
are created in the same context, which in turn would lead to LLVM
verifier failures and miscompiles.
To support this, add a new value store to track a little more
information about thunks beyond what's in the `Function`.
The `-Oz` flag has been [removed from
LLVM](https://github.com/llvm/llvm-project/pull/191363). The documented
replacement is to use `-O2` in conjunction with the `optsize` or
`minsize` attributes, which we already apply in lowering.
This is only fixing the decision about *whether* to produce a witness.
Implementation of the witness is still a TODO, though where a body is
generated, it should also precisely reflect where one _needs_ to be
generated.
Note the tests:
- toolchain/lower/testdata/function/generic/import_core_witness.carbon
- toolchain/lower/testdata/function/generic/import_unused_def.carbon
These tests can probably be produced _without_ Core.Destroy, but I found
the essence of them while trying to build //examples with Core.Destroy
and a simpler minimization wasn't striking me.
Assisted-by: Google Antigravity with Gemini
---------
Co-authored-by: jonmeow <jperkins@google.com>
This also switches to a more Bazel-based install layout, skipping the
FHS-based synthetic layout. The FHS-based layout is still reconstructed
explicitly when building an installable tar-ball.
The biggest change is to configure the just-built install as a Bazel
toolchain, including allowing it to build its own runtime libraries as
native Bazel libraries. This removes the need for a monolithic runtimes
build, all of that code logic is removed.
This should also pave the way to using the just-built toolchain for
doing a full 3-stage bootstrap. Building the 2nd stage is included here
as it was a particularly effective way to test that the Bazel
integration was fully working. Adding a 3rd-stage check for stability is
future work, but should be pretty easy.
There is a down-side: this uses the busybox to do the runtimes
compilation, which means they will be re-built after ~any change to
Carbon. However, the integration with Bazel should largely pay for this,
and we can continue to factor the tests away from depending on built
runtimes in most cases.
Now that we're building and testing the runtimes more directly, this
surfaced a problem with the layout of runtimes on macOS that is fixed
here. All of the Darwin OSes use a custom layout for their resource
directory compared to other targets. We now model this in both the C++
built runtimes and the Bazel built runtimes.
Assisted-by: Gemini via Antigravity
When a `var` is not explicitly given an initializer, initialize it in
one of two ways:
* If its type implements the new interface `Core.Default`, call
`Core.Default.Op` to initialize it.
* Otherwise, if its type implements `UnformedInit`, leave it in an
unformed state. For now, this is always an uninitialized state, but that
will change in the future.
* If neither of those apply, the `var` declaration is ill-formed.
This is a step towards implementing leads decision #6739 and proposals
#257 and #5913.
Assisted-by: Gemini 3.1 Pro via Antigravity
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Add an `IntFitsIn` interface with a custom witness, such that `T impls
IntFitsIn(U)` if `T` is an integer type all of whose values fit
losslessly into the integer type `U`. Use it to constrain implicit
conversions between integer types.
So far, this has not been extended to the
`CppCompat.[U]{Long32,LongLong64}` types, only to `Core.Int(N)` and
`Core.UInt(N)`.
Assisted-by: Gemini 3 Pro via Antigravity
Roll LLVM to `6811a83c81500ee373adfc0d9978ff9625a4cf1c`.
This includes https://github.com/llvm/llvm-project/pull/183831 which
moved the functionality of `finish()` on `DiagnosticConsumer`s into the
destructors, and removed the `finish()` method. So, our callers to
`finish()` are migrated to cause the destructor to run at that time
instead.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Turn a few section options on by default in Clang's options, and
propagate the setting from Clang to Carbon. These settings can't be
different between the two sides of the compilation, so merging the
behavior of Carbon's defaults and Clang's flags seems best.
Previously we only allowed conversions from `void*` to `U*` this way,
requiring casting via `void*` to get from `T*` to `U*`. That seems like
an unnecessary circumlocution.
Some module metadata changed - because rather than linking one module
with one module metadata value (eg: PIC Level 0, or unspecified) and one
module with a different one (PIC level 2, in clang) - we use Clang's
Module as-is, no merging required, so Clang's module metadata sticks
rather than being merged with default values from Carbon.
Also tweaked the name we use for Clang's module name so it matches the
carbon file name.
Otherwise the IR changes seem to be just reorderings - C++ interop goes
first, then Carbon, rather than the other way around.
Background:
https://docs.google.com/document/d/1wi85FRiWh4X9A-gCYMVGKR40-q5fM6-3JaSpePk-XCY/edit?usp=sharing
And specifically this work is essentially an alternative to #5543
Clang's code generation is implemented through an ASTListener
(clang::CodeGenerator) that is attached throughout Clang's
parsing/sema/code
generation phases and acts on Clang AST incrementally throughout that
process.
Prior to this patch, Carbon has only created the CodeGenerator during
Carbon's
`lower` phase, missing out on key callbacks that would be made by Clang
during
`check`. Some of these issues were addressed by #6237 and #6483 - but
there were
still remaining cases where the delayed processing lead to missing
functionality.
With #6483 much of the Clang code that made multithreaded complexity of
#5543 is
no longer present, and we have access to the point of ASTListener
registration
so we can register the CodeGenerator there and consume its resulting
llvm::Module during lower.
Examples of some of the bugs this addresses are seen in the linked doc,
and
checked in as tests in this change in
`clang_code_generator_callbacks.carbon`
An indicental bug that's also fixed, and caused all the other test case
churn,
is that the `CodeGenerator` created during `lower` wasn't getting passed
the
Clang `CodeGenOpts` and was creating its own default - so, most notably,
optimization flags were not respected. This meant that the LLVM IR from
Clang
was always -O0 style IR (optnone, no inlinehint, no TBAA, etc). With
this
change, now the Clang IRGen gets the real `CodeGenOpts` and respects
optimization/other flags specified there.
This is only meant to be a rough proof of concept - I'm totally open to
reworking this in any way (even quite substantially) if folks have ideas
about
how this should be implemented most generally/elegantly/etc.
Previously, we used the FHS "prefix" concept as the basis of the
install, but this makes it hard to integrate an installed toolchain with
Bazel (or similar) build system where it wants the "root" of the
toolchain to have some specific files (`MODULES.bazel` or
`BUILD.bazel`), and cannot reference anything outside that directory
tree.
An easy solution is to make the `lib/carbon` directory the root of the
install and never walking up from it. Then we simply have a `bin/carbon`
symlink to the busybox that is useful for getting the command into the
PATH, but isn't used for anything else. The FHS-constrained install
paths surround a root we fully control the layout and files within.
While initially motivated by trying to make a single toolchain structure
that works both for installation and for Bazel, it actually makes the
paths we end up using in the toolchain much simpler. We no longer have
awkward `.../lib/carbon/../../lib/carbon/...` sequences in the toolchain
which is cleaner and even a (trivial) efficiency gain.
As I was doing this I noticed several out-of-date comments that I tried
to fix, and I tried to improve some code reuse rather than re-computing
paths.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Support an implicit conversion from `T*` to `Cpp.void*` and to `const
Cpp.void*`, and an `unsafe as` conversion in the opposite direction.
In order to support C++ calls taking and returning `void*` (which get
mapped to Carbon `Optional(Cpp.void*)`, also support conversions from
`Optional(T)` to `Optional(U)` if there's a conversion from `T` to `U`.
Fix a bug in `OptionalStorage` for `T*` where its `HasValue` was exactly
backwards.
This unifies the default Clang arguments between the `clang` subcommand,
the `link` subcommand, and the `ClangInvocation` built for C++ interop.
This sets the stage to integrate either pre-built or on-demand runtimes
flags for both of these. However, this PR should have very little
practical difference. The biggest functional change is wrapping the
default arguments in flags to allow unused flags so that we can build a
collection of flags viable across compile and link.
### Description
Mangling collisions occur when implementing interfaces with generic
parameters. The mangler does not use the specific id, causing the same
symbol `_C[FunctionName].[PackageName]:[InterfaceName].[PackageName]` to
be generated for all of the implementations below:
```carbon
// Generic interface parameters ignored
impl C as I(A)
impl C as I(B)
// Generic class parameters ignored
impl D(A) as I
impl D(B) as I
// Both ignored
impl D(A) as I(A)
impl D(B) as I(B)
```
### Changes
Updated the mangling logic for `SemIR::ClassDecl` and
`SemIR::InterfaceDecl` to include the specific id. Now the mangling
ensures unique symbols for generic implementations using the format:
`_C[FunctionName].[FunctionSpecificId].[PackageName]:[InterfaceName].[InterfaceSpecificId].[PackageName]`.
Closes#6498
Pursuant to recent decisions on #6124, switch `Destroy` to use a
`CustomWitness` for its implementation. Right now this is manufacturing
no-op implementation functions on each lookup, which obviously isn't
ideal but is intended as a first pass. I'm mostly trying to find the
right balance between updating the approach to reflect new decisions,
while still breaking apart work in a way.
The `CoreInterface` logic is intended to build on `CoreIdentifier`
support. We have a number of additional interfaces that require
specialized logic, and that'll extend pretty far with C++ interop, so it
seemed easiest to have a generic function for it. That's what's
replacing the logic inside C++ interop that was doing string comparisons
(which could have already been moved to `CoreIdentifier`, I just missed
it in my first pass).
This adds `CustomWitness` support because the `Destroy` witnesses can be
imported cross-file. `CustomWitness` was previously only used for C++
types, which don't yet support import, which is why that wasn't
previously an issue. The addition of `query_specific_interface_id` is
similarly needed in order to get correct sorting of witness blocks when
imported.
This PR also removes builtin constraint logic (note this is in a
separate commit to help review; it's not a separate PR because it's
difficult to split apart without tests breaking). This had been made
generic with the expectation that destroy, copy, move, and conversions
would all need related support. Under the new decision, we are not going
to do blanket impls and will instead just manufacture a `CustomWitness`
for everything.
A lot of SemIR fingerprints change, but that's probably because the
addition of `Destroy` on core classes is yielding structural changes.
Instead of naming the root namespace `package` (because it's accessed by
the `package` keyword), change it to use the current package name. Note,
buried in the checksum changes,
`toolchain/check/testdata/package_expr/fail_not_found.carbon`:
```
- // CHECK:STDERR: fail_not_found.carbon:[[@LINE+4]]:16: error: member name `x` not found in `package` [MemberNameNotFoundInInstScope]
+ // CHECK:STDERR: fail_not_found.carbon:[[@LINE+4]]:16: error: member name `x` not found in `Main` [MemberNameNotFoundInInstScope]
```
for:
```
// CHECK:STDERR: var y: i32 = package.x;
// CHECK:STDERR: ^~~~~~~~~
```
I'll leave it to you if you prefer this; the alternative I see is to
just rename `IsCorePackage` to `IsImportedCorePackage`, and/or change it
to a helper that takes a `Context` and does the right thing with
`parse_tree` (which, I need for `Destroy`-related reasons and was my
default approach).
This is a prerequisite for support for interop with C++ template names.
No behavior change here, except that it sadly changes the fingerprinting
for a lot of tests.
This adds just enough debug info for i32/int parameters and return
values, with a path forward for adding DWARF type metadata for other
types.
As it happens, return type information is carried separately from
parameter information:
* Return type information is carried in the `type` of the `DISubprogram`
(as a `DISubroutineType` - which does carry parameter type information
as well, but that's unused when the DWARF is emitted by LLVM)
* Parameter information is carried by `DILocalVariable`s with a non-zero
`arg` value (representing the order of function parameters)
In the absence of locations for the parameters (future work), nothing
would usually keep the `DILocalVariable` live/reachable when emitting
DWARF - so for cases where this can happen (for clang, this happens in
optimized builds where all references to the parameter variable might be
optimized away) the variables can be "retained" in a list on the
`DISubprogram` - achieved by passing `AlwaysPreserve` parameter to
`createParameterVariable` (adds them to a list, then that list gets
attached to the `DISubprogram` when it's finalized later)
For now, any unsupported types are emitted as `void*` (except void
return, which is implemented as void) as a placeholder.
Given this example:
```
import Core library "io";
class MyClass {
}
fn Unsupported(v: MyClass) {
}
fn Ret() -> i32 {
return 42;
}
fn Arg(x: i32) {
Core.Print(x);
}
fn Run() {
}
```
this is the resulting DWARF:
```
DW_TAG_compile_unit
DW_AT_name ("test.carbon")
DW_TAG_subprogram
DW_AT_name ("Unsupported")
DW_TAG_formal_parameter
DW_AT_type (0x00000066 "void *")
DW_TAG_subprogram
DW_AT_name ("Ret")
DW_AT_type (0x00000062 "int")
DW_TAG_subprogram
DW_AT_name ("Arg")
DW_TAG_formal_parameter
DW_AT_type (0x00000062 "int")
DW_TAG_subprogram
DW_AT_name ("Run")
DW_TAG_base_type
DW_AT_name ("int")
DW_TAG_pointer_type
```
And the debugger:
```
(gdb) p Ret()
$1 = 42
(gdb) p Arg(4)
4
$2 = void
```
I'm not sure if there's a way this logic should be merged with the logic
for making the `llvm::Function` type (which the `DISubroutineType`
building code was inspired by/copied from) - since they're done at
different times/places, I don't think there's an easy way to do it in
one pass, but maybe the code can be shared (even if it's run twice) in
some generic `SemIR::Function` type walker.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
This helps at least lldb handle calling functions (currently the debug
info describes every function as `void()`, so no parameters or return
values are supported) - seems gdb and lldb both depend on demangling to
varying degrees in C code (marking a function as "prototyped" in C in
DWARF does seem to also address this problem).
Given:
```
fn PrintThree() {
Core.Print(3);
}
```
Before:
```
(lldb) p PrintThree()
error: Couldn't look up symbols:
PrintThree
Hint: The expression tried to call a function that is not present in
the target, perhaps because it was optimized out by the compiler.
```
After:
```
(lldb) p PrintThree()
3
(lldb)
```
Adds a flag `--optimize=<mode>` that specifies what to optimize for:
* `--optimize=none` turns off the optimizer as much as possible, but
still respects always_inline.
* `--optimize=debug` aims to be the equivalent of `-Og` / `-O1`, and
provides optimizations that don't affect the ability to debug the
program. This is the default.
* `--optimize=size` optimizes for the size of the produced program, and
aims to be the equivalent of `-Oz`.
* `--optimize=speed` optimizes for the execution time of the produced
program, and aims to be the equivalent of `-O3`.
Following the approach taken by Clang, the optimization level feeds into
both the configuration of the LLVM pass pipeline and the attributes
added to function definitions generated by the frontend.
Optimization is performed in a new phase, `optimize`, which runs between
`lower` and `codegen`.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Adds minimal defaults for codegen so that those tests don't need to
specify the full command line.
Moves driver/testdata/compile to check/testdata/basics/raw_sem_ir, which
seems like it better reflects the focus. Removes the textual IR test
because we have plenty of those now.
Moves a multiline token diagnostic test to parse because (a) diagnostics
doesn't have other tests and (b) this is really testing the way that
parse structures the location.
Drops `--include-diagnostic-kind` from some driver tests that aren't
testing a diagnostic, so the flag felt a bit like noise (even before
this change, there are a few tests that don't specify it because they're
not intended to test an emitted diagnostic, just high-level diagnostic
behaviors).
Adds some comments to reflect my understanding of why a test exists,
when I was pausing to think about it.
Modifies the stdin test to do more dumps; which happens to expose we
currently misbehave.
Instead of building an eval block as a separate pass at the end of a
generic, build the eval block incrementally.
The larger change here is that asking for the type or constant value of
an instruction now always returns an unattached type or constant value,
in order to preserve the behavior that we previously achieved by doing
the rewrite to attached types and constant values at the end of handling
the generic.
This also incidentally fixes some subtle issues where attached types and
constant values would leak out into check and cause it to get confused
about differences between attached and unattached values. Check should
no longer see attached values except where it explicitly asks for them.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Don't import `ImplWitnessTable` into the `constants` block, because we
generally don't put `Unique` constants there. This matches the handling
of the other kinds of `Unique` constants. In order to keep the
instruction visible in formatted SemIR, add it to the `imports` block
instead.
Also fix a bug in the instruction formatter that resulted in
instructions in the `imports` block being omitted from the output if
they were only referenced by earlier instructions in the `imports` block
and by instructions in the `constants` block. This was already resulting
in some referenced instructions being omitted from the output, but also
occurred frequently for `impl_witness_table` instructions after this
change because it is common for the only reference to those instructions
to be from `impl_witness` instructions in the `constants` block.
This was originally needed to support constant evaluation of name
expressions, but that's now done in a different way.
This is actually a step toward treating all patterns as constants. The
upcoming change will do so in a slightly different way, and so it will
simplify the review to start from a baseline where patterns are never
constant.
TypeInstId is an InstId whose constant value has a type of TypeType.
This includes:
- Type value instructions, the `ClassType` or `IntLiteralType`
instructions.
- Constraint value instructions, which are the `FacetType` and
`TypeType` instructions, each of which also have type TypeType.
TypeInstId encodes in the type system that it is safe to convert the
instruction's value to a TypeId, and CHECKs at construction that this
invariant is maintained.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
In preparation for shifting from `TypeId`s potentially representing
attached types to always representing unattached types, using
[terminology suggested on
Discord](https://discord.com/channels/655572317891461132/963846118964350976/1359286326779973712).
This change causes us to track slightly more type spelling information
through SemIR.
One change that has significant impact on the SemIR output is that we
now build a `struct_type` instruction in each class representing the
types of the fields, including the spelling used for those types. This
is now no longer always identical to the corresponding canonical
`struct_type` for the object representation, so it's built separately
and owned by the class.
Also remove `TypeBlock` support entirely, as its only use was
representing `TupleType`s, which now use an `InstBlock`.
Instead of using None, use an explicit ImplWitnessTablePlaceholder in
the witness table for entries that have not yet been populated, to aid
debugging. This would ensure they would show up very clearly in the
SemIR. This uncovered some `<invalid>` in the SemIR under erroneous
conditions that have now been turned into `<error>`.
Add the ImplWitnessAssociatedConstant instruction which wraps the
canonical instruction found from the constant value of the rewrite
constraint. This ensures that we have an instruction inside the eval
block for a generic impl declaration for each rewrite constraint's
value, which allows Subst to be performed to rewrite the symbolic
constant of the ImplWitnessAssociatedConstant instruction to associate
it with the generic. This will prevent the otherwise orphaned symbolic
constant of the rewrite's value from being used which can not have a
specific applied to them.
While applying the new insts in InitialFacetTypeImplWitness(), rearrange
the function to use less nesting. And avoid using entity names from
imported instructions (as we found is not effective in deduce.cpp) and
use a local instruction by going through the constant value.
This PR is part of the effort to allow a rewrite to name a generic
parameter, such as `impl forall [T:! type] T as Z where .X = T`, however
tests for this involve a final impl so that we can typecheck that the .X
value is a specific T, so the tests will come with that work. This piece
is split off because introducing new instructions causes a lot of SemIR
churn, and I wanted to get that done separately.
Each of these types takes another type as an operand. Instead of storing
that other type as a `TypeId`, store it as an `InstId` so that we can
track how it was written, not only its canonical form.
The canonical constant values of these types continue to store the
canonical constant values of their operands, as normal.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Factor out logic to evaluate `EntityNameId` instead of duplicating it
between `BindSymbolicName` and `SymbolicBindingPattern`. Remove support
in `SymbolicBindingPattern` for evaluating a pattern to the constant
value of the corresponding binding, which doesn't really make any sense
given that patterns don't generally evaluate to the value that they
matched.
This results in the handling for `SymbolicBindingPattern` being simply
the default handling for an always-constant instruction, so remove the
special case for it entirely and change its constant kind to `Always`.
It's not entirely clear that it makes sense for `SymbolicBindingPattern`
to be treated as a constant when other patterns aren't, but we seem to
be relying on this in various places, so leave it as a constant for now.
Changing it to never be constant will be a smaller change now -- it just
requires changing the `constant_kind`.
The IR changes in the tests are fairly widespread, but mechanical, and
there are two kinds of things changing:
- `symbolic_binding_pattern`s in specifics now evaluate to
`symbolic_binding_pattern`s, not to the argument values. This means in a
few cases we end up with additional `symbolic_binding_pattern`
constants.
- We evaluate the type operand of `symbolic_binding_pattern` now, so an
error in the type will now properly be propagated into an error in the
pattern's constant value.
Move call_params_id from EntityBaseWithParams to FunctionFields.
No visible difference for Function. Since field call_params_id is
function specific fits better in FunctionFields.
- Explicitly document that `*Param` and `*ParamPattern` insts represent
`Call` parameters.
- Stop wrapping compile-time parameter patterns in `ValueParamPattern`
insts (because they aren't `Call` parameters).
- Document how `MatchContext::results_` relates to the `Call`
parameters, and be more consistent about when it's written to.
- Remove `RuntimeParamIndex::Unknown`: we no longer need to distinguish
"this `Param`'s runtime index is unknown" from "this `Param` isn't a
runtime param", because we no longer use `Param`s at all in the latter
case.
- Rename `RuntimeParamIndex` to `CallParamIndex`.
As a side effect of removing the `ValueParamPattern` insts, this fixes a
minor diagnostic bug where `NoteInitializingParam` didn't identify the
specific parameter that led to a deduction failure, because it expects
generic parameters to only be represented by `SymbolicBindingPattern`s,
but before this change they could be wrapped in `ValueParamPattern`s.
Expands the CompleteTypeInfo to include information about abstract
classes and computes that information as part of completing the type.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Dana Jansens <danakj@orodu.net>
Treat template bindings as introducing template phase, and propagate it
in the same way we propagate the checked generic phase.
Rename "symbolic" to "checked symbolic" to make room for "template
symbolic". Also rename "phase" to "dependence".
Change parse tree from `template (T:! type)` to `(template T):! type`,
so that we have information about whether a binding is a template
binding available when forming the representation of the binding
pattern. This incidentally fixes a bug that we would accept `template
addr A:! B` instead of the intended `addr template A:! B`.
Track whether a symbolic binding is a template binding on the
`EntityName` object. I'm borrowing a bit from the `CompileTimeBindIndex`
for this in order to avoid making `EntityName`s larger. Longer-term, we
should think about using a different representation for symbolic
bindings, to avoid including these fields in all `EntityName`s, but
that's out of scope for this change.
So far, template bindings are treated as having the same phase as
checked bindings, but that will change in a future PR.
Richard Smith
Dana Jansens
Geoff Romer
Jon Ross-Perkinsandjonmeow
Chandler Carruth
David Blaikie
Özgür
Boaz Brickner
Alina Sbirlea
