Fix a bunch of cases where we use the same external name to mean
multiple different things in the same test. We've historically gotten
away with this, but under `--share-cpp-ast`, it becomes an error, at
least if the entity is either defined in, or used from, C++ code.
Assisted-by: Gemini via Antigravity (original change) and Claude Code
(suggested edits in review)
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Move `--output-last-file-only` and output filename synthesis logic out
of the general-purpose compile driver and into the `carbon compile`
subcommand, which is the only thing that should be using them. Track on
CompilationUnit whether it is being lowered, or whether it exists only
to be imported into other units.
`carbon compile` now never lowers inputs that it discovered for itself,
only inputs that were specified on the command line. In particular, it
doesn't lower (and throw away the result of lowering) the prelude any
more. This makes the toolchain tests about 10% faster in my crude
measurements.
Also, we now do not create a clang `CodeGenerator` for input files that
we are not lowering, similarly saving compilation time for units that
exist only to be imported, not lowered.
One minor change: we use the same mechanism to determine whether an
input is being lowered and to determine what the output filename is.
This means that `--phase=lower` and `--phase=optimize`, which lower but
don't produce an output file, still need an output filename to be
specified now in some cases. Given those are just debugging tools, I
think that's fine.
Assisted-by: Gemini via Antigravity
Instead of creating a CodeGenerator per CppDomain, and then crashing in
lowering when we try to consume the same llvm Module multiple times,
create a CodeGenerator for each CppFile within the domain.
For now, we mulitplex all of Clang's ASTConsumer output to all code
generators, which means that any strong external definitions within a
Carbon file (for example, in an inline `Cpp` fragment) will be emitted
to all output files in the same `CppDomain`, resulting in link errors
due to symbol redefinitions. This will be addressed later. But this
should be sufficient for Carbon compilations in which such symbols are
not defined.
We also don't yet attempt to classify which compilations will need C++
code generation, and instead create a clang `CodeGenerator` for every
Carbon file that has C++ imports. For `carbom compile`, only one Carbon
file will need code generation, and yet we still build multiple
`CodeGenerator` objects in general. Fixing this requires more plumbing
from the driver, and this will also be handled in a follow-up.
Assisted-by: Gemini via Antigravity
We can end up emitting the same thunk from multiple compilations in some
cases -- in particular, when the thunk is wrapping a function that is
either synthesized by the compiler or imported from C++. When this
happens, we will have multiple-definition link errors unless we allow
redefinitions across multiple files.
It'd be nice to detect when we need to do this and when we don't, but
that's a bit tricky to do in practice. Ideally, in fact, we would use a
different strategy, and emit the thunks as discardable definitions in
each compilation that *uses* them. But for now emitting them with
weak_odr linkage seems like a good way to make progress.
Fix a few API issues. There's also a newly-added file in compiler-rt
that is not supposed to be built by default but is not being excluded
properly by a glob. Added a patch to exclude that and sent
https://github.com/llvm/llvm-project/pull/208861 upstream.
Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
Each file dump now starts with a `; ---` comment and ends with a blank
line. This makes it easier to visually scan the dump for a file of
interest. The comment format is somewhat arbitrary; I chose `---` to
align with the `--- filename.carbon` separator in SemIR dumps, but
without the filename, because that appears on each of the next two lines
already.
The bulk of this change is changing most pattern insts to be `Always`
rather than `AlwaysUnique` constants, so that they can be wrapped in
`SpecificConstant`s to perform substitution. That then lets thunking
rely much more on `SpecificConstant` wrappers instead of deep-copying
the inst tree with modified types.
This approach to thunking should scale better, particularly as things
like form generics make function signatures more complex, because we can
leverage the existing support for constant evaluation and substitution.
Unfortunately, applying this approach to binding patterns will require
more work; see the TODO near the top of `thunk.cpp` for details.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Fix import logic to make all imported packages be children of the
`NameScopeId::Package` scope. Previously, indirectly-imported packages
would end up as children of their importing package's scope, which
resulted in them not being treated as packages at all, and in particular
not being fingerprinted as packages.
Fixing that caused a failure in the fingerprinting logic as we started
to encounter packages with no correspoding import scopes. Instead of
looking for import scopes, use a simpler mechanism to map packages to
their package names, and clean up.
Unfortunately the latter change churns all the fingerprints again :(
Hopefully this is the last time for a while.
When we import from another library in the same package, its entities
end up with our library as their parent scope, resulting in cross-file
fingerprint mismatches. Instead, only include the library ID when
fingerprinting either a package-private entity or an `ImportIRId` that
refers to a particular `SemIR::File`.
Carbon-side thunks (for example the `Copy`/`Destroy` witness thunks
generated for imported C++ types) are mangled by Carbon, and their names
incorporate a fingerprint of the involved types. The instruction
fingerprinter identifies a class only by its name and parent scope,
which is sufficient for Carbon classes but not for imported C++ classes:
different specializations of one class template (and other cases such as
types in anonymous namespaces) share a Carbon name and parent scope. As
a result, the thunks for two distinct specializations could mangle to
the same name, producing a single LLVM function with two definitions and
failing `verifyModule` during lowering.
When fingerprinting a class imported from C++, also include the Clang
mangled name of its type.
Test: toolchain/lower/testdata/interop/cpp/thunks.carbon gains a split
with two specializations of one class template, each requiring a thunk;
their thunks now get distinct mangled names instead of colliding.
Assisted-by: Claude Code
---------
Co-authored-by: Christopher Di Bella <cjdb.ns@gmail.com>
Fixes link failures when referencing a symbol involving a fingerprint
from a different package.
Previously we included the `Namespace`'s `import_id` as part of its
fingerprint, which caused local and imported namespaces to get different
fingerprints. We now store the `import_id` on the `NameScope` instead of
on the `Namespace` inst to avoid this problem.
Also, when we reach a package-level `NameScopeId`, consistently
fingerprint it as a (package name, library name) pair. Previously the
fingerprinting depended on whether it was imported or not, as an
imported `NameScopeId` had a parent scope (the current package). We need
to include the library name here so that private entities with the same
name in different libraries have different fingerprints.
Previously, we picked a single Carbon parameter pattern for each C++
parameter pattern. This doesn't work well in cases where the Carbon
semantics and the C++ semantics are not perfectly aligned. In
particular, when a parameter is passed by value in C++, that might mean
either pass-by-move (which in Carbon would best be modeled by a `var`
pattern, as no other form of parameter would perform a move) or
pass-by-copy (which in Carbon would best be modeled by a value
parameter, as a `var` parameter would force an extra copy).
After this change, we compute a passing mode for each parameter based on
the implicit conversion sequence from the argument to the parameter as
determined by C++ overload resolution, and use that to determine the
Carbon pattern corresponding to each C++ parameter. This results in
potentially generating multiple different thunks for the same C++
function if it's called in different ways, but we already did that to
handle default arguments and list-initialization. The passing modes are
included in the thunk mangling.
Add a new value store for clang decl signatures, which capture the
information about parameter passing mode as well as the other existing
information about different ways that a C++ function might be imported
to Carbon.
Most of the rules for computing passing modes are the same as before:
const references use pass by value, non-const lvalue references use
pass-by-ref, non-const rvalue references use pass-by-var. But for C++
non-reference parameters, pick between pass-by-value and pass-by-var
based on whether the implicit conversion sequence was effectively
performing a copy. Prefer pass-by-value if either would work and they'd
do the same thing. We still use pass-by-value for const references, even
when the argument is an lvalue and we could pass a reference; we may
want to change this in future.
For virtual functions, we try to pick a worst-case passing mode, as we
can only pick a single signature for what goes in the vtable. Calls to
virtual functions will still use a thunk to C++, allowing variance in
the calling convention at call sites. We don't allow variance in the
overriders as we don't implement support for thunks for virtual
functions yet. We currently use pass-by-value for const reference
parameters here, but that should probably change at some point.
Assisted-by: Gemini via Antigravity
Richard SmithandChandler Carruth
Geoff Romer
David Blaikie
josh11bandJosh L
Christopher Di Bella