Lexer diagnostics used a single `const char*` as their source location,
which only allowed pointing at a single character rather than a run of
characters. This limited the underlining of unrecognized tokens, and
also made escape sequence diagnostics particularly strange by stopping
at whatever point the lexer discovered the error and pointing only at
that location. For example: `"\w"` was reported at the `w`, and `"\x"`
at the closing `"`.
This change tries to always keep the primary location of a diagnostic
the point at which typing (inserting or deleting) might fix the error.
So for `"\x"`, `\x` is underlined, but the position is at the `"` as
that's where you would type a hex code.
The lexer's emitter now takes a special lexing source location that
represents either a single position _between_ two characters, a single
character, or a range of the source buffer. The lexer diagnostics are
updated to construct these for the character range as appropriate.
Parser diagnostics for a missing token were located at the _next_ token,
which is particularly confusing for terminators like missing `;`s. Now
they are located at a position after the previous token, with or without
a space depending on the token, but typically on the same line.
Diagnostics still sort by how far the parser has read.
The parser diagnostics for unclosed groups also now attaches a label to
the opening token when it is on a separate line. And the diagnostics for
mismatched indent in a multi-line string attaches a label to the closing
`'''` that establishes that indent.
There are likely more location improvements we should do here, but this
seemed like a reasonable incremental improvement and is what Claude
found when doing an initial audit. One example where I think we should
make further improvements is around suggesting raw identifiers when we
find an unexpected `Cpp` or `Core` or `base`. We do this consistently
now, but raw identifiers should be rare. A TODO is added for these
cases.
Assisted-by: Claude Code
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
There were a number of iterations on the design that weren't reflected
in the initial port to the terminal rendering library. This PR tries to
complete the rendering side of things to match the design we landed on:
code-focused, with messages attached by lines to the code, and condensed
as much as possible to avoid the vertical space.
It also improves the styling to make things more visually
distinguishable such as adding a `->` to the main error line, and making
the underlines and connectors for it have a heavy weight (when the font
supports that).
Another major change is having multiple messages on a single line be
drawn in source lexical order. This requires the connectors to cross
without joining, but makes it easier to read the messages.
This PR also has some refactorings to enable these improvements, and a
_number_ of improvements to comment prose and terminology to try and
avoid jargon and make things read more naturally. Sorry for the fact
that Claude left a bunch of this mannerly language in comments despite
my efforts, but I'm continuing to try and drive the prose towards
something direct and simple to read. Some of the terminology
improvements also needed to be made in the design doc and those have
been made here.
Note that I don't think this rendering is perfect or should be viewed as
final. I'm happy to iterate on it further, and have a few adjustments in
mind already, but would like to focus this on getting the
_implementation_ of the current rendering design in place, and then we
can make targeted changes or tweaks as needed. For example, I'm not yet
fully happy with the rendering when messages are too long and we move
them all the way to the left, but am leaving that for a follow-up.
Assisted-by: Claude Code
Determining that a type has trivial destruction requires checking
whether all of its subobjects have trivial destruction and whether its
type has a custom witness for `Destroy.Op`. This isn't possible when
`Destroy` is the custodian for `Destroy.Op` and `SubobjectDestroy.Op`,
so we need to make `SubobjectDestroy` its own interface.
`Destroy.SelfDestruct` becomes a standalone function because it depends
on `SubobjectDestroy` and `Destroy` being implemented, and is also
responsible for generating any custom witnesses for both interfaces.
Making `SelfDestruct` an interface was experimented with, but this
required a blanket impl that caused issues for facet types.
`Destroy` depends on `SubobjectDestroy` to be implemented. We'd ideally
use `require impls SubobjectDestroy`, but it isn't complete yet, and it
also causes problems when `Destroy` is implemented inside the class
definition. We work around this by temporarily renaming the `Destroy`
interface to `DestroyImpl`, and introducing a constraint called
`Destroy` so that user code doesn't need to write `Core.SubobjectDestroy
& Core.Destroy`.
Treat the function as having a `-> auto` return type. Reject calls to
functions that still have a non-deduced `-> auto` return type, and
rewrite the function to have the actual return type once it is known.
Assisted-by: Gemini via Antigravity
Importing a function with a `-> val` return form, such as `fn F() -> val
i32`, crashed because `TryResolveInstCanonical` had no case for
`ValueForm`. Also, an imported `RefForm` was given type `type` rather
than `Core.Form`.
Import all three primitive forms with a single function template.
Assisted-by: Claude via Antigravity
Diagnostic rendering was part of the data model: `Loc::FormatLocation`
and
`Loc::FormatSnippet` wrote plain text to a stream, so no capability,
color, or
terminal width could reach them.
A diagnostic is now one message with labels, contexts, and location
information
attached, and `Diagnostics::Renderer` draws it into a
`Terminal::Buffer`. The
design is in `toolchain/docs/diagnostics_rendering.md` (#7668),
authoring in
`toolchain/docs/diagnostics.md`.
Before:
```
example.carbon:6:3: error: 1 argument passed to function expecting 0 arguments
Run0(1);
^~~~~~~
example.carbon:3:1: note: calling function declared here
fn Run0() {}
^~~~~~~~~~~
```
After:
```
error: 1 argument passed to function expecting 0 arguments
╭─┤ example.carbon:6:3
┆
3 │ fn Run0() {}
· ─────┬─────
· ╰──┤ calling function declared here
┆
-> 6 │ Run0(1);
· ───────
╰────────────
```
`--no-diagnostic-snippets` gives a compact form, `example.carbon:6:3-9`.
File
tests use it; the rendering tests under `toolchain/diagnostics/testdata`
ask for
snippets.
C++ interop diagnostics are data now rather than text Clang drew:
`SemIR::ClangSourceLocId` names a `clang::SourceRange`, and a Clang
location is
marked as the point it names.
`StreamConsumer` moves to a `:renderer` target that `:emitter` does not
depend
on, keeping `common/terminal` out of `consumer.h`. Capabilities come
from the
file behind the error stream, so tests get plain ASCII.
`renderer_fuzzer` draws diagnostics built from fuzzer bytes with every
capability combination, since a shape the renderer doesn't expect must
never be
why the compiler dies while reporting a problem.
TODOs mark the follow-ups: line weights and crossing lines, wrapping
that keeps
a quoted type whole, and ordering diagnostics that share a byte offset.
Also: `IncorrectExtension`'s hint moves into its message, since its note
had no
location of its own, and `strip_noise` in `check_diagnostics.py` blanks
multi-line `#define`s instead of deleting them.
Assisted-by: Claude Code
Proposal p5434 specifies `-> val T`, `-> ref T`, and `-> var T` as
return forms. `-> ref T` is parsed as a `RefPrimitiveForm`; extend that
to parse a leading `val` or `var` after `->` as a `ValPrimitiveForm` or
`VarPrimitiveForm`, reusing the nodes produced by `form(val T)` and
`form(var T)`, and check them as forms.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
When identifying a facet type, defer evaluation of LookupImplWitnesses
by redirecting them to evaluate into IdentifiedWitness. Then the caller
of identify will need to substitute the IdentifiedWitnesses, either with
witnesses of their own, or with LookupImplWitness to cause a lookup
operation.
This allows:
- An `impl as` on a named constraint can subst the witnesses in
designators from the named constraint with a witness to the impl itself.
This prevents them from looking for and finding some other overlapping
impl. (This PR)
- Impl lookup to collect rewrite/equality requirements from all named
constraints and then use the impls it has already found to evaluate any
designators in those requirements instead of performing impl lookup
again (creating a cycle). (Done in
https://github.com/carbon-language/carbon-lang/pull/7944)
- Similarly, an `impl as` can collect impls and equality requirements
from all named constraints and verify them correctly by replacing
witnesses that should refer to the impl with the correct witness instead
of performing a lookup and finding an overlapping impl. (Done in
https://github.com/carbon-language/carbon-lang/pull/7937)
- ImplWitnessAccess to be able to find rewrites from a named constraint,
not just from the initial DeclaredFacetType. (TBD)
To do this we need to introduce the concept of "identified" specifics.
In order to identify a facet type, we must construct a specific for each
require decl in a named constraint. These specifics will have
IdentifiedWitnesses inside them, but their args would otherwise match a
specific for the require decl constructed in another context. To avoid
polluting the canonical specific namespace, we introduce a second
namespace of specifics that are "identified" since they are created by
identifying.
As we have introduced a new global state that modifies eval behaviour,
we must disable that state when resolving a specific (as per
https://github.com/carbon-language/carbon-lang/pull/7923). But inside an
"identified" specific, we preserve the behaviour so that the witnesses
modified by identify (through SubstPeriodSelf or through evaluating a
require specific) have their evaluation deferred.
Previously `-> ref T` was parsed as a type expression whose outermost
node is a `ref` prefix operator (the same one used for `F(ref x)`
arguments), and `ReturnExprAsForm` recognized the resulting `RefTagExpr`
and turned it into a `RefForm`.
Per p5434, a return form that begins with `val`, `ref`, or `var` is not
a type expression. Parse a leading `ref` after `->` as a
`RefPrimitiveForm` node wrapping the type expression, reusing the node
produced by `form(ref T)`, and check it as a form. This prepares for
supporting `-> val T` and `-> var T` in the same way.
The only SemIR change is that the `ref_tag` instruction is no longer
emitted, and the return slot pattern refers to a `type_component_of` the
form, matching `->? form(ref T)`.
---------
Co-authored-by: Nicholas Bishop <nbishop@nbishop.net>
Removes support for unspecified default values. Fixes canonicalization
of the `DefaultValuePattern` instruction by making them immutable after
they are issued, and by removing the `DefaultValueId` operand which
wasn't being canonicalized.
This change partially implements [PR #7362], which revises how objects
are destroyed. It is a partial implementation for two reasons:
1. This change moves `Destroy.Op`'s current behaviour into
`Destroy.SubobjectDestroy`, but it doesn't add support for objects with
non-trivial destruction.
2. `Destroy.SubobjectDestroy` is a workaround for `require impls
SubobjectDestroy`. We aren't able to use the latter until the dependents
add their requirements' implementations to their own witness tables.
[PR #7362]: https://github.com/carbon-language/carbon-lang/pulls/7362
The type `type` is now a `FacetType` inst with no constraints. This
brings the model implemented in the toolchain into better alignment with
the language design. The `SemIR::TypeType` struct remains as a scope for
holding the `TypeInstId`, `ConstantId`, and `TypeId` constants, but is
not an `InstKind` anymore.
The `TypeType` inst looks a lot like singletons, but there are many
`FacetType` insts so it doesn't quite fit that model. So we put it
alongside singletons with a fixed inst id but refer to it as a more
general "builtin" inst that is not a singleton.
`Namespace::PackageInstId` is similar, and we group it with `TypeType`
conceptually as another builtin instruction with a fixed id.
No conversion is needed anymore to use a `type` as a facet, since types
also have a `FacetType` type. This simplifies and removes a number of
helpers and branches throughout the code.
The `TypeType` inst is now part of the constant store, so we end up
printing it in the constants block in every test. But it's also named
`type` rather than `%type` to preserve the majority of existing
formatting behaviour, though this does look different from other
constants.
Assisted-by: Opus 5 was used to generate a first draft and validate the
refactoring. Though nearly everything non-trivial the tool wrote has
been modified or rewritten.
Per #7737 we move the default value `InstId` storage from
a block in the `SemIR::Function` data structure to a
`SemIR::File` scoped `ValueStore`.
Moves the default value consistency checking to the general
merge argument pattern matching logic, which changes the
error message issued to the generic one.
Per feedback on #7665, this PR switches the default value
table storage from canonical constant inst_ids to
non-canonical.
Furthermore, this PR simplifies the default value support in
check by requiring that the first owned declaration of a
function completely specify all of its default values.
Updates the diagnostic code and tests to reflect this new
stricter requirement.
Use a single `SemIR::Function` per `Core` interface method, whether it's
generated locally or imported. This prevents generating duplicate
functions, which lead to different types when the witness appears in a
`FacetValue` as part of a specific for a class.
We use a `CanonicalValueStore` of `GeneratedFunction` objects that allow
finding an existing FunctionId for a `Generated` special function before
(re-)generating it. Mangling for `Generated` functions is also moved to
use the values from the `GeneratedFunction`'s canonicalization key, so
that we have a consistent source of truth for the unique ID of a
`Generated` function across all files.
New tests are in
`toolchain/check/testdata/impl/custom_witness/destroy.carbon`.
Updates the pattern matching code to support unspecified default values.
Adds logic to decl and def merge code to diagnose mismatches in defaults
if specified in both places, or if let entirely unspecified.
Per https://github.com/carbon-language/carbon-lang/pull/7521.
Adds a virtual node to `DefaultValuePattern` to end the
`ExprRegionForPattern` before checking the expression
for the default value.
When checking the default value expression, the context
was still configured to interpret expressions as patterns,
which caused some corruption of state with tuple-pattern
subpatterns.
Corrects an assertion failure I found while working on
feedback from #7665.
Adds check functionality to transform the parse node to SemIR. Only
supported for single declarations of functions, re-declaration and
imports to come in a subsequent PR.
Per #7521.
Implementing interface modifiers causes an infinite loop when generating
fingerprints because the witness value generates a fingerprint that's
dependent on something dependent on the witness value. We've debugged
this to the witness table's `elements_id` field.
This hack is a workaround for creating a new block type whose value is
not codependent with its identity.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This solves the problem where `interface` imports are incorrectly
diagnosed as duplicate names in impl files.
Follows the implementation logic used in `handle_class.cpp`.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
During impl lookup, for each (generic) impl candidate, we form a
specific for that impl by deducing its generic arguments. Then we
compare the query interface against the impl's specific interface. That
comparison needs the deduced arguments applied to the impl's specific
interface. Previously we were doing this by getting the impl's
constraint facet type with the impl's specific applied (via
`GetConstantValueInSpecific()`) and then identifying that facet type
with the impl's deduced self.
Identify is a fairly expensive operation. It runs subst, trying to
replace `.Self` references. It walks named constraints. It collects
require declarations. We're looking at making it do _more_ in the future
too, including rewrite constraint resolution and collecting rewrite and
same-type constraints. For this reason we have a cache to make it cheap
on the second run, but it's still a very heavyweight operation to
involve in impl lookup, when all we want is to apply the impl's specific
to its target interface.
We almost have all the information we need to avoid the identification
step. We have the impl's specific after deduction. And we have the
SpecificInterface that the impl is targeting in the `Impl` struct. When
we form the specific for the impl itself, we resolve the declaration
block and form new constant values for all instructions in there, but
that does not cover the SpecificInterface that we're storing in the
`Impl` struct. So we add a new instruction to the impl's eval block,
which will be symbolic when the impl is generic and the target interface
depends on a generic parameter. And we store the `InstId` in the `Impl`
struct. This allows us to gets its constant value later with the impl's
specific applied. From that constant value we can then pull out the
SpecificInterface that the impl is targeting.
Instead of tracking the cleanup scope depth on entry to each scope,
track an "ambient" cleanup scope depth that's *after* the destructors of
local variables in that scope. This gets increased to include the
destructors of local variables when we create a name-binding
declaration. Then, when we reach a point where temporaries should be
destroyed, run cleanups that are after the ambient cleanup scope depth
on the stack. This happens:
* At the `;` of a statement expression.
* At the `)` of an `if` or `while` statement.
* After performing the implied `HasValue()` call in a `for` statement.
Per informal agreement with leads, this means we lifetime-extend all
temporaries created in the initializer of a name-binding declaration to
the full scope of that declaration, but that temporaries created in an
expression statement are destroyed at the `;`.
Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
This adds SemIR structs and implements building `observe` lists, as well
as naming, formatting, and importing `observe` declarations.
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
In some cases the pattern block can depend on the initializer, so it
must be sequenced after it. See #7469 for a more detailed explanation of
why this is necessary.
The bulk of this change is changing most pattern insts to be `Always`
rather than `AlwaysUnique` constants, so that they can be wrapped in
`SpecificConstant`s to perform substitution. That then lets thunking
rely much more on `SpecificConstant` wrappers instead of deep-copying
the inst tree with modified types.
This approach to thunking should scale better, particularly as things
like form generics make function signatures more complex, because we can
leverage the existing support for constant evaluation and substitution.
Unfortunately, applying this approach to binding patterns will require
more work; see the TODO near the top of `thunk.cpp` for details.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Implements proposal #7016: `self` moves from the deduced implicit list
(`fn F[self: Self]()`) to the front of the explicit list. Its type may
be written explicitly (`fn F(self: Self)`) or omitted, in which case it
defaults to `Self` (`fn F(self)`, `fn F(ref self)`); `self` in the
implicit list is rejected.
Throughout checking, `self` is modeled as the first explicit parameter.
Because a method is just a function whose first parameter is `self`, it
can also be called as an ordinary function with the receiver passed
explicitly (`Type.M(obj, ...)`), not only as `obj.M(...)`. A new
`SemIR::CallArgParamPatterns` helper chooses the parameters matched
against the explicit arguments, excluding a leading `self` only when it
is supplied as a method-call receiver; arity checking, conversion, and
generic deduction use it. The resulting SemIR and lowering are
unchanged: `self` is still `call_param0`, and witnesses, thunks, and
vtables are unaffected.
An omitted `self` type is parsed as a `SelfBindingPattern` node with no
type expression; checking synthesizes the `Self` type so it behaves
exactly like `self: Self`. However, the exact spelling used must match
between a forward declaration and a definition, following #3763's rules
around declaration matching.
Generated functions, thunks, and C++ interop import/export build `self`
as the first explicit parameter, and the `self`-type override (e.g.
Derived->Base for a virtual override) applies to the explicit `self`.
Placement is validated by new diagnostics: `SelfInImplicitParamList`,
`SelfNotFirstParam`, and `SelfOutsideParamList`. The benchmark source
generator and the documentation adopt the `(self)` shorthand; the
prelude, the examples, and the test data are migrated in the following
commits.
Assisted-by: Claude Code with Claude Opus 4.7
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Fix import logic to make all imported packages be children of the
`NameScopeId::Package` scope. Previously, indirectly-imported packages
would end up as children of their importing package's scope, which
resulted in them not being treated as packages at all, and in particular
not being fingerprinted as packages.
Fixing that caused a failure in the fingerprinting logic as we started
to encounter packages with no correspoding import scopes. Instead of
looking for import scopes, use a simpler mechanism to map packages to
their package names, and clean up.
Unfortunately the latter change churns all the fingerprints again :(
Hopefully this is the last time for a while.
When we import from another library in the same package, its entities
end up with our library as their parent scope, resulting in cross-file
fingerprint mismatches. Instead, only include the library ID when
fingerprinting either a package-private entity or an `ImportIRId` that
refers to a particular `SemIR::File`.
Fixes link failures when referencing a symbol involving a fingerprint
from a different package.
Previously we included the `Namespace`'s `import_id` as part of its
fingerprint, which caused local and imported namespaces to get different
fingerprints. We now store the `import_id` on the `NameScope` instead of
on the `Namespace` inst to avoid this problem.
Also, when we reach a package-level `NameScopeId`, consistently
fingerprint it as a (package name, library name) pair. Previously the
fingerprinting depended on whether it was imported or not, as an
imported `NameScopeId` had a parent scope (the current package). We need
to include the library name here so that private entities with the same
name in different libraries have different fingerprints.
When a class extends an interface, referring to a member name of the
interface as an unqualified name should refer to the class's
corresponding associated entity value, not to the associated entity
itself. Similarly, in an `impl`, unqualified names of associated
entities should refer to the `impl`'s corresponding value for that
entity.
To support this, we treat `impl`s as `extend`ing their implemented facet
type, and we make lookups into an extended facet type use the `Self`
type of the extending `impl` or `class` if lookup finds an associated
entity. We already did the latter if the extending entity was an
interface; this extends the existing support for these other cases.
See
[here](https://docs.google.com/document/d/1rWcueFwIfZox6GKVGxiUG4cBzjrZ6djXiIDGyJDtrE4/edit?tab=t.0)
for the design doc.
This also removes the default value of the `result_type_inst_id`
parameter of `HandleAction`, moves it before the action in the parameter
list, and documents it. This solves two problems:
- The default made it easy to forget, leading to unnecessary
`TypeOfInst` instructions.
- When it was present, putting it after the fairly "bulky" action
argument tended to make the callsite harder to read.
- A function with a return declaration always has exactly one
`ReturnSlotPattern`, representing the whole return declaration (whereas
previously that was omitted for value and reference returns).
- The `ReturnSlotPattern` always has a subpattern with the same form.
`OutParamPattern` already plays that role for initializing forms, and
`TuplePattern` will play that role for tuple forms. This change
introduces `ValueReturnPattern` and `RefReturnPattern` to represent
value and reference return forms.
- As before, the `ReturnSlotPattern` has a corresponding `ReturnSlot`
that represents the output that is initialized by a `return` statement.
Its structure parallels the structure of the `ReturnSlotPattern`, so we
need `ValueReturn` and `RefReturn` insts that correspond to
`ValueReturnPattern` and `RefReturnPattern`.
This is a step toward supporting generic return forms, where the
`ReturnSlotPattern`'s subpattern may be an action: this change ensures
that evaluating the action for a specific form produces the same SemIR
as if the form were concrete to begin with. More speculatively, this
should simplify the implementation of `return` statements with compound
return forms.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This is only fixing the decision about *whether* to produce a witness.
Implementation of the witness is still a TODO, though where a body is
generated, it should also precisely reflect where one _needs_ to be
generated.
Note the tests:
- toolchain/lower/testdata/function/generic/import_core_witness.carbon
- toolchain/lower/testdata/function/generic/import_unused_def.carbon
These tests can probably be produced _without_ Core.Destroy, but I found
the essence of them while trying to build //examples with Core.Destroy
and a simpler minimization wasn't striking me.
Assisted-by: Google Antigravity with Gemini
---------
Co-authored-by: jonmeow <jperkins@google.com>
This is a step toward removing the index from `InitForm`, so that equal
form values always have equal representations.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Chandler CarruthandRichard Smith
Christopher Di Bella
David Blaikieandjosh11b
Dana Jansens
Geoff Romer
Lucile Rose Nihlen
Özgür T. Önsoy
Jon Ross-Perkinsandjonmeow
