This is a re-do of #7888
Outside a call, the grammar scoped a name as a type or a variable by its
capitalization. A theme could not distinguish a type from a value with
the same spelling, or a parameter, a member, or an implemented interface
from any other name. Pointer member access `->` was scoped as a result
arrow, which scoped `Lookup` in `regions->Lookup(x)` as a type.
`entity.name.type` now applies only in a type position (after `:`, `->`,
`as`, `adapt`, or `require`, or in an `impl`) and to the name a nominal
declaration declares. A qualifier, in a declared name like `fn N.Base(T:
type).F()` or in a type, is `entity.name.scope-resolution` nested in
`entity.name.type`. As in the parser, a name is a qualifier when its
parameter lists are followed by `.`. Parameters, `let` bindings, choice
alternatives, members, implemented interfaces, primitive types,
`import`, and the strings naming a builtin or a library have their own
scopes, and brackets use the C++ grammar's scopes. A `->` written
directly after its operand is member access; no result arrow in the
repository is written that way.
The sample palette renders each scope that depends on context
differently, and `tmlanguage.py` now supports nested captures.
In 17 of the 19 themes measured, the VS Code built-ins plus One Dark
Pro, Night Owl, Tokyo Night, and Nord, 13% to 27% of the characters in
hand-written Carbon change color or style. A theme loses a color where
`.` is now punctuation, where word operators like `as` are no longer
control-flow keywords, or where a name outside a type position is no
longer a type.
Assisted-by: Claude Code
Outside a call, the grammar scoped a name as a type or a variable by its
capitalization. A theme could not distinguish a type from a value with
the same
spelling, or a parameter, a member, or an implemented interface from any
other
name. Pointer member access `->` was scoped as a result arrow, which
scoped
`Lookup` in `regions->Lookup(x)` as a type.
`entity.name.type` now applies only in a type position (after `:`, `->`,
`as`,
`adapt`, or `require`, or in an `impl`) and to the name a nominal
declaration
declares. A qualifier, in a declared name like `fn N.Base(T: type).F()`
or in a
type, is `entity.name.scope-resolution` nested in `entity.name.type`. As
in the
parser, a name is a qualifier when its parameter lists are followed by
`.`.
Parameters, `let` bindings, choice alternatives, members, implemented
interfaces, primitive types, `import`, and the strings naming a builtin
or a
library have their own scopes, and brackets use the C++ grammar's
scopes. A
`->` written directly after its operand is member access; no result
arrow in
the repository is written that way.
The sample palette renders each scope that depends on context
differently, and
`tmlanguage.py` now supports nested captures.
In 17 of the 19 themes measured, the VS Code built-ins plus One Dark
Pro, Night
Owl, Tokyo Night, and Nord, 13% to 27% of the characters in hand-written
Carbon
change color or style. A theme loses a color where `.` is now
punctuation, where
word operators like `as` are no longer control-flow keywords, or where a
name
outside a type position is no longer a type.
Assisted-by: Claude Code
The images beside the samples were screenshots taken by hand, so they
drifted: they still showed `package Carbon api;`, syntax the language
dropped in 2024.
`render_sample.py` renders a sample the way the grammar in this
repository actually highlights it, using `tmlanguage.py`, a small
TextMate tokenizer. VS Code runs grammars under Oniguruma, which we
cannot depend on here, but this grammar uses no Oniguruma-only syntax,
so `re` runs its regexes unchanged and the two agree on every character
of every Carbon file in the repository. The output is SVG, so
regenerating needs nothing but Python, and a later grammar change gets a
same-path image diff showing what it did to real code.
The samples change where a construct left the language (`api`, `Carbon`
as the package name, `StringView`, `destructor`), refresh the
`keywords.carbon` inventory against `token_kind.def`, and add sections
for octal, raw and block literals, character literals, lambdas, and raw
identifiers. `interop.carbon` is new, covering inline C++. These are
highlighting fixtures rather than programs, so the deliberately invalid
lines stay.
Assisted-by: Claude Code
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
This PR improves the vscode syntax highlighting.
- Added `comment` keys.
- Added highlighting of invalid numbers such as `0x`, `0b`, `0xa`, etc.
- Restricted highlighting of numeric type literals to common types to
avoid highlighting identifiers such as `i1`.
- Changed the highlighting of named operators (e.g., `as`).
- Added `char` and `str` to type literals.
- Added `const` to modifier keywords.
- Removed `addr` keyword.
- Refactored some rules to use `begin`/`end` to handle line breaks.
- Added highlighting to `choice` values as `enum` values.
- Updated the rules for matching `types`.
- Added highlighting to rhs of `adapt`, `alias`, `choice`, `constraint`,
`impl`, `interface`, `as`, and `impls`.
- Added highlighting to rhs of bindings.
- Added highlighting to function return types.
- Updated the rules for matching `functions`.
- Updated the rules for matching `variables`.
- Added highlighting unidentified words as `variable`.
- Added examples and before/after screenshots.
| Before | After |
| :---: | :---: |
| <img width="424" alt="before"
src="https://github.com/user-attachments/assets/e84d0ff9-237b-40c2-845b-ec550b8f7bea">
| <img width="431" alt="now"
src="https://github.com/user-attachments/assets/2c18640b-318a-4cd5-952c-bad61d3fdbca">
|
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>