The images beside the samples were screenshots taken by hand, so they
drifted: they still showed `package Carbon api;`, syntax the language
dropped in 2024.
`render_sample.py` renders a sample the way the grammar in this
repository actually highlights it, using `tmlanguage.py`, a small
TextMate tokenizer. VS Code runs grammars under Oniguruma, which we
cannot depend on here, but this grammar uses no Oniguruma-only syntax,
so `re` runs its regexes unchanged and the two agree on every character
of every Carbon file in the repository. The output is SVG, so
regenerating needs nothing but Python, and a later grammar change gets a
same-path image diff showing what it did to real code.
The samples change where a construct left the language (`api`, `Carbon`
as the package name, `StringView`, `destructor`), refresh the
`keywords.carbon` inventory against `token_kind.def`, and add sections
for octal, raw and block literals, character literals, lambdas, and raw
identifiers. `interop.carbon` is new, covering inline C++. These are
highlighting fixtures rather than programs, so the deliberately invalid
lines stay.
Assisted-by: Claude Code
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
This PR improves the vscode syntax highlighting.
- Added `comment` keys.
- Added highlighting of invalid numbers such as `0x`, `0b`, `0xa`, etc.
- Restricted highlighting of numeric type literals to common types to
avoid highlighting identifiers such as `i1`.
- Changed the highlighting of named operators (e.g., `as`).
- Added `char` and `str` to type literals.
- Added `const` to modifier keywords.
- Removed `addr` keyword.
- Refactored some rules to use `begin`/`end` to handle line breaks.
- Added highlighting to `choice` values as `enum` values.
- Updated the rules for matching `types`.
- Added highlighting to rhs of `adapt`, `alias`, `choice`, `constraint`,
`impl`, `interface`, `as`, and `impls`.
- Added highlighting to rhs of bindings.
- Added highlighting to function return types.
- Updated the rules for matching `functions`.
- Updated the rules for matching `variables`.
- Added highlighting unidentified words as `variable`.
- Added examples and before/after screenshots.
| Before | After |
| :---: | :---: |
| <img width="424" alt="before"
src="https://github.com/user-attachments/assets/e84d0ff9-237b-40c2-845b-ec550b8f7bea">
| <img width="431" alt="now"
src="https://github.com/user-attachments/assets/2c18640b-318a-4cd5-952c-bad61d3fdbca">
|
---------
Co-authored-by: Dana Jansens <danakj@orodu.net>
Chandler CarruthandDana Jansens
ÖzgürandDana Jansens