carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-04 22:02:52 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity
Files
fa12d9ded28bf9db192bace18cdccfe987ab4e2a
carbon-lang/utils/textmate/Samples/main.svg
T
Chandler Carruth 437be71f1f Overhaul the TextMate grammar (#7746) 
Four regions had `end` patterns that could fail to match, so `return
var;`, a `fn` with no parameter list, and an unterminated `"` each
swallowed the rest of the file; 77 of 1696 testdata files lost their
highlighting partway through. Operators were wrapped in `\b`, which only
holds next to a word character, so `a + b` highlighted nothing. And the
keywords had drifted about two years behind the lexer.

Identifiers are now classified by naming convention plus a call-site
lookahead, the way the Rust grammar does it, so nothing carries between
lines. Regions survive only for strings and embedded C++, where a
terminator reliably turns up, and raw strings spell out hash levels 0
through 2, so `\n` is an escape in `"..."` and plain text in `#"..."#`.
Trailing comments, character literals, raw identifiers, `$0`, `0o`
octal, arbitrary integer widths and six missing keywords are covered
now, `destructor` is gone, and `i32` reads as a type rather than as a
keyword.

Highlighting stays forgiving rather than diagnostic: anything after `//`
is a comment and odd numeric spellings still read as numbers. Pointing
out mistakes is the toolchain's job, and lenient rules hold steady while
you are still typing.

Every keyword and symbol in `token_kind.def` is covered, and unscoped
tokens across examples and the toolchain drop from 39% to 21%.

Note that I haven't tried to read and reason about every minute change
here as there are just too many. But I'm working on a follow-up PR that
adds testing that should be significantly easier te review.

Assisted-by: Claude Code
2026-09-15 22:03:22 +00:00

51 lines
11 KiB
XML
Raw Blame History

<svg xmlns="http://www.w3.org/2000/svg" width="721" height="898" viewBox="0 0 721 898">
<rect width="100%" height="100%" fill="#1f1f1f"/>
<g xml:space="preserve" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="14">
<text style="white-space:pre" x="12" y="26"><tspan fill="#6e7681"> 1 </tspan><tspan fill="#6a9955">// Part of the Carbon Language project, under the Apache License v2.0 with LLVM</tspan></text>
<text style="white-space:pre" x="12" y="45"><tspan fill="#6e7681"> 2 </tspan><tspan fill="#6a9955">// Exceptions. See /LICENSE for license information.</tspan></text>
<text style="white-space:pre" x="12" y="64"><tspan fill="#6e7681"> 3 </tspan><tspan fill="#6a9955">// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception</tspan></text>
<text style="white-space:pre" x="12" y="83"><tspan fill="#6e7681"> 4 </tspan></text>
<text style="white-space:pre" x="12" y="102"><tspan fill="#6e7681"> 5 </tspan><tspan fill="#6a9955">// More single line comments</tspan></text>
<text style="white-space:pre" x="12" y="121"><tspan fill="#6e7681"> 6 </tspan></text>
<text style="white-space:pre" x="12" y="140"><tspan fill="#6e7681"> 7 </tspan><tspan fill="#569cd6">package</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#4ec9b0">Carbon</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="159"><tspan fill="#6e7681"> 8 </tspan></text>
<text style="white-space:pre" x="12" y="178"><tspan fill="#6e7681"> 9 </tspan><tspan fill="#569cd6">interface</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#4ec9b0">HasValueParam</tspan><tspan fill="#d4d4d4">(</tspan><tspan fill="#4ec9b0">T</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">type</tspan><tspan fill="#d4d4d4">, </tspan><tspan fill="#4ec9b0">V</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">T</tspan><tspan fill="#d4d4d4">) {</tspan></text>
<text style="white-space:pre" x="12" y="197"><tspan fill="#6e7681">10 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">fn</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">Go</tspan><tspan fill="#d4d4d4">(</tspan><tspan fill="#569cd6">self</tspan><tspan fill="#d4d4d4">) -&gt; </tspan><tspan fill="#4ec9b0">T</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="216"><tspan fill="#6e7681">11 </tspan><tspan fill="#d4d4d4">}</tspan></text>
<text style="white-space:pre" x="12" y="235"><tspan fill="#6e7681">12 </tspan></text>
<text style="white-space:pre" x="12" y="254"><tspan fill="#6e7681">13 </tspan><tspan fill="#569cd6">impl</tspan><tspan fill="#d4d4d4"> () </tspan><tspan fill="#c586c0">as</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">HasValueParam</tspan><tspan fill="#d4d4d4">(</tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4">, </tspan><tspan fill="#b5cea8">5</tspan><tspan fill="#d4d4d4">) {</tspan></text>
<text style="white-space:pre" x="12" y="273"><tspan fill="#6e7681">14 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">fn</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">Go</tspan><tspan fill="#d4d4d4">(</tspan><tspan fill="#569cd6">self</tspan><tspan fill="#d4d4d4">) -&gt; </tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4"> { </tspan><tspan fill="#c586c0">return</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#b5cea8">42</tspan><tspan fill="#d4d4d4">; }</tspan></text>
<text style="white-space:pre" x="12" y="292"><tspan fill="#6e7681">15 </tspan><tspan fill="#d4d4d4">}</tspan></text>
<text style="white-space:pre" x="12" y="311"><tspan fill="#6e7681">16 </tspan></text>
<text style="white-space:pre" x="12" y="330"><tspan fill="#6e7681">17 </tspan><tspan fill="#569cd6">class</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#4ec9b0">Point</tspan><tspan fill="#d4d4d4"> {</tspan></text>
<text style="white-space:pre" x="12" y="349"><tspan fill="#6e7681">18 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">fn</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">Origin</tspan><tspan fill="#d4d4d4">() -&gt; </tspan><tspan fill="#569cd6">Self</tspan><tspan fill="#d4d4d4"> {</tspan></text>
<text style="white-space:pre" x="12" y="368"><tspan fill="#6e7681">19 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#c586c0">return</tspan><tspan fill="#d4d4d4"> {.</tspan><tspan fill="#9cdcfe">x</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#b5cea8">0</tspan><tspan fill="#d4d4d4">, .</tspan><tspan fill="#9cdcfe">y</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#b5cea8">0</tspan><tspan fill="#d4d4d4">};</tspan></text>
<text style="white-space:pre" x="12" y="387"><tspan fill="#6e7681">20 </tspan><tspan fill="#d4d4d4"> }</tspan></text>
<text style="white-space:pre" x="12" y="406"><tspan fill="#6e7681">21 </tspan></text>
<text style="white-space:pre" x="12" y="425"><tspan fill="#6e7681">22 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">var</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">x</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="444"><tspan fill="#6e7681">23 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">var</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">y</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="463"><tspan fill="#6e7681">24 </tspan><tspan fill="#d4d4d4">}</tspan></text>
<text style="white-space:pre" x="12" y="482"><tspan fill="#6e7681">25 </tspan></text>
<text style="white-space:pre" x="12" y="501"><tspan fill="#6e7681">26 </tspan><tspan fill="#569cd6">fn</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">Procedure</tspan><tspan fill="#d4d4d4">() -&gt; </tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4"> {</tspan></text>
<text style="white-space:pre" x="12" y="520"><tspan fill="#6e7681">27 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#c586c0">returned</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">var</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">zoop</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#b5cea8">0</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="539"><tspan fill="#6e7681">28 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#c586c0">while</tspan><tspan fill="#d4d4d4"> (</tspan><tspan fill="#dcdcaa">DoSomeJob</tspan><tspan fill="#d4d4d4">()) {</tspan></text>
<text style="white-space:pre" x="12" y="558"><tspan fill="#6e7681">29 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">zoop</tspan><tspan fill="#d4d4d4"> += </tspan><tspan fill="#b5cea8">1</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="577"><tspan fill="#6e7681">30 </tspan><tspan fill="#d4d4d4"> }</tspan></text>
<text style="white-space:pre" x="12" y="596"><tspan fill="#6e7681">31 </tspan></text>
<text style="white-space:pre" x="12" y="615"><tspan fill="#6e7681">32 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#c586c0">return</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">var</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="634"><tspan fill="#6e7681">33 </tspan><tspan fill="#d4d4d4">}</tspan></text>
<text style="white-space:pre" x="12" y="653"><tspan fill="#6e7681">34 </tspan></text>
<text style="white-space:pre" x="12" y="672"><tspan fill="#6e7681">35 </tspan><tspan fill="#569cd6">fn</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">Main</tspan><tspan fill="#d4d4d4">() -&gt; </tspan><tspan fill="#4ec9b0">i32</tspan><tspan fill="#d4d4d4"> {</tspan></text>
<text style="white-space:pre" x="12" y="691"><tspan fill="#6e7681">36 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#4ec9b0">str</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#ce9178">"Hello world"</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="710"><tspan fill="#6e7681">37 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">hex</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#b5cea8">0xABCDEF1234567890</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="729"><tspan fill="#6e7681">38 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">bin</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#b5cea8">0b0000111001010010</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="748"><tspan fill="#6e7681">39 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">dec</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#b5cea8">123456789012345678</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="767"><tspan fill="#6e7681">40 </tspan></text>
<text style="white-space:pre" x="12" y="786"><tspan fill="#6e7681">41 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">big</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">Core</tspan><tspan fill="#d4d4d4">.</tspan><tspan fill="#dcdcaa">Int</tspan><tspan fill="#d4d4d4">(</tspan><tspan fill="#b5cea8">1024</tspan><tspan fill="#d4d4d4">) = </tspan><tspan fill="#b5cea8">1234</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="805"><tspan fill="#6e7681">42 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">aaa</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">auto</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#ce9178">"Carbon"</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="824"><tspan fill="#6e7681">43 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#569cd6">let</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#9cdcfe">view</tspan><tspan fill="#d4d4d4">: </tspan><tspan fill="#4ec9b0">str</tspan><tspan fill="#d4d4d4"> = </tspan><tspan fill="#ce9178">"Carbon"</tspan><tspan fill="#d4d4d4">;</tspan></text>
<text style="white-space:pre" x="12" y="843"><tspan fill="#6e7681">44 </tspan></text>
<text style="white-space:pre" x="12" y="862"><tspan fill="#6e7681">45 </tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#c586c0">return</tspan><tspan fill="#d4d4d4"> </tspan><tspan fill="#dcdcaa">Procedure</tspan><tspan fill="#d4d4d4">();</tspan></text>
<text style="white-space:pre" x="12" y="881"><tspan fill="#6e7681">46 </tspan><tspan fill="#d4d4d4">}</tspan></text>
</g></svg>
Reference in New Issue View Git Blame Copy Permalink