mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-09-24 20:40:11 +01:00
Jon Ross-Perkins
13d5fe9eed
As part of using the evolution process with the toolchain, alternatives should be in proposals. This proposal migrates existing alternatives here. Assisted-by: Google Antigravity with Gemini 3 Flash
1.0 KiB
1.0 KiB
Lex
Table of contents
Overview
Lexing converts input source code into tokenized output. Literals, such as string literals, have their value parsed and form a single token at this stage.
Bracket matching
The lexer handles matching for (), [], and {}. When a bracket lacks a
match, it will insert a "recovery" token to produce a match. As a consequence,
the lexer's output should always have matched brackets, even with invalid code.
While bracket matching could use hints such as contextual clues from indentation, that is not yet implemented.