mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-10-01 22:02:45 +01:00
Jon Ross-Perkins
13d5fe9eed
As part of using the evolution process with the toolchain, alternatives should be in proposals. This proposal migrates existing alternatives here. Assisted-by: Google Antigravity with Gemini 3 Flash
36 lines
1.0 KiB
Markdown
36 lines
1.0 KiB
Markdown
# Lex
|
|
|
|
<!--
|
|
Part of the Carbon Language project, under the Apache License v2.0 with LLVM
|
|
Exceptions. See /LICENSE for license information.
|
|
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
-->
|
|
|
|
<!-- toc -->
|
|
|
|
## Table of contents
|
|
|
|
- [Overview](#overview)
|
|
- [Bracket matching](#bracket-matching)
|
|
- [Alternatives considered](#alternatives-considered)
|
|
|
|
<!-- tocstop -->
|
|
|
|
## Overview
|
|
|
|
Lexing converts input source code into tokenized output. Literals, such as
|
|
string literals, have their value parsed and form a single token at this stage.
|
|
|
|
## Bracket matching
|
|
|
|
The lexer handles matching for `()`, `[]`, and `{}`. When a bracket lacks a
|
|
match, it will insert a "recovery" token to produce a match. As a consequence,
|
|
the lexer's output should always have matched brackets, even with invalid code.
|
|
|
|
While bracket matching could use hints such as contextual clues from
|
|
indentation, that is not yet implemented.
|
|
|
|
## Alternatives considered
|
|
|
|
- [Bracket matching in parser](/proposals/p6716.md#bracket-matching-in-parser)
|