Files
carbon-lang/docs/design/lexical_conventions
Richard Smith ee7a108da4 Incorporate proposals #142 and #143 into the design
Add design text based on the contents of two proposals:

#142 Unicode source files
#143 Numeric literals
2020-11-03 16:39:05 -08:00
..

Lexical conventions

Table of contents

TODO

This is a skeletal design, added to support the overview. It should not be treated as accepted by the core team; rather, it is a placeholder until we have more time to examine this detail. Please feel welcome to rewrite and update as appropriate.

See PR 17 for context -- that proposal may replace this.

Lexical elements

The first stage of processing a source file is the division of the source file into lexical elements.

A lexical element is one of the following:

The sequence of lexical elements is formed by repeatedly removing the longest initial sequence of characters that forms a valid lexical element.