mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-11-17 12:18:10 +00:00
Carbon currently requires a comment to be the only non-whitespace on its line. A `//` comment that follows other content on a line, called a _trailing comment_, is a lexer error. This proposal removes that restriction, allowing a comment to follow other content on a line. Everything else about comments is unchanged: a comment still begins with `//`, still requires whitespace after the `//`, and still runs to the end of the line. Carbon continues to provide only line comments; no block or intra-line comments are added. Three observations motivate the change. First, trailing comments are well suited to short _annotations_ attached to a specific entity or value on a line. Second, the lexer design now makes it trivial to lex trailing comments, and in fact requires extra logic and potentially cost to reject them. Third, C++ code routinely uses trailing comments, so allowing them lets Carbon carry the layout of migrated code over directly, rather than reworking each comment to read well in a different structure. Implementation notes (beyond the proposal's design): Keeping trailing comments cheap to lex required a few supporting changes, all of which keep the cost off the lexer's hot path: - The lexer already dispatches `//` to comment lexing wherever it appears, so classifying a comment as trailing is a single O(1) check of whether the `//` is the line's first non-whitespace (`start + indent`). The hot comment path is otherwise unchanged. - That check relies on each line's recorded indentation being its real leading whitespace. Multi-line string literals previously recorded the column where the literal opened for the lines they span; they now record the true (closing-delimiter) indentation instead. - Parser error recovery (`SkipPastLikelyEnd`) had relied on that opening-column indentation to keep tokens following a multi-line string literal attached to the same construct. It now reconstructs that relationship directly by consulting the line on which the literal opened, including when other tokens follow the closing delimiter (such as `''' + "more"`). This is on the cold recovery path. - `CommentData` records the trailing bit in the high bit of its length field, keeping it at 8 bytes. Assisted-by: Claude Code
63 lines
3.2 KiB
Plaintext
63 lines
3.2 KiB
Plaintext
// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
|
|
// Exceptions. See /LICENSE for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
//
|
|
// For multiline string literals, force autoupdates to the end so that we don't
|
|
// accidentally mix diagnostics with the literal.
|
|
//
|
|
// AUTOUPDATE
|
|
// TIP: To test this file alone, run:
|
|
// TIP: bazel test //toolchain/testing:file_test --test_arg=--file_tests=toolchain/lex/testdata/multiline_string_literals.carbon
|
|
// TIP: To dump output, run:
|
|
// TIP: bazel run //toolchain/testing:file_test -- --dump_output --file_tests=toolchain/lex/testdata/multiline_string_literals.carbon
|
|
|
|
// --- fail_indent_mismatch.carbon
|
|
|
|
'''
|
|
a
|
|
'''
|
|
|
|
// --- fail_quotes.carbon
|
|
|
|
"""
|
|
a
|
|
"""
|
|
|
|
// --- introducer_trailing_comment.carbon
|
|
|
|
// A trailing comment may follow the file type indicator on the introducer line.
|
|
// It is treated like trailing whitespace: it is part of the literal's introducer
|
|
// and produces no comment token, and it may contain `'`, `#`, and `"` even
|
|
// though the file type indicator may not.
|
|
//
|
|
// TODO: Surface this comment through the lexer's comment records.
|
|
var s: String = '''py // A trailing comment with ', #, and " in it.
|
|
content
|
|
''';
|
|
|
|
// --- AUTOUPDATE-SPLIT
|
|
|
|
// CHECK:STDERR: fail_indent_mismatch.carbon:3:1: error: indentation does not match that of the closing `'''` in multi-line string literal [MismatchedIndentInString]
|
|
// CHECK:STDERR: a
|
|
// CHECK:STDERR: ^
|
|
// CHECK:STDERR:
|
|
// CHECK:STDERR: fail_quotes.carbon:2:1: error: use `'''` delimiters for a multi-line string literal, not `"""` [MultiLineStringWithDoubleQuotes]
|
|
// CHECK:STDERR: """
|
|
// CHECK:STDERR: ^
|
|
// CHECK:STDERR:
|
|
// CHECK:STDOUT: - filename: fail_indent_mismatch.carbon
|
|
// CHECK:STDOUT: tokens:
|
|
// CHECK:STDOUT: - { index: 1, kind: "StringLiteral", line: {{ *}}2, column: 3, indent: 3, spelling: "'''\na\n '''", value: "a\n", has_leading_space: true }
|
|
// CHECK:STDOUT: - filename: fail_quotes.carbon
|
|
// CHECK:STDOUT: tokens:
|
|
// CHECK:STDOUT: - { index: 1, kind: "StringLiteral", line: {{ *}}2, column: 1, indent: 1, spelling: "\"\"\"\na\n\"\"\"", value: "a\n", has_leading_space: true }
|
|
// CHECK:STDOUT: - filename: introducer_trailing_comment.carbon
|
|
// CHECK:STDOUT: tokens:
|
|
// CHECK:STDOUT: - { index: 1, kind: "Var", line: {{ *}}8, column: 1, indent: 1, spelling: "var", has_leading_space: true }
|
|
// CHECK:STDOUT: - { index: 2, kind: "Identifier", line: {{ *}}8, column: 5, indent: 1, spelling: "s", identifier: 0, has_leading_space: true }
|
|
// CHECK:STDOUT: - { index: 3, kind: "Colon", line: {{ *}}8, column: 6, indent: 1, spelling: ":" }
|
|
// CHECK:STDOUT: - { index: 4, kind: "Identifier", line: {{ *}}8, column: 8, indent: 1, spelling: "String", identifier: 1, has_leading_space: true }
|
|
// CHECK:STDOUT: - { index: 5, kind: "Equal", line: {{ *}}8, column: 15, indent: 1, spelling: "=", has_leading_space: true }
|
|
// CHECK:STDOUT: - { index: 6, kind: "StringLiteral", line: {{ *}}8, column: 17, indent: 1, spelling: "'''py // A trailing comment with ', #, and \" in it.\n content\n '''", value: "content\n", has_leading_space: true }
|
|
// CHECK:STDOUT: - { index: 7, kind: "Semi", line: {{ *}}10, column: 6, indent: 3, spelling: ";" }
|