carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-05 16:01:04 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Bracket the tokenized buffer output. (#2446)

Browse Source 
This makes TokenizedBuffer more consistent with ParseTree and SemanticsIR, which also wrap with [] to produce a sequence value.

It also makes it possible in driver.cpp to just prefix the line with a name, so it ends up with:

var_name: [
  (content)
]
Noticed this due to bracketing comments on #2443 and trying to think of better answers. With this, we can also say that the [] bracket a variable.
This commit is contained in:
Jon Ross-Perkins
2022-12-07 11:22:33 -08:00
committed by GitHub
parent 5f85822caa
commit 04f0288cd2
7 changed files with 118 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files
toolchain/driver/driver_test.cpp
+23 -23
View File
@@ -133,29 +133,29 @@ TEST(DriverTest, DumpTokens) {
auto tokenized_text = test_output_stream.TakeStr();
EXPECT_THAT(Yaml::Value::FromText(tokenized_text),
ElementsAre(Yaml::MappingValue{
{"token", Yaml::MappingValue{{"index", "0"},
{"kind", "Identifier"},
{"line", "1"},
{"column", "1"},
{"indent", "1"},
{"spelling", "Hello"},
{"identifier", "0"},
{"has_trailing_space", "true"}}},
{"token", Yaml::MappingValue{{"index", "1"},
{"kind", "Identifier"},
{"line", "1"},
{"column", "7"},
{"indent", "1"},
{"spelling", "World"},
{"identifier", "1"},
{"has_trailing_space", "true"}}},
{"token", Yaml::MappingValue{{"index", "2"},
{"kind", "EndOfFile"},
{"line", "1"},
{"column", "12"},
{"indent", "1"},
{"spelling", ""}}}}));
ElementsAre(Yaml::SequenceValue{
Yaml::MappingValue{{"index", "0"},
{"kind", "Identifier"},
{"line", "1"},
{"column", "1"},
{"indent", "1"},
{"spelling", "Hello"},
{"identifier", "0"},
{"has_trailing_space", "true"}},
Yaml::MappingValue{{"index", "1"},
{"kind", "Identifier"},
{"line", "1"},
{"column", "7"},
{"indent", "1"},
{"spelling", "World"},
{"identifier", "1"},
{"has_trailing_space", "true"}},
Yaml::MappingValue{{"index", "2"},
{"kind", "EndOfFile"},
{"line", "1"},
{"column", "12"},
{"indent", "1"},
{"spelling", ""}}}));
// Check that the subcommand dispatch works.
EXPECT_TRUE(driver.RunFullCommand({"dump", "tokens", test_file_path}));