carbon-language/carbon-lang
1
0
Fork 0
mirror of https://github.com/carbon-language/carbon-lang.git synced 2026-10-04 22:02:52 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

Move information on the token associated with a parse node from the .def file into the typed node. (#4001)

Browse Source 
Instead of tracking the token associated with a parse node in the `.def`
file macro, track it on the typed node instead. List the token as a
field inside the node structure to show the order of the token relative
to the other components of the grammar production, and to allow the
token index to be accessed when the node is extracted.

Remove the corresponding information from the `.def` file, leaving
behind just a list of parse node kinds in the majority of cases.

This also removes the checking of the token kind associated with a parse
node in the case where the parse node has errors. Previously we had a
flag on the node kind to indicate whether we should check this, but per
[discord
discussion](https://discord.com/channels/655572317891461132/655578254970716160/1246214418979881052),
we have decided to remove this.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This commit is contained in:
Richard Smith
2024-05-31 23:11:51 +00:00
committed by GitHub
co-authored by Jon Ross-Perkins
parent 5bb318cae6
commit 3c01ee69ed
14 changed files with 921 additions and 927 deletions
Download Patch File Download Diff File Expand all files Collapse all files
toolchain/parse/node_kind.cpp
+3 -24
View File
@@ -4,7 +4,6 @@
#include "toolchain/parse/node_kind.h"
#include "common/check.h"
#include "llvm/ADT/StringExtras.h"
#include "toolchain/parse/typed_nodes.h"
@@ -37,30 +36,10 @@ CARBON_DEFINE_ENUM_CLASS_NAMES(NodeKind) = {
#include "toolchain/parse/node_kind.def"
};
auto NodeKind::CheckMatchesTokenKind(Lex::TokenKind token_kind, bool has_error)
-> void {
static constexpr Lex::TokenKind TokenIfValid[] = {
#define CARBON_IF_VALID(LexTokenKind) LexTokenKind
#define CARBON_PARSE_NODE_KIND_WITH_TOKEN(Name, LexTokenKind) \
Lex::TokenKind::LexTokenKind,
// Check that each typed node defines a `token` member.
#define CARBON_PARSE_NODE_KIND(Name) \
static_assert(requires(Name node) { node.token; });
#include "toolchain/parse/node_kind.def"
};
static constexpr Lex::TokenKind TokenIfError[] = {
#define CARBON_IF_VALID(LexTokenKind) Error
#define CARBON_PARSE_NODE_KIND_WITH_TOKEN(Name, LexTokenKind) \
Lex::TokenKind::LexTokenKind,
#include "toolchain/parse/node_kind.def"
};
Lex::TokenKind expected_token_kind =
has_error ? TokenIfError[AsInt()] : TokenIfValid[AsInt()];
// Error indicates that the kind shouldn't be enforced.
CARBON_CHECK(Lex::TokenKind::Error == expected_token_kind ||
token_kind == expected_token_kind)
<< "Created parse node with NodeKind " << *this << " and has_error "
<< has_error << " for lexical token kind " << token_kind
<< ", but expected token kind " << expected_token_kind;
}
auto NodeKind::has_bracket() const -> bool {
return definition().has_bracket();