On invalid parse nodes, if the token may vary, allow any token. (#3484)

Note #3486 rewrites the macro behavior, and is already approved: so this
PR is only for the changed enforcement during error. Also, #3493 already
changed several things to allow any token while this PR was awaiting
review, but this still changes enforcement for `For` and `If`.

This was brought up on
[#toolchain](https://discord.com/channels/655572317891461132/655578254970716160/1182066616456970251),
and I think this any-on-error approach gets at least some support. We
could try setting it to the introducer, but it's quite possible we want
it to be something like the token which led to the parse error, rather
than a static token. That leads to a conclusion that, most typically,
we'll expect arbitrary tokens when error conditions may lead to tokens
which aren't the expected token.

A couple related, recent `CARBON_IF_ERROR` crash fixes can be found in
#3404 and #3424. Something like #3404 would've been needed regardless
because `namespace` didn't have `CARBON_IF_ERROR` before, although I
might've missed the underlying issue with declarations because only
`namespace` had a relevant test (that is, if #3404 had added
`CARBON_ANY_TOKEN_ON_ERROR`, I wouldn't have had a crash in #3462).
#3424 would've been avoided with this change because there was a
`CARBON_IF_ERROR`, and it was just too restrictive.
This commit is contained in:
Jon Ross-Perkins
2023-12-13 20:28:48 +00:00
committed by GitHub
parent 7c7afc9e32
commit 550559e30e
4 changed files with 105 additions and 71 deletions
+8 -5
View File
@@ -50,6 +50,11 @@ auto NodeKind::child_count() const -> int32_t {
// NOLINTNEXTLINE(readability-function-size): It's hard to extract macros.
auto CheckNodeMatchesLexerToken(NodeKind node_kind, Lex::TokenKind token_kind,
bool has_error) -> void {
// As a special-case, a placeholder node may correspond to any lexer token.
if (node_kind == NodeKind::Placeholder) {
return;
}
switch (node_kind) {
// Use `CARBON_LOG CARBON_ANY_TOKEN` to discover which combinations happen
// in practice.
@@ -58,16 +63,14 @@ auto CheckNodeMatchesLexerToken(NodeKind node_kind, Lex::TokenKind token_kind,
<< " and has_error " << has_error << " for lexical token " \
<< token_kind << "\n";
#define CARBON_ANY_TOKEN return;
#define CARBON_TOKEN(Expected) \
if (token_kind == Lex::TokenKind::Expected) { \
return; \
}
#define CARBON_IF_ERROR(MatchActions) \
if (has_error) { \
MatchActions \
#define CARBON_ANY_TOKEN_ON_ERROR \
if (has_error) { \
return; \
}
#define CARBON_CASE(Name, MatchActions) \