Files
carbon-lang/toolchain/lexer/token_kind.cpp
T
Chandler CarruthandRichard Smith cf42cc8498 Make TokenKind API inlinable. (#3142)
Previously, accessing any property of a token's kind required both
a function call to an out-of-line method and loading the relevant data
out of a table. These method calls were often roughly as expensive as
the load out of the table -- they would push and pop registers, and so
would touch a stack cache line not unlike the table cache line.

We can inline them in a way that carefully leaves the table definitions
in the single TU so we don't get tons of copies of data that have to be
merged by the linker. But the access to the table can be in an inline
function that allows the call overhead to evaporate and these to turn
into just table loads in the callers as well.

While in theory we could rely on forms of LTO to acheive this, it
doesn't seem worth relying on that. This is an easy win with very little
cost in reality.

Several of the lexer benchmarks improve by 1% or 2% from removing the
function call overhead. It's not a huge win, but it's nice.

Even more nice is that the profile is cleaned up significantly, clearly
focusing on the core functions where time is spent in lexing and
parsing.

This also pulls my big-picture benchmark of 10mloc file down to under
4.3s to lex and parse excitingly. Basically, we're above 2.3mloc/s
lexing and parsing. Not *quite* the 10mloc/s that I'm hoping for, but
still progress.

---------

Co-authored-by: Richard Smith <richard@metafoo.co.uk>
2023-08-25 23:07:23 +00:00

83 lines
2.6 KiB
C++

// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
// Exceptions. See /LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
#include "toolchain/lexer/token_kind.h"
namespace Carbon {
CARBON_DEFINE_ENUM_CLASS_NAMES(TokenKind) = {
#define CARBON_TOKEN(TokenName) CARBON_ENUM_CLASS_NAME_STRING(TokenName)
#include "toolchain/lexer/token_kind.def"
};
constexpr bool TokenKind::IsSymbol[] = {
#define CARBON_TOKEN(TokenName) false,
#define CARBON_SYMBOL_TOKEN(TokenName, Spelling) true,
#include "toolchain/lexer/token_kind.def"
};
constexpr bool TokenKind::IsGroupingSymbol[] = {
#define CARBON_TOKEN(TokenName) false,
#define CARBON_OPENING_GROUP_SYMBOL_TOKEN(TokenName, Spelling, ClosingName) \
true,
#define CARBON_CLOSING_GROUP_SYMBOL_TOKEN(TokenName, Spelling, OpeningName) \
true,
#include "toolchain/lexer/token_kind.def"
};
constexpr bool TokenKind::IsOpeningSymbol[] = {
#define CARBON_TOKEN(TokenName) false,
#define CARBON_OPENING_GROUP_SYMBOL_TOKEN(TokenName, Spelling, ClosingName) \
true,
#include "toolchain/lexer/token_kind.def"
};
constexpr TokenKind TokenKind::ClosingSymbol[] = {
#define CARBON_TOKEN(TokenName) Error,
#define CARBON_OPENING_GROUP_SYMBOL_TOKEN(TokenName, Spelling, ClosingName) \
ClosingName,
#include "toolchain/lexer/token_kind.def"
};
constexpr bool TokenKind::IsClosingSymbol[] = {
#define CARBON_TOKEN(TokenName) false,
#define CARBON_CLOSING_GROUP_SYMBOL_TOKEN(TokenName, Spelling, OpeningName) \
true,
#include "toolchain/lexer/token_kind.def"
};
constexpr TokenKind TokenKind::OpeningSymbol[] = {
#define CARBON_TOKEN(TokenName) Error,
#define CARBON_CLOSING_GROUP_SYMBOL_TOKEN(TokenName, Spelling, OpeningName) \
OpeningName,
#include "toolchain/lexer/token_kind.def"
};
constexpr bool TokenKind::IsOneCharSymbol[] = {
#define CARBON_TOKEN(TokenName) false,
#define CARBON_ONE_CHAR_SYMBOL_TOKEN(TokenName, Spelling) true,
#include "toolchain/lexer/token_kind.def"
};
constexpr bool TokenKind::IsKeyword[] = {
#define CARBON_TOKEN(TokenName) false,
#define CARBON_KEYWORD_TOKEN(TokenName, Spelling) true,
#include "toolchain/lexer/token_kind.def"
};
constexpr llvm::StringLiteral TokenKind::FixedSpelling[] = {
#define CARBON_TOKEN(TokenName) "",
#define CARBON_SYMBOL_TOKEN(TokenName, Spelling) Spelling,
#define CARBON_KEYWORD_TOKEN(TokenName, Spelling) Spelling,
#include "toolchain/lexer/token_kind.def"
};
constexpr int8_t TokenKind::ExpectedParseTreeSize[] = {
#define CARBON_TOKEN(Name) 1,
#define CARBON_TOKEN_WITH_VIRTUAL_NODE(size) 2,
#include "toolchain/lexer/token_kind.def"
};
} // namespace Carbon