mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-10-02 14:15:50 +01:00
Chandler CarruthandJon Meow
3512c2218f
Only change is to update the path to the fuzzer build extension. Original main commit message: > Add an initial parser library. (#30) > > This library builds a parse tree, very similar to a concrete syntax > tree. There are no semantics here, simply introducing the basic > syntactic structure. > > The current focus has been on the APIs and the data structures used to > represent the parse tree, and not on the actual code doing the > parsing. The code doing the parsing tries to be reasonably efficient > and reasonably easy to understand recursive descent parser. But there > is likely much that can be done to improve this code path. A notable > area where very little thought has been given yet are emitting good > diagnostics and doing good recovery in the event of parse errors. > > Also, this code does not try to match the current under-discussion > grammar closely. It is only partial and reflects discussions from some > time ago. It should be updated incrementally to reflect the current > expected grammar. > > The data structure used for the parse tree is unusual. The first > constraint is that there is a precise one-to-one correspondence > between the tokens produced by the lexer and the nodes in the parse > tree. Every token results in exactly one node. In that way, the parse > tree can be thought of as merely shaping the token stream into a tree. > > Each node is also represented with a fixed set of data that is densely > packed. Combined with the exact relationship to tokens, this allows us > to fully allocate the parse tree's storage, and to use a dense array > rather than a pointer-based tree structure. > > The tree structure itself is implicitly defined by tracking the size > of each subtree rooted at a particular node. See the code comments for > more details (and I'm happy to add more comments where necessary). The > goal is to minimize both the allocations (one), the working set size > of the tree as a whole, and optimize common iteration patterns. The > tree is stored in postorder. This allows depth-first postorder > iteration as well as topological iteration by walking in reverse. > > Building the parse tree in postorder is a natural consequence of the > grammar being LR rather than LL, which is a consequence of supporting > infix operators. > > As with the Lexer, the parser supports an API for operating on the > parse tree, as well as the ability to print the tree in both > a human-readable and machine-readable format (YAML-based). It includes > significant unit tests and a fuzz tester. The fuzzer's corpus will be > in a follow-up commit. > > This is the largest chunk of code already written by several of us > prior to open sourcing. (There are a few more pieces, but they are > significantly smaller and less interesting.) If there are major things > that folks would like to see happen here, it may make sense to move > them into issues for tracking. I have tried to update the code to > follow the style guidelines, but apologies if I missed anything, just > let me know. We also have issues #19 and #29 to track things that > already came up with the lexer. Co-authored-by: Jon Meow <46229924+jonmeow@users.noreply.github.com>
70 lines
2.4 KiB
C++
70 lines
2.4 KiB
C++
// Part of the Carbon Language project, under the Apache License v2.0 with LLVM
|
|
// Exceptions. See /LICENSE for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
|
|
#ifndef PARSER_PARSE_NODE_KIND_H_
|
|
#define PARSER_PARSE_NODE_KIND_H_
|
|
|
|
#include <cstdint>
|
|
#include <iterator>
|
|
|
|
#include "llvm/ADT/StringRef.h"
|
|
|
|
namespace Carbon {
|
|
|
|
// A class wrapping an enumeration of the different kinds of nodes in the parse
|
|
// tree.
|
|
//
|
|
// Rather than using a raw enumerator for each distinct kind of node produced by
|
|
// the parser, we wrap the enumerator in a class to expose a more rich API
|
|
// including bidirectional mappings to string spellings of the different kinds
|
|
// and any relevant classification.
|
|
//
|
|
// Instances of this type should always be created using the `constexpr` static
|
|
// member functions. These instances are designed specifically to be usable in
|
|
// `case` labels of `switch` statements just like an enumerator would.
|
|
class ParseNodeKind {
|
|
public:
|
|
// The formatting for this macro is weird due to a `clang-format` bug. See
|
|
// https://bugs.llvm.org/show_bug.cgi?id=48320 for details.
|
|
#define CARBON_PARSE_NODE_KIND(Name) \
|
|
static constexpr auto Name()->ParseNodeKind { return KindEnum::Name; }
|
|
#include "parser/parse_node_kind.def"
|
|
|
|
// The default constructor is deleted as objects of this type should always be
|
|
// constructed using the above factory functions for each unique kind.
|
|
ParseNodeKind() = delete;
|
|
|
|
auto operator==(const ParseNodeKind& rhs) const -> bool {
|
|
return kind == rhs.kind;
|
|
}
|
|
auto operator!=(const ParseNodeKind& rhs) const -> bool {
|
|
return kind != rhs.kind;
|
|
}
|
|
|
|
// Gets a friendly name for the token for logging or debugging.
|
|
[[nodiscard]] auto GetName() const -> llvm::StringRef;
|
|
|
|
private:
|
|
enum class KindEnum : uint8_t {
|
|
#define CARBON_PARSE_NODE_KIND(Name) Name,
|
|
#include "parser/parse_node_kind.def"
|
|
};
|
|
|
|
constexpr ParseNodeKind(KindEnum k) : kind(k) {}
|
|
|
|
// Enable conversion to our private enum, including in a `constexpr` context,
|
|
// to enable usage in `switch` and `case`. The enum remains private and
|
|
// nothing else should be using this.
|
|
explicit constexpr operator KindEnum() const { return kind; }
|
|
|
|
KindEnum kind;
|
|
};
|
|
|
|
// We expect the parse node kind to fit compactly into 8 bits.
|
|
static_assert(sizeof(ParseNodeKind) == 1, "Kind objects include padding!");
|
|
|
|
} // namespace Carbon
|
|
|
|
#endif // PARSER_PARSE_NODE_KIND_H_
|