Files
carbon-lang/toolchain/docs
Dana Jansens d58b523a5e Add INCLUDE-FILE: and --custom-core for file tests to specify a minimal prelude library (#5080)
The INCLUDE-FILE option is only used in the toolchain tests for now. If
specified in a file test, the given file path is added to the test's
arguments. For toolchain tests this makes the file's package available
to the test. The `--custom-core` command line flag is added to the
driver, which avoids adding the production `Core` package to the command
line. Together, these allow a test to provide their own minimal `Core`
package.

For example, this would replace `Core` with the package and prelude in
`facet_types.carbon`.
```
// INCLUDE-FILE: toolchain/testing/min_prelude/facet_types.carbon
// EXTRA-ARGS: --custom-core
```

To support this:
* //testing knows how to parse INCLUDE-FILE out of the header of a test
file.
* //testing adds the file to the virtual file system, and includes it in
the test's arguments.
* //toolchain/driver grows the --custom-core command line flag to avoid
loading the production `Core` package.

Tests that were creating their own minimal prelude to define BitAnd on
types are now pointed to
toolchain/testing/min_prelude/facet_types.carbon as the prelude. They no
longer need to `import Core` in each test as a result.

Such tests are no longer `no_prelude`, but instead have their own
prelude. So they are moved to a `min_prelude` subdirectory.

Closes #5076
2025-03-10 19:44:50 +00:00
..

Toolchain architecture

Table of contents

Goals

The toolchain represents the production portion of Carbon. At a high level, the toolchain's top priorities are:

  • Correctness.
  • Quality of generated code, including performance.
  • Compilation performance.
  • Quality of diagnostics for incorrect or questionable code.

TODO: Add an expanded document that details the goals and priorities and link to it here.

High-level architecture

The main components are:

Design patterns

A few common design patterns are:

  • Distinct steps: Each step of processing produces an output structure, avoiding callbacks passing data between structures.

    • For example, the parser takes a Lex::TokenizedBuffer as input and produces a Parse::Tree as output.

    • Performance: It should yield better locality versus a callback approach.

    • Understandability: Each step has a clear input and output, versus callbacks which obscure the flow of data.

  • Vectorized storage: Data is stored in vectors and flyweights are passed around, avoiding more typical heap allocation with pointers.

    • For example, the parse tree is stored as a llvm::SmallVector<Parse::Tree::NodeImpl> indexed by Parse::Node which wraps an int32_t.

    • Performance: Vectorization both minimizes memory allocation overhead and enables better read caching because adjacent entries will be cached together.

  • Iterative processing: We rely on state stacks and iterative loops for parsing, avoiding recursive function calls.

    • For example, the parser has a Parse::State enum tracked in state_stack_, and loops in Parse::Tree::Parse.

    • Scalability: Complex code must not cause recursion issues. We have experience in Clang seeing stack frame recursion limits being hit in unexpected ways, and non-recursive approaches largely avoid that risk.

See also Idioms for abbreviations and more implementation techniques.

Adding features

We have a walkthrough for adding features.