Rich diagnostic rendering needs a layer underneath it that knows what
the attached terminal can do and can position styled text in two
dimensions. This adds that layer, both as the foundation the diagnostics
rendering work will build on and as something usable directly for
ordinary CLI output. Nothing depends on it yet, so it lands and is
reviewed on its own.
Four libraries, each with its own tests:
- `color`: a color, either one of the 16 named ANSI colors or a 24-bit
RGB value, and the escape sequences that select it at a given color
depth.
- `style`: colors plus text attributes, and the escapes that move a
terminal from one style to another.
- `capabilities`: what the terminal behind a stream supports, detected
from the environment.
- `buffer`: a grid of styled cells that layout code draws into and that
renders itself once.
Rationale for the design decisions lives in the headers, next to what it
explains. Four things are worth review attention in particular:
- The color detection precedence documented on `ChooseColorMode`. It
settles how a `--color` flag, `NO_COLOR`, `CLICOLOR`, `FORCE_COLOR`, and
the terminal itself interact. The policy is a pure function of those
inputs, so the whole table is tested without touching the process
environment.
- `Charset`, which decides whether any UTF-8 processing happens at all.
Column counts only follow from code points if the terminal agrees about
the encoding, so anything short of a locale naming UTF-8 is treated as
bytes.
- `Buffer` owning column accounting instead of its callers, which is
what keeps double-width characters, combining marks, and stray bytes
from misaligning everything after them.
- The API surface, which is held to operations that nothing else covers.
Junctions in line art come only from lines overlapping, and turning a
style on or off is spelled as a transition to or from the default style.
`terminal_benchmark` covers style transitions, full-screen rendering,
and text drawing. On an M-series laptop, rendering an 80x24 screen in
which every cell changes style costs about 14us with color off and
76-95us with it, and drawing a 40-column line of source costs about
140ns without UTF-8 processing and 394ns with it.
Assisted-by: Gemini and Claude
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Adds an algorithm to compute where to insert brackets to repair
bracketing mismatches during lexing. This takes indentation, as well as
a number of other cues, into account to predict where the brackets
should have gone. Detects when there is ambiguity between solutions and
makes no suggestion in that case. Reduces the problem by splitting on
properly bracketed top-level constructs, then uses a beam search to find
good candidate solutions quickly.
This includes both a fuzzer and an eval tool that can be used to
determine how well the algorithm fares against a given corpus of valid
Carbon code, by damaging it in various ways and seeing whether the
algorithm can correctly fix it. On all the eval modes, this algorithm
can correctly infer the positions for over 80% of lost brackets (and can
correctly restore 95+% of brackets in some modes), with low rates of
incorrect suggestions.
See added documentation for full details.
Assisted-by: Gemini via Antigravity, Claude via Claude Code
The main direction of this change is the edits to `destroy.carbon`
(matching in both prelude and min_prelude).
Previously there was a no-op blanket impl for `Destroy`, which hid all
missing implementations of `Destroy`. This does a few things:
- Sets up builtin aggregate destruction for struct and tuple types as
before, but also adds C++ class types and array types to the same
handling. (all as a TODO for actual implementation)
- Also maybe-unformed destruction, for now at least. (there's a chance I
may try a different approach on this, but the impl lookup wasn't working
as I'd hope in order to write it in code)
- Adds handlers for simple things that are easy to do in code: `type`,
`bool`, pointers. (because these are no-op destruction)
- Redirect `const T` destruction to `T` destruction.
This leaves as future issues:
- `partial T` destruction. (this can't be done similar to `const`
because it only works for non-`final` class types; I think `class`
definitions should just generate what's needed)
- Destruction of other prelude-provided types. (will probably come up as
we implement class destruction, that the adapted builtin type doesn't
implement `Destroy` -- but may end up special-casing that in a way that
moots it)
This moves the `&` operator from `facet_types.carbon` to
`convert.carbon` because more things need to handle type and now that
we're getting separate copy and destroy interfaces. It should be
low-cost (an interface and builtin) so hopefully this is the right
balance for complexity and re-use.
A few tests are also edited in order to focus them more on what they
intend to test, and avoid a `Destroy` dependency.
This is the first step to having Clang's runtime libraries fully
available for the Carbon toolchain. This PR focuses on the lowest level
runtimes, the CRT files and the builtins library.
The goal is to intercept Clang runs where it needs these
target-dependent pieces to be available, and build them on demand using
our Clang-running infrastructure. This avoids most of the subprocess
overhead, but there is still some due to missing features in Clang.
This requires exporting the sources for these runtimes from the Bazel
build, and installing them in our target-independent resource directory.
We then build a simplified "build" of these sources within the
`ClangRunner` itself to produce the specific artifacts and layout
expected by Clang.
It also required fixing our use of Clang on macOS to have a default
system root in order to successfully compile or link.
It also required cleaning up how the `ClangRunner` used target
information more generally -- instead of taking the target as
a constructor parameter, it manages its target internally and relies on
the Clang target-specifying command line flags.
I looked at whether we could split this into another layer separate from
the `ClangRunner`, but that proved frustratingly difficult to manage.
While we support building these on-demand as part of a detected link,
that doesn't seem feasible as we don't have the necessary separation
between compilation runs of Clang and link runs of Clang. However,
I have tried to factor the internals to provide as clear of separation
as I could across these.
I have also created a stand-alone subcommand to directly build the
runtimes which allows for easy testing. It also supports building them
into a specific directory, and that directory can in turn be passed to
a Clang invocation. This is designed to work both at the API level with
`ClangRunner` and at the subcommand level.
Currently, the only part of the commandline that is detected and
forwarded to the runtimes build is the target. Eventually, the plan is
to expand this so that we can build a maximally tailored set of runtimes
for a given compilation.
The other big TODO here is to actually implement caching storage of
these runtimes so they aren't built on every execution. Right now, this
uses a somewhat hack-y build of a temporary directory, but this isn't
expected to be suitable long-term. Building these runtimes on *every*
link makes those commands take approximately 15 seconds with an ASan
build like our default development build, and just over 2 seconds in an
optimized build. Because of this, I've kept all of this disabled by
default for now. The goal is that once caching and some other
improvements land, we can enable this by default.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This builds on #5212 which is adding ArgAndKind. This further modifies
CARBON_KIND_SWITCH support so that we can use it with ArgAndKind in
addition to Inst. That creates a quirk where it's easier if ArgAndKind
provides `kind` as an accessor instead of a data member, so I'm just
switching it to a class.
In order to format comments, it's helpful if they're tracked. This
tracks them separately from tokens in order to avoid interfering with
parse; it'd be inconvenient if comment tokens could show up in arbitrary
locations, albeit possible to support.
This additionally extracts out the TokenIterator support into a template
in order to generally have it available for IndexBase types. I'm only
adding it for CommentInfo, not sure if we'll want it elsewhere, but this
structure still felt like a good fit.
Demo site: https://jonmeow.carbon-lang.dev/
I'm trying to keep work under the `/website` subdirectory so that the
misc files don't interfere with unrelated views of the repository. The
`prebuild.py` script does some work to move things around and add
frontmatter, helping the jekyll generation.
I'm using the "just-the-docs" theme because I think it's a decent match
for what we want, and getting jekyll up and running with it wasn't too
difficult. Note #1526 proposed using Docusaurus; I started out there,
but was having trouble getting it working with newer versions. The
plugins in particular I got stuck trying to make work, which sent me
looking for options that we could have working with less customization.
I do lean towards jekyll though, because it's what GH uses so hopefully
we can get a more consistent experience.
Having a website has been approved for a while under #1492, but hasn't
been a priority. I'm mainly doing this because I want to just be able to
point people to carbon-lang.dev and have easy links that way.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This injects a customization point for hashtable-specific equality
testing that the key context uses by default. While this is rarely
needed, there are LLVM types where it is necessary and it seems a good
general tool to have to avoid unnecessary complexity from custom key
contexts when a simple customization of equality is all that is
required.
This also adds a CRTP mixin for implementing a common pattern of key
contexts where the context provides translation of some key types into
another type, potentially using state. Rather than having to implement
the entire key context API, code can derive from this template and
simply provide a set of overloads for the types it wants to translate.
Any key types used which can be passed to one of those overloads will
get translated before following the same logic as the default key
context. While this updates the only usage so far of this pattern, a
subsequent PR will add several more users making the pattern worth
abstracting here.
The hash table design is heavily based on Abseil's ["Swiss
Tables"][swiss-tables] design. It uses an array of bytes storing
metadata about each entry and an array of entries where each is a pair
of key and value. The metadata byte consists of 7-bits of hash of the
key (distinct from the bits used to index the table), and one bit
indicating the presence of a special entry -- either empty or deleted.
[swiss-tables]: https://abseil.io/about/design/swisstables
There are a large range of optimizations and other nuanced aspects of
this hash table design and implementation, a good point to understand
that context is `raw_hashtable.h` which has an overview of the design
and references to various other files for relevant details.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
This makes it possible to specify both deduced and regular parameters on types. It reorganizes the handling of parameter lists in order to allow more reuse of code in this approach. Both functions and types use the new DeclarationNameAndParams handling. Overall the goal here is to take advantage of commonality in structure.
Regarding destructors, the likely approach would be to use ParameterListAsDeduced directly because `destructor` is a keyword with no declaration name and no regular parameters.
This switches to single list storage of SemanticsNode. The driving motivation behind this is to simplify cross-references within a given IR. Types of nodes will frequently refer to other blocks. This causes a significant increase in the number of cross-references, which can become difficult to manage (and reason about). By reducing to a single list of nodes, cross-references are only needed when crossing IR boundaries.
Because cross-references now only have 2 things to track (IR and index), they can be a regular SemanticsNode and don't need further indirection. This wasn't motivating, but feels like it reinforces the simplification.
Note this isn't being used to deduplicate nodes, at least right now. That could lead to difficult-to-update situations, but also most nodes are associated with the underlying ParseTree::Node in order to track sources for diagnostics; as a consequence, nodes representing equal text in different source locations wouldn't be the same node. There may be future opportunities here, discussed with @zygoloid, but no action is taken at present.
We may eventually want to switch the storage of NodeBlocks to have `[start, end)` ranges instead of individual numbers, but I'm leaving that alone for now.
As an aside, I noticed I was accidentally overloading the copy constructor on SemanticsIR. I've added some disambiguation on that, but am not deleting the copy constructor per style advice (even though the type should never be copied due to storage size).
codespell tries to change `CrossReference -> cross-reference` so disabling it there.
codespell now sees "falsy" as a mis-spelling of either "false" or "falsely"; adding it since I think this it's occasionally used this way in programming. e.g., https://developer.mozilla.org/en-US/docs/Glossary/Falsy
Adjusts to use the new check-copyright support for lines starting with a dash.
Add support for nominal (or "named") classes with encapsulation. Inheritance will be in a later proposal. Here is an example of the proposed syntax:
```
class Circle {
fn Create(c: Point, r: f32) -> Self {
return {.center = c, .radius = r};
}
fn Diameter[me: Self]() -> f32 {
return me.radius * 2;
}
fn Expand[addr me: Self*](distance: f32);
private var center: Point;
private var radius: f32;
}
fn Circle.Expand[addr me: Self*](distance: f32) {
me->radius += distance;
}
```
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Chandler CarruthandRichard Smith
Jon Ross-Perkins
Geoff RomerandJon Meow
josh11bandChandler Carruth