Also makes the style guide explicitly comment on void, but this was the
intent IIRC because it matches Carbon's `-> ()` (and "always" versus
"except for void", which we definitely went back and forth on).
Includes adjusting function pointers, which I definitely forget this
syntax works sometimes.
Excludes utils/tree_sitter/src/scanner.c because it claims to be C, but
really we should probably fix that to be cpp.
At present, we typically define a DiagnosticConverter, then store an
instance of it and a DiagnosticEmitter that wraps it. This is relatively
minor in general, but I've been trying to create more self-contained
DiagnosticEmitter classes (which hold their own DiagnosticConverter,
similar to NullDiagnosticEmitter), and there it just gets in the way.
Since we don't reuse DiagnosticConverter instances, this combines the
definition into DiagnosticEmitter. Mainly this means we don't have a
separate object in play, and less to carry around.
The most impact is probably to SemIRDiagnosticConverter, which was also
the most complex. Now `SemIRLocDiagnosticEmitter`, this gets some
different construction flow. Note in the PR I've split the file rename
to its own commit, to try to help delta views. However, the most
substantial parts of the refactoring are split into #4876, which this
depends upon.
At present, lower depends on `Check::SemIRDiagnosticConverter` for debug
info. That was to support a quick implementation of debug info, but
isn't great because it's both an unusual dependency on check's
implementation, and relying on diagnostic structures for debug info.
This cleans that up by splitting relevant logic out to a library in
sem_ir, and having lowering use sem_ir's library instead of check's.
Additionally, a small refactoring of `Parse::TreeAndSubtrees` to allow
getting locations in lowering without going through a `DiagnosticLoc`.
I'm adding `Parse::GetTreeAndSubtreesFn` in because it's a complex
signature to have in so many spots.
I chose to have `ResolveNodeId` return a `SmallVector` because it seemed
likely to be fairly compact, but that could also be using an optional
callback to handle resolved node IDs, possibly just returning the last
entry. This could be switched if preferred.
Note this change shouldn't affect behavior, it's just moving code
around.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
I'm looking at eliminating `DiagnosticConverter`. This change removes
`NodeLocConverter` (albeit adding `UnitAndImportsDiagnosticConverter`),
and in doing so, refactors lex conversion functions to extract them out
from the `DiagnosticConverter` functions.
I'll be following up with changes that collapse `DiagnosticConverter`
logic into `DiagnosticEmitter` locations. The intent is that we
shouldn't need separate ownership of both types.
This switches most error printing to use diagnostics instead of direct
stream writes, even when not a specific file diagnostic. I'm allowing
empty filenames for this use-case.
This allows a little more specific testing to validate coverage of
output using the diagnostic coverage test. I'm adding a few tests to
cover things that weren't previously tested.
Separately, this also forces a little more standardization in format...
considering how changes like #4568 show effort being spent to _mirror_
diagnostic style, my thought is now to just use diagnostic code where
possible.
Note this also allows incrementally better testing of the language
server; I'm changing the crash fix from #4847 in favor of diagnostic
testing.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
The language server needs stdin, and for tests we should be passing it
around. My intent is to pass in a faux stdin to Driver for language
server tests.
As long as I'm adding a new parameter, I was looking at also changing
the way streams are passed in to Driver for style (pointers since
they're held past construction lifetime). Since these are all stored in
DriverEnv, I thought it might be a net improvement to use the struct
directly, getting more explicit parameter names and also removing the
need for `SetFuzzing`.
I'm trying here to avoid functional changes, but there are a couple
additional fixes like removing an obsolete `find_insensitive` and
refactoring how `ValidateOptions` handles errors (because it reduces the
number of spots that operate on error_stream).
Hi,
I fixed a small issue that I found inside the "compile subcommand"
component:
The program can be crashed by running ```bazel run -- toolchain:carbon
compile --dump-mem-usage "non-existing-file.carbon"``` - i.e. by
activating the memory usage dump flag and passing a non-existing file.
Best regards,
oz
The offsets were originally added to deal with churn from builtins in
the raw semir. In textual semir, we mostly see instruction IDs for
imports, and builtins have also settled down more.
On imports, where possible, use the `EntityNameId` for an import instead
of printing an instruction. Next, show the source location if we have a
node. Only show the instruction if there's no location.
This also exposes `Parse::Tree` and `TokenizedBuffer`, so that we can
pass a `SemIR::File` without the component parts. In particular this
allows us to get the `TokenizedBuffer` for import IRs without
substantial structural modifications. We may want to make these optional
for serialized `SemIR` later, but the nodes/tokens contain source
location, which we'd need for debug information -- so it's not clear how
much we can really make them optional without substantial information
loss.
Reduce arguments to just `File` in a few spots, as a result of the
accompanying `TokenizedBuffer` and `Parse::Tree`. Also updates style to
pass around `const File*` where the reference is maintained, instead of
`const File&`.
I was considering keeping a direct reference to the tree and tokens on
`Context`, but initially my thought was it wouldn't make much
difference. I can re-add those if desired, just as direct caching of the
`File` fields.
The current representation has two nested presence indicatators (the
optional bool, the null pointer). Currently the pointer is never null,
but the style guide suggests that T* should be used for parameters that
may or may not be present, so we do not need the optional here.
> When passing an object's address as an argument, use a reference
> unless one of the following cases applies:
>
> - If the parameter is optional, use a pointer and document that it
> may be null.
Once the parameter is just a pointer, the ScopedTiming field does not
need an optional either, and can just store the pointer.
It would be more preferable to have an optional representation of a
sometimes-null pointer like optional<T&> to describe a sometimes-null
pointer, as this would allow clearer runtime diagnostics when used
incorrectly (a check failure in unwrapping) and would be better
self-documenting through syntax instead of a comment. But we do not
currently have such a primitive.
If they were called twice for a CompilationUnit, they would destroy
objects that they created and returned a pointer to, leaving a dangling
pointer somewhere else.
The switch defaults to true, so it's displayed as `--no-debug-info` in
the help. The description should be agnostic about whether the flag will
enable or disable the behaviour.
This is really a set of closely related changes:
1) We really shouldn't be creating a Check::Unit for _Lower_. This fixes
that by storing the diagnostic converter for reuse.
2) Rather than passing in node converters to Check as their own array,
pass diagnostic converters as part of Check::Unit.
3) To support creating the converters early, pass SemIR::File
pre-constructed.
4) Since SemIR::File construction was used to track "checked", add
`is_checked` for that.
5) Clarifies a subtle edge case around `input_filename_` use with `-`.
Note the key consequence of this change, where I actually started, is
that `Check` only has one array of `Check::Unit` instead of receiving
`NodeLocConverter` as a separate array.
As discussed in #4530 . This required switching to using
`llvm::IntrusiveRefCntPtr` in a number of places.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
We do not intend to use std::ranges in the Carbon implementation due to
concerns of compile time cost, largely due to implicit instantiation of
types involved in calling and typechecking the functions and their
requires clauses.
In #4539, we converted std::any_of to std::ranges::any_of, but this
replaces that with llvm::any_of from llvm/ADT/STLExtras.h.
This conversion was suggested by the modernize-use-ranges clang-tidy
check. We can keep the check on, and use it to guide conversion to llvm
helpers that do similar things (as was done in this CL now). If it's
being too confusing, then it can be disabled as well.
- Do not include entities imported from files that we are not dumping.
- Do not include constants and import_refs that are not referenced by
something that we are including in the formatted output.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
I think there are a few related ways to do this. I considered
llvm::make_scope_exit, but the return type is difficult to work with. I
particularly was thinking I could encapsulate the duration logic this
way.
This PR adds a `--dump-timings` flag to the `compile` subcommand
(similar to the existing `--dump-mem-usage` flag), which collects timing
data per compilation unit for each compilation phase. For example, on my
2020 M1 MacBook:
```
$ bazel build -c opt //toolchain
$ bazel-bin/toolchain/install/run_carbon compile --phase=lower --dump-timings examples/sieve.carbon | tail
...
---
filename: 'examples/sieve.carbon'
nanoseconds:
lex: 30792
parse: 25458
check: 226625
lower: 1136958
Total: 1419833
...
```
Most of the changes are pretty straightforward. There were a couple I
wasn't sure about though; let me know if I should change:
- new `Timings` class in its own file, pretty similar to the existing
`MemUsage` class
- added a `timings_` field to the `CompilationUnit` class
- added a `timings` field to the `Check::Unit` struct
- renamed `CheckParseTree` function to `CheckParseTreeInner` for ease of
timing with early `return`
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Trying to make split file tests of lex functionality shorter and easier
to read. numeric_literals.carbon in particular has an example of why I'm
interested in this (at the bottom). This also switches from `[]` list
format to `-` list format so that the trailing `]` is removed.
Trimming comments in tokenized_buffer.h because (1) it feels like it's
giving too much detail about what's printed, which has drifted slightly
and (2) it also feels like it's trying to justify YAML output, when
that's just what we're doing in general.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
This is to help identify which diagnostics we're actually using.
Note that driver/testdata still has tests which don't pass this flag,
and so continue to test the kind-less (default) behavior.
This extracts out the SourceBuffer handling of `-` in order to trivially
share it.
Note this still has a number of TODOs, it's just setting up the
essential subcommand infrastructure, with some tests demonstrating that
it at least does something.
This risks diagnsotic formatting crashing, but I think we more
frequently see cases where it'd be interesting to know what diagnostics
were being delayed as part of the default sorting.
This is a primarily automated change:
- Search & replace for capitalization
-
`(CARBON_DIAGNOSTIC\((?:\n\s+)?\w+,(?:\n\s+)?\s\w+,(?:\n\s+)?\s")([A-Z])`
- `$1\L$2`
- Search & replace for period
-
`(CARBON_DIAGNOSTIC\((?:\n\s+)?\w+,(?:\n\s+)?\s\w+,(?:\n\s+)?\s"(?:[^)]|\n)+)\.("[,)])`
- `$1$2`
- Limited search & replace for `ERROR: ` -> `error: ` in streamed things
- Leaving a TODO for command_line because there's more cleanup that can
be done there
- Modify diagnostic_consumer.cpp
- ERROR -> error
- WARNING -> warning
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Note the purpose here is to make it simpler to add more subcommands,
without adding a lot of things to Driver.
This creates a copy of CodegenOptions, but it was double-registered at
present which felt odd. It's also fairly small right now. If this
becomes an issue, maybe we can look into using optional for delayed
initialization, or just go back to straight sharing.
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
I'm separating the options out so that it's easier to review. They
include a lot of boilerplate text that I think won't change much, and
makes it harder to review changes.
To explain filename differences, whereas `CodegenOptions` is shared (by
link and compile), `LinkOptions` and `CompileOptions` are
subcommand-specific. I'm planning to separate out the subcommands, so
I'm putting those in respective subcommand files. I'm still going to try
to use the `.h` to declare the interface, `.cpp` for bigger
implementation details (for better or worse, including comments on
options).
I'm also moving out corresponding Driver members to help shrink deltas
when refactoring. That is, the bodies aren't changing here, but a
refactoring of commands will make some changes. By moving the code to
different files now, it should be easier to identify what's changing
later.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Jon Ross-Perkins
Calvin
Boaz Brickner
ottmar-zittlau
Richard Smith
Dana Jansens
josh11bandJosh L
Sam EstepandJon Ross-Perkins
