Commit Graph
66 Commits
Author SHA1 Message Date
Richard Smith 03939a9223 Add language server support for SemIR in testdata. (#7803)
Add hover cards and jump to declaration / definition / reference for the
formatted SemIR that appears in check tests. This is done by adding a
heuristic "parser" for SemIR to the language server. The
cross-references are strictly best-effort, since this is just a tool for
Carbon developers, not a user-facing facility.

A couple of other changes made along the way:

* file_test tests with an AUTOUPDATE-SPLIT no longer look for CHECK:
lines outside that split. This was motivated by the tests for this new
facility including CHECK: lines as part of the test input.
* An agent skill for working on the language server, tracking some
things that cost Claude time when working on this.

Assisted-by: Claude via Antigravity
2026-09-23 23:44:32 +00:00
Richard Smith f3d67de480 Start to preserve type sugar in diagnostics. (#7768)
Instead of always printing types as canonical, attempt to find a sugared
type where possible, and include that type in the diagnostic. We can
only do this when given the instruction whose type is being printed
(`TypeOfInstId`) rather than the canonical type ID.

Initial support here is intentionally minimal: just looking through
calls to the callee's declared return type, and looking through pointer
dereferences and corresponding pointer types, to build out the initial
infrastructure. More cases can be added later; this degrades gracefully
to using the canonical type if a better type can't be found.

Assisted-by: Claude Opus 5 via Antigravity
2026-09-15 23:11:58 +00:00
DavidLoftusandDavid Blaikie 301172f589 Update Carbon::Format to produce semi-reasonable output. (#7687)
Makes following changes to Carbon::Format()

- TokenKind::Period (i.e. `.`) should never have a space before or after
it.
- TokenKind::CloseSquareParen (i.e. `]`) should be treated as packed
content (no space preceeding it)
  - Only exception I can think of is `impl forall [...]`
- Remove preceeding space from `[` and `(` if previous token was an
identifier (or identifier-ish token)
- Remove seperator following `++` / `--` unary operators.
- Explicit gaps in source code should be retained, up to 2 new lines.

Multiple test files were added to test formatting. 

I imagine eventually this will need to be updated to read parse tree to
gather more context but this atleast lets us get a decent-ish format for
many of our current sample files (e.g. sieve.carbon)

Assisted-With: Gemini / Antigravity

---------

Co-authored-by: David Blaikie <dblaikie@gmail.com>
2026-09-08 16:20:47 +00:00
DavidLoftus dc2ee5bd5e Implement lsp/formatting for Carbon::LanguageServer (#7688)
Implements formatting support within LSP. Current implementation
performs while file formatting, we just replace file with output of
Carbon::Format()

https://github.com/carbon-language/carbon-lang/pull/7687 independently
improves formatting so that this output is somewhat decent.

Assisted-With: Gemini / Antigravity
2026-08-31 16:38:38 +00:00
Richard SmithandGeoff Romer c7dcc50768 language-server: Remove an unnecessary vector copy on each source change. (#7669)
We track a "next" index into each bucket when we insert instructions.
The insert loop effectively shifts each element in the "next" vector
left by one place, so if we instead start the bucket counts shifted one
place to the right, we can use the same vector for "next" and for the
bucket start indexes.

Assisted-by: Claude Code

---------

Co-authored-by: Geoff Romer <gromer@google.com>
2026-08-25 21:26:46 +00:00
Chandler Carruth 631f8fb6d2 Modularize driver subcommands and prune unused dependencies (#7658)
- Subdivide toolchain/driver:driver into granular subcommand targets
(compile_subcommand, format_subcommand, lld_subcommand, etc.) to enable
independent compilation and linking.
- Decouple Target and TargetMachine options from driver headers.
- Prune unused direct dependencies across toolchain BUILD files.

Assisted-by: Antigravity with Gemini
2026-08-20 13:32:46 +00:00
Richard Smith 3bbc03f527 Add better algorithm for repairing mismatched brackets (#7574)
Adds an algorithm to compute where to insert brackets to repair
bracketing mismatches during lexing. This takes indentation, as well as
a number of other cues, into account to predict where the brackets
should have gone. Detects when there is ambiguity between solutions and
makes no suggestion in that case. Reduces the problem by splitting on
properly bracketed top-level constructs, then uses a beam search to find
good candidate solutions quickly.

This includes both a fuzzer and an eval tool that can be used to
determine how well the algorithm fares against a given corpus of valid
Carbon code, by damaging it in various ways and seeing whether the
algorithm can correctly fix it. On all the eval modes, this algorithm
can correctly infer the positions for over 80% of lost brackets (and can
correctly restore 95+% of brackets in some modes), with low rates of
incorrect suggestions.

See added documentation for full details.

Assisted-by: Gemini via Antigravity, Claude via Claude Code
2026-08-19 18:55:32 +00:00
Richard Smith c78751338b language-server: Support simple semantic queries. (#7639)
Add support for "jump to declaration", "find references", type
information on hover. This support is strictly single-file for now; only
references and declarations within the same file are found. We could go
a bit beyond that, but to properly handle cross-file references we'll
need to build an index and a compilation database, which is beyond the
scope of this change.

On hover, we provide the type information for the instruction under the
cursor as-is. This is frequently not very useful, as the type of a
function F is simply "<type of F>", but is a starting point for richer
information.

Assisted-by: Claude Code
2026-08-19 14:35:05 +00:00
Richard Smith 9a33fc1673 language-server: attempt to negotiate UTF-8 positions. (#7636)
We currently provide UTF-8 positions, since our source representation is
UTF-8 and we use byte offsets as column numbers, but the LSP protocol
default is UTF-16 column positions. Negotiate UTF-8 positions where
possible; this is supported by essentially every LSP client other than
VS Code.

In the case where we select a UTF-16 position, we continue to not do any
actual conversions, so our column offsets in such cases will continue to
be wrong on non-ASCII source files in VS Code.

Assisted by: Claude Code
2026-08-17 20:56:37 +00:00
Richard Smith 277815fd94 language-server: Stub out support for some messages we don't handle yet. (#7637)
This suppresses warnings for unhandled messages where we currently have
nothing to do. No functionality change, except for less spam in the VS
Code output tab.

Assisted-by: Claude Code
2026-08-14 20:15:42 +00:00
Richard Smith db762cca85 language-server: Fix malformed symbol output on incomplete declarations. (#7638)
When computing the token range of a declaration, we were using the first
and last parse nodes to determine the first and last tokens. That's not
correct -- the parse tree nodes can be in a different order from the
tokens, so the first parse node need not be at the start of the
declaration. For a malformed declaration such as `fn f` (with no
terminator), the first and last parse nodes were both associated with
the `fn` token for error recovery, meaning that the function name wasn't
even within the symbol we handed back to the LSP client, in violation of
the LSP requirements. This led to VS Code producing an error and
discarding all symbols in the file.

Assisted-by: Claude Code
2026-08-14 19:56:07 +00:00
Richard Smith f473ca994a Fix a collection of small language-server bugs in document_symbol. (#7581)
Don't CHECK-fail if the file contains mismatched braces and generates an
unbalanced parse tree.

Properly balance the start and end of symbols. This previously caused
errors to be reported in clients such as VS Code.

Compute a correct range for definitions. Don't assume we can find the
matching `}` for an opening symbol, since some of our definitions are
delimited by a `;` instead.

Assisted-by: Gemini via Antigravity
2026-07-30 21:35:08 +00:00
Lucile Rose Nihlen 625f2ca629 precompile and cache Carbon prelude (#7432)
Refactors the link driver to automatically compile and cache the carbon
prelude for use in linking.

Implements a `carbon_library` rule for compiling the Core library
dependencies in the examples.
2026-07-16 18:02:14 +00:00
Richard Smith b6179ecbbb Disable prelude import in language-server tests. (#7489)
This was substantially slowing down the overall test suite.

Fixes #7453

Assisted-by: Gemini via Antigravity
2026-07-13 20:48:28 +00:00
Richard Smith 6181259cf1 Language server: prelude support. (#7417)
Support multi-file compilation, and in particular imports of files from
the prelude, in `carbon language_server`.

In order to properly interface with `CompileDriver`, also switch over to
building a proper VFS from the documents we're given.

Assisted-by: Gemini via Antigravity
2026-06-26 20:16:28 +00:00
DavidLoftus de381bded1 Fix off-by-one errors in LanguageServer's GetRange (#7251)
LSP assumes lines are index 0 to n-1, but Carbon Locs are index from 1
to n. We had the logic for this correct for the start of range but not
for the end of range (inclusive range).

Before this was the diagnostic span we would produce:

```carbon
fn F() {
  return ();
  <~~~~~~~~>
}
<~~~~~~~~~~~>
```

after:

```carbon
fn F() {
  return ();
  <~~~~~~~~>
}
```
2026-05-27 17:37:56 +00:00
Geoff Romer 4a0cf6c1fb Track the start of a signature more accurately (#6760)
This change ensures that a function signature always starts with an
`IdentifierNameMaybeBeforeSignature` node (renamed from
`IdentifierNameBeforeParams`), even in the case of function declarations
like `fn F -> T` that have no parameter list. As a consequence, this
ensures that we push new entries onto `pattern_block_stack` and
`full_pattern_stack` when we start processing the function signature.
2026-02-23 22:46:03 +00:00
Jon Ross-Perkins 45ca3d28f5 Drop "diagnostic" from some filenames in the "diagnostics" folder (#6686)
Mainly because "sorting_diagnostic_consumer" is legacy, since
`SortingDiagnosticConsumer` became `SortingConsumer`. Also better
reflecting contents of these files.

Where I'm not renaming, I'm less positive about dropping "diagnostics"
from "file_diagnostics" and "null_diagnostics" (which contain both a
consumer and emitter, and "null.h" seems like poor naming), so not doing
that here. Also "diagnostic.h" contains `struct Diagnostic`, so is a
decent fit.

Assisted-by: Google Antigravity with Gemini 3 Flash
2026-02-04 17:24:55 +00:00
Dana Jansens c64117d0e0 Make IdTag typesafe (#6574)
The IdTag knows the type of the Id its tagging and the type of the Id
being used as the tag. This prevents mixing up tagged and untagged ids,
and avoids having to work with untyped integers.

Adds an Untagged marker struct that's used as the tag type in IdTag when
no tag is desired.

The complexity of ConstantIds and TypeIds became a bit visible: TypeIds
are concrete ConstantIds. And ConstantIds have two different tagging
schemes, one for concrete and one for symbolic ids. And ConstantIds are
actually re-cast InstIds with the same index. The LoweredTypeStore needs
to work with tagged TypeIds, but the tags actually come from an InstId
store in ConstantValueStore. Now this is expressed in the type system by
getting the tags for TypeIds from the ConstantValueStore.

ValueStores without an TagId type parameter are now visibly untagged.

IdTag is now only default constructible when it does not have a tag,
which means ValueStore is only default constructible when the TagId is
untagged. This forces tagged value stores to be constructed correctly
with a tag at compile time, and untagged ones to be constructed without.

FixedSizeValueStore has overloads for dealing with tagged and untagged
Ids, since it can't default-construct ValueStore for tagged ids, and no
longer requires passing in default-constructed tags when there is no tag
in the ids.
2026-01-13 22:44:38 +00:00
Chandler Carruth e7eb3b7b5a Consolidate default Clang argument handling (#6545)
This unifies the default Clang arguments between the `clang` subcommand,
the `link` subcommand, and the `ClangInvocation` built for C++ interop.

This sets the stage to integrate either pre-built or on-demand runtimes
flags for both of these. However, this PR should have very little
practical difference. The biggest functional change is wrapping the
default arguments in flags to allow unused flags so that we can build a
collection of flags viable across compile and link.
2026-01-03 17:35:29 +00:00
Richard SmithandJon Ross-Perkins d208e950c7 Encapsulate clang::ASTUnit in SemIR::CppFile. (#6459)
This intends to avoid proliferation of dependencies on the exact API of
`clang::ASTUnit`, and would enable us to more easily switch to a
different approach that gives us more control over the construction of
the Clang AST.

Also remove some unnecessary tracking of the `CppFile` and instead
always retrieve it from the `SemIR::File`.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2025-12-04 21:05:40 +00:00
Chandler Carruth 5d0d443c98 Move the :install_paths library to //toolchain/base (#6457)
Previously this was kept in `//toolchain/install` so it would be near to
the code that actually defines the installation layout. However, that
creates somewhat unfortunate dependency cycles between
`//toolchain/install` and other directories. Exacerbating this, a
subsequent PR is likely to add dependencies on it from
`//toolchain/base` itself that suggests that is the correct layering.

This PR just moves the code mechanically with as few other edits as
possible.
2025-12-03 17:14:34 +00:00
David BlaikieandRichard Smith 12fa65e53c Check for use of InstIds from the wrong SemIR::File (#5997)
Use the `CheckIRId` as a unique identifier for the scope of an `InstId`
- if an `InstId` is created within the scope of one `CheckIRId` it must
not be used in the scope of a different `CheckIRId`.

This is achieved without extra storage, but with false negatives for
large inputs.

When an `InstId` is created, the original index of the `Inst` is XORed
with a tag derived from the `CheckIRId` to produce the final `InstId`.
When the `InstId` is used, the expected tag is XORed with the `InstId`
to get back to the original index - if the tags don't match, the
resulting index will be corrupted, likely too large - resulting in an
out of bounds index CHECK-failure.

(the tag value is derived as such:
* take the CheckIRId
* left shift one bit (padding zero)
* left shift another bit (padding 1 - used to signify that the resulting
`InstId` has a tag combined into it)
* reverse the bits

In this way, the tag is unlikely to overlap with the index for small
test cases - making it possible to separate out the `CheckIRId` from the
index in these cases to provide more meaningful debugging/CHECK
messages, and more informative `SemIR` textual dumping that can now
include the `CheckIRId` along with the `Inst`'s index in the name of an
`inst`)

The test churn here is improved printing as tagged `InstId`s can now,
with best effort (more likely for small test cases where the `CheckIRId`
and the `Inst` index aren't at risk of overlapping from the high and low
bits), render the `CheckIRId` as part of the inst's name. Going from
`instNN` to `irMM.instNN`.

---------

Co-authored-by: Richard Smith <richard@metafoo.co.uk>
2025-10-02 23:07:36 +00:00
Jon Ross-Perkins 49ba8cf3e1 Switch class to use a blanket impl for Destroy (#6125)
Right now, the class destroy impl is incorrectly generated (first
discussed [in
Discord](https://discord.com/channels/655572317891461132/941071822756143115/1418614787449032826)).
If we want it to be correct, deferred definition logic would need to be
added, and the declaration would need to be moved inside the `class`
scope (along with whatever generic logic that needs).

This instead switches to a blanket impl, to avoid creating latent bugs
with generating the `impl` and function body in the wrong scope. This
approach uses the same blanket impl as aggregate destruction that was
added by #6098.

The intent here is to allow progress on other parts of `Destroy`. For
example, under this model the implementation of the function body could
be done as part of lowering the specific.
2025-09-29 16:05:06 +00:00
Dana Jansens 64139e5d65 Stop using Map for the cache in InstFingerprinter (#6019)
This takes the debug runtime of
`toolchain/check/testdata/interop/cpp/function/arithmetic_types_bridged.carbon`
from 4.7s down to about 4s (so 15% faster overall).

There's still lots of room to improve this test which seems to be
hitting lots of pathological behaviour, but InstNamer is 30% of the
runtime, with fingerprinting's `InstFingerprinter::GetOrCompute`
consuming 10% of cycles. We reduce its impact by using a vector of
vectors instead of a Map for the cache of fingerprints. After this
change InstNamer drops below 24% of the runtime.

Also move the instruction name when giving it to `AllocateName` since it
receives std::string by value, though this doesn't show up in the
profile for the test.
2025-09-08 16:15:10 +00:00
Boaz Brickner 9f108bad6e Rename cpp_ast to clang_ast_unit (#5926)
Followup of #5924.
2025-08-08 08:11:49 +00:00
Jon Ross-PerkinsandGeoff Romer 7209ad7c9f Generate Destroy impls for classes (#5873)
Although this focused on `Destroy` support, some choices here around
`implicit_type_impls` are because copy/move will likely follow a similar
approach. I'm trying not to predict too much about how we'll structure
those, but I'm putting `Destroy` impl logic in a file that could perhaps
be shared with those. They'd likely be interested in similar things,
e.g. traversing members of types (particularly class, struct literal,
tuple literal).

At present this sets the destroy function as `no_op` which is consistent
with current logic, but has a TODO to correctly define.

Constant importing for functions changes slightly due to some issues I
was having with `GetFunctionType`. zygoloid suggested this approach to
avoid `EvalInst` logic.

Adds a flag for controlling whether to generating these impls. While
this does generation for `class`, as noted above this'll also need to be
done for tuples and struct literals, which would leave the `none.carbon`
min_prelude unable to use any types. Note if destruction *would* occur,
it'll still look up `Core.Destroy` for that and fail, but that's already
true of any test using `none.carbon`. I'm trying to use the flag to see
if we can keep `none.carbon` working mostly-consistently.

I'd tried separating out the flag to #5852, but that got a lot of
pushback over whether the behavior was appropriate. I'm hoping that the
interactions here make it clearer why the particular approach -- the
goal is not to enable advanced testing, or create some new end-user
behavior that we really support, it's just to keep no-prelude tests
functional. The main question raised there was why not just keep
generating `impl T as Core.Destroy` if `fn destroy` is present -- but I
think here it should be apparent that would require additional
complexity, as the generation of `impl T as Core.Destroy` is not
currently conditioned based on the implementation of `fn destroy`. I'd
rather add complexity to this flag only if it's enabling interesting
test functionality.

---------

Co-authored-by: Geoff Romer <gromer@google.com>
2025-08-04 19:39:22 +00:00
Richard Smith 25681901bd Improve mapping of Clang diagnostics into Carbon diagnostics (#5894)
Instead of taking the complete text of the Clang diagnostic and using it
as the message portion of a Carbon diagnostic, generate the individual
pieces separately and pass them into the Carbon diagnostic
infrastructure.

* Clang's context lines are generated by running a custom "diagnostic
renderer" and tracking which lines it wants to print as context for a
given source location. When mapping from a C++ source location back to a
Carbon location, the Carbon `Loc` structure is now fully populated,
including filling in the context line and the column number.
* Clang's snippet is generated by running a custom diagnostic renderer
that is a cut-down version of the full text diagnostic renderer that
only prints a snippet. This is then attached to the Carbon diagnostic
manually as an override for the snippet we'd usually create.

We no longer repeat the file location twice on each diagnostic, and no
longer produce a bogus "in import" line for all locations coming from
clang that point arbitrarily to the first C++ import in the Carbon file.
The `[diagnostic kind]` marker is now displayed at the end of the
diagnostic message, not on a line of its own after the snippet.
2025-08-01 01:24:31 +00:00
Jon Ross-Perkins bd4fbb4393 Expand use of CheckIRId stores (#5820)
This is trying to make it clearer when vectors are being indexed with
`CheckIRId`.

The only one that I still kind of want to change is the
`SmallVector<std::unique_ptr<CompilationUnit>>`, but because it's a
`unique_ptr` that's a little more complex. I may not bother.

Note, some of the changes around nuanced `SmallVector` interactions were
based on trying to copy the way `SmallVector` itself takes arguments,
like with range passing.
2025-07-21 20:02:27 +00:00
Richard SmithandChandler Carruth 553dd6e531 Build the clang::CompilerInvocation in the driver. (#5784)
Add driver flags to specify clang driver arguments.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2025-07-18 16:30:47 +00:00
Richard SmithandChandler Carruth 3776e464e0 Properly set up C++ include paths and similar environment settings when parsing imported C++. (#5767)
Stop using the clang tooling library to build an ASTUnit; that library
is set up to process clang frontend arguments, assuming that something
has already built frontend arguments from the compiler arguments. It is
also too encapsulated and doesn't let us inspect and modify the compiler
invocation before it's executed.

Instead, build the AST unit directly in two phases:

* FIrst, take a list of clang driver arguments and convert them into a
list of compiler arguments, using `clang::createInvocation`. Internally,
this uses the clang driver to build a frontend invocation, including
building system-specific include paths as needed.
* Then, directly build an ASTUnit from this compiler invocation.

I've factored this so that we can split out the `createInvocation` step,
with the intention that we may want to move it out of check and into the
carbon driver with the rest of the driver-level argument handling, and
we may want to customize some of the clang options before we invoke the
clang frontend with that set of options.

In order to make the invocation reusable, it no longer depends on the
name of the carbon file importing the C++ code. In place of synthesizing
a header file name as `<foo.carbon>.generated.cpp_imports.h`, we now
insert line marker directives into the generated header so that errors
in that header cause Clang to point a diagnostic back at the Carbon
source file itself. This results in a minor improvement in the
diagnostic output: we no longer refer to a nonexistent generated file.
But the snippet still contains text that doesn't match the source code,
so it remains imperfect.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2025-07-09 03:09:43 +00:00
Jon Ross-Perkins 2de746e83c Switch compile functions to use options structs (#5742)
I've been mulling this mainly for the parameter complexity of
check/lower, but doing lex/parse for symmetry.

I'm motivated by the plan to move dumping for all of them into the
respective functions, because of discussion about llvm-verifier. That
basically would add another bool parameter (or more) to each of these.
My instinct is we're going to probably accrue a little more over time,
so I'm suggesting this as maybe adding the boundary a little simpler
and/or easier to read.

Note it may make sense to refactor a little further, e.g. maybe
Lower::Context could receive the full set of options and pick out what
it wants, but I figured creating the struct itself would be a decent
start.

I'm trying to put things into options when we can produce a reasonable
default if the user doesn't assign a value. I'm using an explicit
constructor so that values can be added without affecting every caller.

A different factoring would be to pass in everything through the param
struct, but that just felt weird when I was trying it out.

Removing `inst_namer` and `module_name` from `LowerToLLVM` params --
both of these can be inferred from `sem_ir`, and I'm not seeing a
particular reason to maintain them at the call site.
2025-06-27 22:08:47 +00:00
Boaz Brickner ec97ec9664 Use InImport pointing to C++ imports in ConvertLocInFile() instead of adding a separate InCppImport when emitting (#5614)
I believe we will need to eventually add the specific C++ import
information in `LocId`.

This also seems to fix the `LanguageServerDiagnosticInWrongFile` issue
(#5604).

We're still not in the state we want to be according to
https://github.com/carbon-language/carbon-lang/pull/5246#issuecomment-2784301206.
Will look into removing the location part from the `"In file included
from ..."` line.

Part of #5245.
2025-06-10 09:02:45 +00:00
Jon Ross-Perkins 20c20595ba Fix language-server crash with cpp_ast (#5604)
Removes the nullptr default for safety.

Note, I think cpp support doesn't allow things like `<version>` or
inline code yet, and language-server support doesn't allow non-hermetic
files, so the best I can test is an error.
2025-06-03 22:09:51 +00:00
Boaz Brickner 852d0191a9 Add support for importing C++ inline functions (#5427)
This requires:
* Making `FunctionDecl` mutable since generating code
(`HandleTopLevelDecl()`) requires a mutable declaration and since we
manually add `used` attribute to force code generation.
* Passing the file system to `Lower` since it's needed by Clang code
generation.
* Creating an internal Clang LLVM module and link it against the Carbon
LLVM module.

Demo:

```c++
// hello_world.h

extern int puts;
inline void hello_world() {
  ((int (*)(const char*))&puts)("hello world");
}
```

```carbon
// main.carbon

library "Main";

import Cpp library "hello_world.h";

fn Run() -> i32 {
  Cpp.hello_world();
  return 0;
}
```

```shell
$ bazel-bin/toolchain/carbon compile main.carbon
$ bazel-bin/toolchain/carbon link main.o --output=demo
$ ./demo
hello world
```

Based on https://github.com/carbon-language/carbon-lang/pull/5406.

Part of #5405.
2025-05-21 07:02:05 +00:00
Jon Ross-Perkins 0683742f19 Cache multi-IR info, particularly include_in_dumps (#5408)
Right now we construct `tree_and_subtrees_getters` a couple different
ways, it's just not obvious because one's abstracted in `check`. But
also, when formatting IR, we'll repeatedly do the `IncludeInDumps`
string check, which felt odd to me since it only needs to be calculated
once per IR.

This also shifts `CheckIRId` selection a little earlier, and in doing so
makes `CheckParseTrees` accept a sparse `units` argument. I actually
think this is a positive: it makes `CheckIRId` a little more stable
across possible command lines, when file loading fails (which is the
only time that a file will have a `CompilationUnit` but not a
`Check::Unit`).

Trying to build on the shared issue between these, I'm adding a
`MultiUnitCache` to store the calculated arrays. For the subtree
getters, this is very minor and avoids at most one incremental array
construction (moving logic out of `CompileSubcommand::Run` might be the
bigger benefit). For `include_in_dumps`, when dumping SemIR, this is
changing a calculation run once per entity (in each IR) to be calculated
once per IR (globally), i.e. O(M*N) -> O(N).

Note this seems to be marginal for performance of file_test:

- Before: Stats over 10 runs: max = 5.3s, min = 4.7s, avg = 4.9s, dev =
0.2s
- After: Stats over 10 runs: max = 4.9s, min = 4.7s, avg = 4.8s, dev =
0.1s

I was mainly thinking about this in the context of dumping SemIR ranges.
There, the impact may actually decrease because a range won't do any
cross-IR printing. But, I'm expecting to add another layer for whether
we're printing IR for a file, and that made the `should_format_entity`
callback stick out for me.
2025-05-02 22:53:46 +00:00
Thomas Köppe bf32da8dad Add missing standard library header inclusions (#5316)
Discovered by clang-tidy.
2025-04-17 15:37:57 +00:00
Jon Ross-Perkins 8c3fa80691 Add cc rule wrappers for cc_env (#5277)
Rules executed by bazel don't necessarily have the right environment to
find the symbolizer, which was the intent of `cc_env` setting
`LLVM_SYMBOLIZER_PATH`. So far, this has kind of been a case-by-case
fix, but every so often I'm trying to debug a crash in a test that
doesn't provide it. Rather continuing down this route, instead add
drop-in wrappers for cc rules so that it's hard to forget.

Note `bazel/cc_rules` is intended to mirror `bazel/carbon_rules` and
`bazel/cc_toolchains`, rather than `@rules_cc`.

AFAICT there isn't a great way to add this as a default for the `bazel
run` environment. It's not typically going to be set on its own,
forwarding `$PATH` would be too broad, and the [action
`env_sets`](https://bazel.build/docs/cc-toolchain-config-reference#using-action-config)
I think are not quite what we need (I think those don't include output
execution, only compilation).
2025-04-11 19:58:54 +00:00
Jon Ross-PerkinsandRichard Smith 422df75a92 Switch tree-sitter from explorer to toolchain testdata (#5292)
Noticed as part of #5290; `srcs` is needed to make `$(locations)` work.

---------

Co-authored-by: Richard Smith <richard@metafoo.co.uk>
2025-04-10 23:45:19 +00:00
Jon Ross-Perkins 0a3efb76ed Use DiagnosticEmitter for phase-specific types (#5188)
Given the namespacing of `Diagnostics` in #5173, now we can use
`DiagnosticEmitter` for phase-specific emitters. This is consistent with
how we do `Context`, and also check had started this with
`DiagnosticBuilder` in anticipation of the namespacing.

Also renames `Emitter::DiagnosticBuilder` to `Emitter::Builder` for
consistency with other `Diagnostics` entities.

In check, I'm still splitting `DiagnosticEmitterBase` and
`DiagnosticEmitter` just to keep the emitter definition separate from
the context.

Also cleans up some incorrect check diagnostic emitter dependencies in
lower.
2025-03-27 00:41:30 +00:00
Jon Ross-Perkins acbe6530c3 Move diagnostics into a namespace (#5173)
What this really does is avoids shadowing names, so that we can
comfortable have things like `Check::DiagnosticEmitter` or
`Check::DiagnosticLoc` without shadowing being a concern.

Note, down this path I'm also thinking about:

- Renaming misc DiagnosticConsumer/DiagnosticEmitter classes, possibly
just to DiagnosticConsumer/DiagnosticEmitter (so
`Check::DiagnosticEmitter` instead of `SemIRLocDiagnosticEmitter`).
- Dropping `Diagnostic` from `Emitter::DiagnosticBuilder`.
- But not for `Check::DiagnosticBuilder`, because `Check::Builder` would
be ambiguous.
- Renaming diagnostics/diagnostic_* to drop "diagnostic".

[Discussion about SemIRLoc ->
DiagnosticLoc](https://discord.com/channels/655572317891461132/655578254970716160/1353771570463768698)
reminded me of this (in particular the older [Check::DiagnosticBuilder
discussion](https://discord.com/channels/655572317891461132/655578254970716160/1344363562608627763)),
but I'd only do that rename if there's matching consensus about a path
forward where we keep SemIRLoc, and in a way that it's only ever used
for diagnostics (the divergence from which is at the root of current
LocId discussion).

I'm trying to keep that separate from a namespace addition for clarity.
2025-03-26 19:12:10 +00:00
DavidLoftusandJon Ross-Perkins c302d0bc7a Allow LSP test_file to autofill didOpen params from previous splits (#5078)
Currently file tests for LSP must provide carbon source code as an
escaped string within notify params, i.e.
```
[[@LSP-NOTIFY:textDocument/didOpen:
  "textDocument": {
    "uri": "file:/class.carbon",
    "languageId": "carbon",
    "text": "class A {\n  fn F();\n  fn G() {}\n}\n"
  }
]]
```

This works fine for simple, single line files but gets annoying when
working with more complicated files which are necessary when testing
more complicated features e.g. goto-definition

```
--- class.carbon
class A {
  fn F();
  fn G() {}
}
--- STDIN
[[@LSP-NOTIFY:textDocument/didOpen:
  "textDocument": {
    "uri": "file:/class.carbon",
    "languageId": "carbon",
    "text": "AUTOFILL"
  }
]]
```

This PR extends file_test_base to be able to parse the notify/call
params and inject files from the test_file's splits into the JSON input.
I purposely avoid using the clangd types and manually parse the
llvm::json::Value here to avoid introducing a depdendency on clangd to
the generic file_test_base, but happy to change if we think that is
fine. Also happy to accept other suggestions on alternative methods to
achieve same result.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2025-03-13 17:22:20 +00:00
Jon Ross-Perkins dc0c2622ac Fix multiline incremental sync. (#5046)
Previously the start index was incorrect. Adding comments where I was
double-checking things along the way.
2025-02-28 23:04:07 +00:00
a881e21432 Language Server - implement incremental document sync (#4926)
The language server is now able to subscribe to and apply incremental
changes to the document. Source code is assumed to be utf-8. It does not
currently handle utf-16 code points.

---------

Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2025-02-15 02:05:56 +00:00
Jon Ross-Perkins d4f15ab26e Publish empty diagnostics on close (#4954)
Without this, diagnostics will linger after closing a file.

This refactors towards a pattern of putting outgoing calls as methods on
`Context`. I'm mixed on this, mainly thinking it's an improvement on
using `outgoing` directly (because it shares the name and structure),
might want to move it to a side-class later that is _only_ LSP wrappers.

I could also make inheritance private on OutgoingMessages and these
kinds of methods public there, but I'm hesitant to adopt that approach
versus a type separation.
2025-02-15 00:09:20 +00:00
DavidLoftusandDavid Blaikie 0931c601be Organize symbol hierarchy in LSP documentSymbol handler (#4914)
This PR updates the HandleDocumentSymbol function to build a tree of
symbols, rather than a flat list. Symbols found while traversing another
symbols body are added as a child.

Adds a simple test showing functions within nested class.

---------

Co-authored-by: David Blaikie <dblaikie@gmail.com>
2025-02-11 22:04:50 +00:00
Jon Ross-Perkins 2fef1cb713 Switch to trailing returns in toolchain and related code. (#4919)
Also makes the style guide explicitly comment on void, but this was the
intent IIRC because it matches Carbon's `-> ()` (and "always" versus
"except for void", which we definitely went back and forth on).

Includes adjusting function pointers, which I definitely forget this
syntax works sometimes.

Excludes utils/tree_sitter/src/scanner.c because it claims to be C, but
really we should probably fix that to be cpp.
2025-02-11 18:11:14 +00:00
DavidLoftus 9547b72b23 Update HandleDocumentSymbol to provide accurate symbol ranges (#4915)
This PR updates the construction of clang::clangd::DocumentSymbol to
produce "range" and "selectionRange" properties that match the
[textDocument/documentSymbol](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_documentSymbol)
spec.

"selectionRange" is constructed by finding start and end position of the
symbols identifier.

"range" property is constructed by finding the left most and right most
token of the AST, the code makes an assumption that left most token is
always the first node in postorder traversal from that node, and the
right most is the node itself. AFAIK this is true for most of the
grammar other than infix operators. For symbols with a body, the right
most token is the open bracket, so we extend the range to its matching
closing bracket.
2025-02-10 23:40:10 +00:00
DavidLoftusandjonmeow 2eddfbd7bf Add restart LSP command to VSCode extension (#4916)
This PR adds a new command to the Carbon VSCode extension "carbon:
Restart language server" which acts similar to clangd / bazel
extensions. While not useful during regular working of the extension
this helps when LSP either crashes or the toolchain is updated.

In order to support graceful restart, I needed to add a Call handler for
"shutdown" which is noop for now. Lsp clients always call shutdown
before sending exit.

---------

Co-authored-by: jonmeow <jperkins@google.com>
2025-02-10 23:39:49 +00:00
Jon Ross-Perkins f89985d0f4 Add support for publishDiagnostics (#4912)
<img width="536" alt="Screenshot 2025-02-06 at 3 02 18 PM"
src="https://github.com/user-attachments/assets/27b7673d-f8d4-4f9f-9e5d-20c3a88b8c78"
/>

Requires the caching work in #4897
2025-02-07 23:04:35 +00:00