Implements formatting support within LSP. Current implementation
performs while file formatting, we just replace file with output of
Carbon::Format()
https://github.com/carbon-language/carbon-lang/pull/7687 independently
improves formatting so that this output is somewhat decent.
Assisted-With: Gemini / Antigravity
Adds an algorithm to compute where to insert brackets to repair
bracketing mismatches during lexing. This takes indentation, as well as
a number of other cues, into account to predict where the brackets
should have gone. Detects when there is ambiguity between solutions and
makes no suggestion in that case. Reduces the problem by splitting on
properly bracketed top-level constructs, then uses a beam search to find
good candidate solutions quickly.
This includes both a fuzzer and an eval tool that can be used to
determine how well the algorithm fares against a given corpus of valid
Carbon code, by damaging it in various ways and seeing whether the
algorithm can correctly fix it. On all the eval modes, this algorithm
can correctly infer the positions for over 80% of lost brackets (and can
correctly restore 95+% of brackets in some modes), with low rates of
incorrect suggestions.
See added documentation for full details.
Assisted-by: Gemini via Antigravity, Claude via Claude Code
Add support for "jump to declaration", "find references", type
information on hover. This support is strictly single-file for now; only
references and declarations within the same file are found. We could go
a bit beyond that, but to properly handle cross-file references we'll
need to build an index and a compilation database, which is beyond the
scope of this change.
On hover, we provide the type information for the instruction under the
cursor as-is. This is frequently not very useful, as the type of a
function F is simply "<type of F>", but is a starting point for richer
information.
Assisted-by: Claude Code
We currently provide UTF-8 positions, since our source representation is
UTF-8 and we use byte offsets as column numbers, but the LSP protocol
default is UTF-16 column positions. Negotiate UTF-8 positions where
possible; this is supported by essentially every LSP client other than
VS Code.
In the case where we select a UTF-16 position, we continue to not do any
actual conversions, so our column offsets in such cases will continue to
be wrong on non-ASCII source files in VS Code.
Assisted by: Claude Code
When computing the token range of a declaration, we were using the first
and last parse nodes to determine the first and last tokens. That's not
correct -- the parse tree nodes can be in a different order from the
tokens, so the first parse node need not be at the start of the
declaration. For a malformed declaration such as `fn f` (with no
terminator), the first and last parse nodes were both associated with
the `fn` token for error recovery, meaning that the function name wasn't
even within the symbol we handed back to the LSP client, in violation of
the LSP requirements. This led to VS Code producing an error and
discarding all symbols in the file.
Assisted-by: Claude Code
Don't CHECK-fail if the file contains mismatched braces and generates an
unbalanced parse tree.
Properly balance the start and end of symbols. This previously caused
errors to be reported in clients such as VS Code.
Compute a correct range for definitions. Don't assume we can find the
matching `}` for an opening symbol, since some of our definitions are
delimited by a `;` instead.
Assisted-by: Gemini via Antigravity
Right now, the class destroy impl is incorrectly generated (first
discussed [in
Discord](https://discord.com/channels/655572317891461132/941071822756143115/1418614787449032826)).
If we want it to be correct, deferred definition logic would need to be
added, and the declaration would need to be moved inside the `class`
scope (along with whatever generic logic that needs).
This instead switches to a blanket impl, to avoid creating latent bugs
with generating the `impl` and function body in the wrong scope. This
approach uses the same blanket impl as aggregate destruction that was
added by #6098.
The intent here is to allow progress on other parts of `Destroy`. For
example, under this model the implementation of the function body could
be done as part of lowering the specific.
Although this focused on `Destroy` support, some choices here around
`implicit_type_impls` are because copy/move will likely follow a similar
approach. I'm trying not to predict too much about how we'll structure
those, but I'm putting `Destroy` impl logic in a file that could perhaps
be shared with those. They'd likely be interested in similar things,
e.g. traversing members of types (particularly class, struct literal,
tuple literal).
At present this sets the destroy function as `no_op` which is consistent
with current logic, but has a TODO to correctly define.
Constant importing for functions changes slightly due to some issues I
was having with `GetFunctionType`. zygoloid suggested this approach to
avoid `EvalInst` logic.
Adds a flag for controlling whether to generating these impls. While
this does generation for `class`, as noted above this'll also need to be
done for tuples and struct literals, which would leave the `none.carbon`
min_prelude unable to use any types. Note if destruction *would* occur,
it'll still look up `Core.Destroy` for that and fail, but that's already
true of any test using `none.carbon`. I'm trying to use the flag to see
if we can keep `none.carbon` working mostly-consistently.
I'd tried separating out the flag to #5852, but that got a lot of
pushback over whether the behavior was appropriate. I'm hoping that the
interactions here make it clearer why the particular approach -- the
goal is not to enable advanced testing, or create some new end-user
behavior that we really support, it's just to keep no-prelude tests
functional. The main question raised there was why not just keep
generating `impl T as Core.Destroy` if `fn destroy` is present -- but I
think here it should be apparent that would require additional
complexity, as the generation of `impl T as Core.Destroy` is not
currently conditioned based on the implementation of `fn destroy`. I'd
rather add complexity to this flag only if it's enabling interesting
test functionality.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Currently file tests for LSP must provide carbon source code as an
escaped string within notify params, i.e.
```
[[@LSP-NOTIFY:textDocument/didOpen:
"textDocument": {
"uri": "file:/class.carbon",
"languageId": "carbon",
"text": "class A {\n fn F();\n fn G() {}\n}\n"
}
]]
```
This works fine for simple, single line files but gets annoying when
working with more complicated files which are necessary when testing
more complicated features e.g. goto-definition
```
--- class.carbon
class A {
fn F();
fn G() {}
}
--- STDIN
[[@LSP-NOTIFY:textDocument/didOpen:
"textDocument": {
"uri": "file:/class.carbon",
"languageId": "carbon",
"text": "AUTOFILL"
}
]]
```
This PR extends file_test_base to be able to parse the notify/call
params and inject files from the test_file's splits into the JSON input.
I purposely avoid using the clangd types and manually parse the
llvm::json::Value here to avoid introducing a depdendency on clangd to
the generic file_test_base, but happy to change if we think that is
fine. Also happy to accept other suggestions on alternative methods to
achieve same result.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
The language server is now able to subscribe to and apply incremental
changes to the document. Source code is assumed to be utf-8. It does not
currently handle utf-16 code points.
---------
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This PR updates the HandleDocumentSymbol function to build a tree of
symbols, rather than a flat list. Symbols found while traversing another
symbols body are added as a child.
Adds a simple test showing functions within nested class.
---------
Co-authored-by: David Blaikie <dblaikie@gmail.com>
This PR updates the construction of clang::clangd::DocumentSymbol to
produce "range" and "selectionRange" properties that match the
[textDocument/documentSymbol](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_documentSymbol)
spec.
"selectionRange" is constructed by finding start and end position of the
symbols identifier.
"range" property is constructed by finding the left most and right most
token of the AST, the code makes an assumption that left most token is
always the first node in postorder traversal from that node, and the
right most is the node itself. AFAIK this is true for most of the
grammar other than infix operators. For symbols with a body, the right
most token is the open bracket, so we extend the range to its matching
closing bracket.
This PR adds a new command to the Carbon VSCode extension "carbon:
Restart language server" which acts similar to clangd / bazel
extensions. While not useful during regular working of the extension
this helps when LSP either crashes or the toolchain is updated.
In order to support graceful restart, I needed to add a Call handler for
"shutdown" which is noop for now. Lsp clients always call shutdown
before sending exit.
---------
Co-authored-by: jonmeow <jperkins@google.com>
Add caching of parsed documents, and testing of the textDocument
handlers. This is based on #4896, which splits out some of the
boilerplate to calls.
Note, this caches the entire parse state because we'll want to try to
emit diagnostics when we see the update, without waiting. It may be
helpful to do that asynchronously, but we don't want to wait for another
call (such as documentSymbol). Really, we'll probably want to also add
check for diagnostics, at least.
DavidLoftus
Richard Smith
Jon Ross-Perkins
