This file group exists to allow a `genquery` rule and a Python test to
verify our non-test dependency graph. We don't actually need to build
the binaries in the file group as part of that. The `genquery` rule
seems to do the right thing -- building it directly doesn't cause the
binaries in the group to be built. But without a manual tag, the group
itself is part of `:all` and thus part of `//...` and part of the rules
that will be built even with PR #3106. A consequence is that any change
to the toolchain causes several other binaries to be built as well
because this file group is in the impacted set. Making it manual should
avoid all of this, and without breaking the actual use from `genquery`.
For example, before this change, in a fully cached build after a `bazel
clean`:
```
> bazel test //bazel/check_deps:all
INFO: Invocation ID: 2d83ebee-4c00-425d-be33-23f42b079614
INFO: Analyzed 3 targets (103 packages loaded, 7137 targets configured).
INFO: Found 2 targets and 1 test target...
INFO: Elapsed time: 4.081s, Critical Path: 2.61s
INFO: 3111 processes: 2796 disk cache hit, 315 internal.
INFO: Build completed successfully, 3111 total actions
```
After this change:
```
> bazel test //bazel/check_deps:al
INFO: Invocation ID: c94089e8-a420-4d3c-9902-134e6b55b297
INFO: Analyzed 2 targets (92 packages loaded, 568 targets configured).
INFO: Found 1 target and 1 test target...
INFO: Elapsed time: 0.700s, Critical Path: 0.01s
INFO: 7 processes: 2 disk cache hit, 5 internal.
INFO: Build completed successfully, 7 total actions
```
While here, re-generate the file group, and fix several issues it
uncovers: mark test utilities as `testonly` and update our LLVM package
allowlist to include `clangd`'s package.
Also add `name_reference_untyped` for references to non-first-class
names without types, which currently covers namespaces and functions.
This improves the fidelity of the SemIR representation, and fixes some
issues where we would use the wrong location for nodes and diagnostics
downstream of a name reference.
We're still missing a representation for dotted name expressions, such
as `Namespace.Function`, and we don't use the `untyped` node as an
operand of any other node yet.
These routines have a regular signature and are used to build a table of
function pointers for fast dispatch. However, the previous approach
relied on lambdas to build these functions which resulted in very hard
to read functions in the profile and backtrace.
In preparation for expanding dispatch to handle (many) more cases in the
lexer and also enabling more aggressive inlining into the dispatch
routines, I wanted to tidy up how they appear.
This PR alone shouldn't have any interesting functional effect, it's
just re-organizing the code.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This removes a (very) hot branch in the lexer where we need to special
case when a token is the first token and can't look at its previous
token. It also seems like a generally nice change to the structure of
both the token buffer and parse tree as there are now bracketing
elements for both ends and we should be able to avoid similar branching
in the future.
Mostly mechanical updates to the lexer and parser code to handle this,
but also needed to special case the location information in the
autoupdate code. And then the usual large body of auto-updated tests.
No benchmark data for this change alone as in isolation and in the
current lexer structure it doesn't make a big difference. But this
branch was particularly difficult to handle when trying to update the
whitespace skipping code to be faster, and so I think it is worth
systematically avoiding the special case here.
Trust semantics to have put them in the right places.
Many parts of lowering still need to be updated to use the value
representation chosen at the semantics layer, but this is an incremental
step towards that.
Minor changes to the blaze command line executed by our autoupdate
scripts:
- Don't change the convenience symlinks. Running autoupdate shouldn't
cause `./bazel-bin/...` to switch to running a different binary.
- Don't produce so much spam. Bazel will still log its build progress if
necessary, and still report compile and runtime errors, but won't
produce half a dozen lines of INFO at the start of the command.
Continued from part 1: #3231. Second step updating
`docs/design/generics/details.md`. There remains some work to
incorporate proposal #2200.
- The biggest changes are incorporating much of the text of proposals:
- #2173
- #2687
- It incorporates changes from proposals:
- #989
- #1178
- #2138
- #2200
- #2360
- #2964
- #3162
- It also updates the text to reflect the latest thinking from leads
issues:
- #996
- #2153 -- most notably deleting the section on `TypeId`.
- Update to rule for prioritization blocks with mixed type structures
from [discussion on
2023-07-18](https://docs.google.com/document/d/1gnJBTfY81fZYvI_QXjwKk1uQHYBNHGqRLI2BS_cYYNQ/edit?resourcekey=0-ql1Q1WvTcDvhycf8LbA9DQ#heading=h.7jxges9ojgy3)
- Adds reference links to proposals, issues, and discussions relevant to
the text.
- Also tries to use more precise language when talking about
implementations, to avoid confusing `impl` declaration and definitions
with the `impls` operator used in `where` clauses, an issue brought up
in
- #2495
- #2483
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Combine the initialization, implicit conversion, and value category
conversion functions into a single function.
This substantially reduces the duplication between these steps, and
ensures that we support the same set of conversions in all these
contexts. This also fixes some issues where we would not use the proper
value representation for tuples and structs after performing implicit
conversions.
This makes the difference between errors and lower-level diagnostics
visible to users, and aligns the toolchain's behavior with the
expectations in `driver_fuzzer.cpp`.
Fix a bug where we would perform the computation of the return location
in SemIR after we have already used it in some cases, leading to
assertion failures during lowering. Instead, accumulate a sequence of
instructions to compute the return location in a temporary block, and
overwrite the return slot with those instructions when we perform
initialization.
StubReference is replaced by a more general SpliceBlock node, that takes
a code block and a result value, executes the instructions in the block,
and produces the result. This is used in the uncommon case where more
than one instruction is required to compute the return slot, which can
happen if we need to first emit a temporary and then index into it, or
if we need to perform multiple levels of indexing before we reach an
entity to initialize.
Includes proposals:
- #990
- #2188
- #2138
- #2200
- #2360
- #2760
- #2964
- #3162
Also tries to use more precise language when talking about:
- implementations, to avoid confusing `impl` declaration and definitions
with the `impls` operator used in `where` clauses, an issue brought up
in #2495 and #2483;
- "binding patterns", like `x: i32`, and "bindings" like `x`.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Factor out the common code to find the return slot for an initializing
expression, and use it to simplify the two different ways we finalize
initializers, as either initializing a temporary or initializing some
specific object.
This slightly changes the SemIR we create for function calls: instead of
rewriting the Call node to have a different destination and replacing
its temporary with a `no_op`, we now replace its temporary with a
`stub_reference` to the new destination. This results in the same amount
of SemIR being produced, but allows calls and other kinds of
initializers to be handled uniformly.
The speculative insertion of StubReferences after elements in an
argument list turned out to not be necessary, because we decided we want
to insert per-argument initialization steps after all arguments are
evaluated, rather than interleaving them. The StubReferences we insert
are causing some minor code complexity, so remove them.
We still create StubReferences when performing patch-ups of
already-emitted code, but we no longer ever need to look through them
when determining whether an initializer was a literal or when evaluating
a type expression.
This implements initializing expression semantics for structs and
tuples, following #2006 and discussions since.
Tuple and (and analogously, struct) literals are treated as having a
mixed expression category that is later resolved based on how the
literal is used, as either a tuple initializer or a tuple value, at
which point we create a `TupleInit` or `TupleValue` that represents the
formation of the tuple initializer or tuple value from the tuple
literal.
There's quite a lot of TODOs here, and the SemIR representation is still
not quite right, but this seems like a good place to checkpoint some
incremental progress.
First step in updating `docs/design/generics/details.md`. It
incorporates changes from proposals: #989#2138#2173#2200#2360#2964#3162 , but there are still more changes from those proposals to be
made.
It also switches away from suggesting static-dispatch witness tables,
and creates an appendix to describe that decision.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Rationale: this convention avoids forcing closely-related code to be far
apart in the namespace hierarchy, and vice versa. By the same token, it
makes the namespace hierarchy more consistent with the directory
hierarchy.
This is just aiming for an API simplification. Instead of the typical
`non_check_line_->line_number(), non_check_line_->indent()` with an
`INT_MAX, ""` special-case, instead pass `to_file_end` and let the
functions infer whether `non_check_line_` should be examined. (I'm also
hoping removing `INT_MAX` improves understandability)
Note this depends on #3234
Previous code could fail silently, only caught with yaml output
mismatches. This uses ErrorOr to be explicit about the error detection;
recovery isn't supported because we should only print valid yaml (in
tests, at least).
Also moves the test helpers from //toolchain/base to
//toolchain/testing.
Switches tree_test to use the matchers so that, on mismatch, gtest
prints a path to the mismatch (the straight value means it was just
printing "not equal").
Addresses
https://github.com/carbon-language/carbon-lang/pull/3217#discussion_r1323586836
Updates to a recent LLVM commit.
The patch file changes because there's now an upstream BUILD.bazel for
compiler-rt in the overlay, although it still doesn't expose the
libfuzzer target, so we need to keep patching it.
Clang's FileManager API changed slightly so we update migrate_cpp for
that.
Trying to more proactively catch when autoupdate is missed. Most of the
execution time of these tests should be in running the program under
test, not processing output, so this should have marginal overhead in
order to produce a useful reminder.
- `Peek()` no longer returns a useful value, because we often don't
allocate a `NodeBlockId` until we finish building the block. Remove it
and update both its callers.
- Rename `PeekForAdd()` to `PeekOrAdd()` since it's no longer used to
get a block ID to add elements into.
- `PushForAdd()` was unused. Remove it.
- Add `NodeBlockStack::AddNode` to combine the operations of adding a
node and inserting it into a block.
Instead of modeling array initialization as a thin wrapper around tuple
initialization, handle it like a function call, with a return slot as
part of its input. This better matches how initialization via a call to
`ImplicitAs::Convert` will eventually work, and in particular lets us do
in-place initialization of arrays rather than always creating a
temporary.
I'm finding the current autoupdate difficult to reason about. What I'm
trying to do here is use the class to make it easier to add helper
functions.
For example, I merge the vector+cursor for stdout/stderr into an object,
passed to helpers together instead of as two parameters.
ShouldAddCheckLine can check against output_file_number_ without passing
that through a couple levels of function calls. In turn,
ShouldAddCheckLine is shared with the end-of-file logic instead of that
having its own comparison from what AddCheckLines does.
Also, I'm trying to get the pre-AUTOUPDATE edits in their own loop,
distinct from the main code. The class means AddRemappedNonCheckLine is
a helper function to share code, instead of a lambda (which I was
thinking would just confuse the flow further).
I'm also changing the input non_check_lines to a single vector to match
stdout/stderr. Because of the AUTOUPDATE + SPLIT lines, we're guaranteed
to have at least one line per file.
I realized file_offset_in_new_lines is redundant with output_line_number
so code now uses the latter (because it's older -- I think one's as good
as the other, otherwise).
Note, this change deliberately does not affect output. I'm only trying
to make it easier to read for the next changes, using the lack of change
in results as a good indicator that this is getting it right.
`SemIR` now only stores complete blocks, and placeholders for blocks
where we need a block ID before we know the block contents.
This is a preparation step towards the new node block allocation design.
This removes the risk of accidentally performing a vector copy when
calling these functions, and is a preparation step towards the new node
block allocation design.
This required changing how we build call expressions. Instead of
finishing the argument block and then later adding a return slot, we now
delay finishing the argument block and checking for conversions to
parameter types until after we've added the return slot to it.
Include information in the test crash output to identify which test we
were running, and the corresponding resolved argument list, if either
`bazel test` or autoupdate crashes.
Building on #3214 and #3215, updates sem_ir yaml to be:
```
- filename: name
sem_ir: [ ... ]
```
Also, changes the textual format from `package { ... }` to `file
<filename> { ... }`. My thought on packages there is:
```
file "foo.carbon" {
package MyPackage
...
}
```
The reason for putting the file first is that it's easier if we put what
we're grouping on first, whereas the package is an "annotation" on the
file.
Lex now prints its yaml as:
```
- filename: name
tokens: [ ... ]
```
New support in file_test allows the `filename` marker at the top to
define the default file number for later lines, meaning multi-file
output from lexing is now associated with the appropriate file. Similar
support will probably also apply to lowering, semir, and other places
that print a filename once for the full dump.
This hammers a bit at how line number replacements work in file_test,
allowing stacking them so that lex errors and stdout can both be
line-associated properly. I've tried to make the autoupdate more
frequently work in one pass, now also taking into account the file index
when doing line replacements.
There are still some issues with EndOfFile that it may be good to
discuss: because CHECK lines are appended to the end of the file now,
and the EndOfFile token points at the last line including comments, new
lex tests now take two runs to autoupdate (because without CHECK lines,
the EndOfFile points at a content line, which content is then inserted
after). Note that removing CHECK lines from the test is not a solution:
autoupdate also started inserting blank lines, which breaks this for a
similar reason. One solution here might be to not have EndOfFile
associate with a line or column, which has been a bit of an issue
regardless.
Also fixes a small issue with toolchain's autoupdate script.
We're seeing issues building on the 20230911.1.0 release. I think there
may be a misconfiguration; using `/usr/lib/llvm-15/bin/clang++`, I'm
getting errors such as:
```
external/com_google_absl/absl/base/config.h:56:10: fatal error: 'cstddef' file not found
#include <cstddef>
^~~~~~~~~
```
e.g.:
https://github.com/carbon-language/carbon-lang/actions/runs/6166275499/job/16735504697
(the passing ubuntu run is on a 20230903.1.0 image using llvm-14)
(note if this PR fails testing, it needs to be merged before it would
take effect)
Reported on https://github.com/actions/runner-images/issues/8253
This better reflects the purpose of these semantics nodes, and prepares
for adding TupleValue and TupleInit nodes to represent forming values
and initializers from literals.
When printing callable declarations, it does not print the self pattern
in the deduced bindings.
Example:
User code:
```
class A {
fn Fun[self: Self]() {}
}
```
Print output is missing the self pattern
```
class A {
fn Fun ()
{
}
}
```
This PR fixes it and includes the self pattern in the print output.
```
class A {
fn Fun [self: Self]()
{
}
}
```
---------
Co-authored-by: Geoff Romer <gromer@google.com>
In order to maintain diagnostic quality, add a mechanism to add notes to
any diagnostics that are produced as part of initialization of function
parameters. As suggested in review of #3205.
In passing, fix the only caller of `ImplicitAsRequired` outside of the
implementation of `Check::Context` to instead use
`ConvertToValueExpressionOfType`. This causes some missing
`value_binding` nodes to be added to the produced SemIR. Also fixed a
matching bug in lowering where a bogus load was being added, that
resulted in assertion failures when the checker bug was fixed.
The warning `-Wnon-virtual-dtor` starts producing false-positive
warnings after this change. Replace it with the fixed version,
`-Wdelete-non-virtual-dtor`.
Chandler Carruth
Richard Smith
josh11b
Geoff Romer
Jon Ross-Perkins
Prabhat SachdevaandGeoff Romer