Allow interleaving of STDOUT and STDERR check lines. Put STDOUT lines
after the line they're attached to, and STDERR lines before. If no
STDOUT check line is attached to any line, then put them all at the end
of the file instead.
This is intended to better handle the case where stdout contains
unreplaced mentions of line numbers, and also reflects that stdout is
typically a consequence of the test rather than commentary on it, so
placing it after the test seems likely to read better.
The intent of this change is that instead of paths looking like `explorer/testdata/foo/bar.carbon` (repo-relative), they're now just `bar.carbon` (local). The consequence is that paths should be a bit more durable in various environments, and just paths should be shorter and easier to read.
The explorer's prelude is an exception to this since it comes from data, rather than being the test target. Due to the change in approaches, it needs the regex again.
Uses #2829
Isolating test execution time (no build time included):
- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s
In GH actions:
- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
- test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
- test / test (macos-12, opt) (pull_request_target) Successful in 9m
I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
This PR is making two main changes to the Explorer and Toolchain:
- Replace the `is` keyword in `where SomeType is SomeInterface` with `impls`, so it is `where SomeType impls SomeInterface`
- Rewrite uses of the "impls" to something else to avoid, frequently "`impl` declarations" or "implementations", to avoid confusion with the `impls` keyword.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Also make minor updates to the skeletal design in
docs/design/name_lookup.md following #2113, as there are no longer any prelude names that are made available to unqualified name lookup by default.
Add `type` to the keyword list in
docs/design/lexical_conventions/words.md, following #2360.
Adds a simple script to merge stdout/stderr and put on labels. This is hidden to the RUN line using lit.cfg.py.
This is my solution to addressing how errors printed by the toolchain break sorting of stdout output; just put stdout first. We could also have toggles for interleaving output or such, which might help test whether we do it properly.
This also moves some previous-distributed replacement logic into lit_autoupdate_base.py: I think having that adjacent to lit.cfg.py is probably the better choice, and it reduces duplication in toolchain scripts. It happens here because I need to change the resulting commands to include the merge.
I've refactored the script in order to make it work in more contexts, which is why the delta is lost. I've actually refactored a significant amount with the intent of making the logic easier to understand, because I was also adjusting bits of it.
Some key notes:
- Removes the multi-pass update that was dealing with unfixed line numbers in explorer (I think the current script should work in one pass)
- Fixed explorer to handle multiple line numbers on the same line (turns out we can rely on local format for line numbers).
- Using execv instead of imports because making Python imports work in a setup like this feels like it's not worth it; only a nuisance.
- Adding __init__.py to satisfy mypy, which otherwise considers the lit_autoupdate.py scripts to be issues.
- Using py because I was thinking sh would be more platform-dependent. py should port better to Windows.
- Getting rid of [[ID#]] capture groups in the semantics-ir tests because with the autoupdate it's kind of moot (also, hard to autogenerate the pairs without relying on the %### value).
Note this does mean tests switch to more of a "make a change, see which tests change" setup. I don't know that that's a _bad_ thing though -- it's pretty much how tests are being written right now, which is why I went down this rabbit hole. It's a nuisance to make a change then _manually_ have to update a bunch of code.
My intent is to use this for to convert parse-tree tests to lit, but I wanted to do this with _existing_ tests first as a proof of concept and to make sure there's agreement.
On #2224 @zygoloid pointed out we needed --implicit-check-not to ensure we were correctly matching output. This is the standard way we're writing explorer tests, so I was looking at unifying our lit approaches.
This is one take on it, making more use of substitutions to bring various testing into alignment, as well as symlinks to avoid config skew (maybe I'll eventually figure out a better solution than symlinks).
Makes a couple small fixes in explorer to remove end-of-line whitespace on output.
Instead of complaining that the name is not completely declared, say it's not
declared at all yet.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Following #875, diagnose any use of a name prior to the point where it is introduced and its type is known.
This works by performing name resolution on a top-level class, interface, or impl twice: the first pass performs name-resolution for everything other than nested function bodies, and the second pass performs name resolution on function bodies. At the moment, the second pass does a superset of the work done by the first pass, and as a consequence, some identifier expressions now have their target set twice to the same thing.
Use FileCheck's `[[@LINE+n]]` mechanism to refer to the next line.
This is made awkward by a couple of things:
* We want to keep the `CHECK` lines in the original order.
* Errors are sometimes more than one line long.
The approach we use is to interleave the original lines and the check lines, putting each check line as early as possible subject to two rules:
1) Check lines never precede the 'AUTOUPDATE' line
2) Except when required by rule (1), a check line that refers to a source line by line number is never placed earlier than a source line that precedes that source line.
The actual `[[@LINE+n]]` annotations are created in a second pass after we've interleaved the lines so that we can work out the correct offsets.
Co-authored-by: Jon Meow <jperkins@google.com>
Richard Smith
Jon Ross-Perkins
josh11bandGeoff Romer