Instead of crashing when run outside of `bazel`, make the toolchain's
`file_test` binary work properly when no test-specific environment
variables are set. This makes it a lot easier to run `file_test` under a
debugger.
There are two main changes here:
- Don't crash if `$TEST_TMPDIR` is unset. Instead, fall back to LLVM's
temporary directory (typically `$TMPDIR`). We already did this in some
places in tests. We now do it in more places.
- Don't fall back to a target label of `<target>` in the reproduction
commands if `$TEST_TARGET` is unset, because this causes all the tests
to fail because their output doesn't match the expected output due to a
differing bazel run command. Instead explicitly specify the target from
the `FileTestBase`-derived class.
Infrastructure for this has been added generally, but only rolled out to
the toolchain `file_test` binary for now.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Small cleanup for code sharing. Note, `*ReadFile` will trigger a
CHECK-failure on error; the relevant implementations would previously
have failed silently (empty string).
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This switches most error printing to use diagnostics instead of direct
stream writes, even when not a specific file diagnostic. I'm allowing
empty filenames for this use-case.
This allows a little more specific testing to validate coverage of
output using the diagnostic coverage test. I'm adding a few tests to
cover things that weren't previously tested.
Separately, this also forces a little more standardization in format...
considering how changes like #4568 show effort being spent to _mirror_
diagnostic style, my thought is now to just use diagnostic code where
possible.
Note this also allows incrementally better testing of the language
server; I'm changing the crash fix from #4847 in favor of diagnostic
testing.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
The language server needs stdin, and for tests we should be passing it
around. My intent is to pass in a faux stdin to Driver for language
server tests.
As long as I'm adding a new parameter, I was looking at also changing
the way streams are passed in to Driver for style (pointers since
they're held past construction lifetime). Since these are all stored in
DriverEnv, I thought it might be a net improvement to use the struct
directly, getting more explicit parameter names and also removing the
need for `SetFuzzing`.
I'm trying here to avoid functional changes, but there are a couple
additional fixes like removing an obsolete `find_insensitive` and
refactoring how `ValidateOptions` handles errors (because it reduces the
number of spots that operate on error_stream).
This adds a RawStringOstream. Versus TestRawOstream, which is
consolidated over to RawStringOstream, it uses a string for storage
instead of a vector, mainly to support move-to-string semantics. Versus
llvm::raw_string_ostream, it owns the string and supports pwrite (which
is needed for driver and its fd_ostream compatibility requirement).
This converts most uses of llvm::raw_string_ostream, leaving behind a
few in InstNamer that explicitly cannot own the string, such as:
```
llvm::raw_string_ostream(name)
<< "_" << tree.tokens().GetColumnNumber(token);
```
I have this as its own library so that it can use CHECK.
Yes this doesn't save much code, but it's code we repeatedly write.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Hi,
I fixed a small issue that I found inside the "compile subcommand"
component:
The program can be crashed by running ```bazel run -- toolchain:carbon
compile --dump-mem-usage "non-existing-file.carbon"``` - i.e. by
activating the memory usage dump flag and passing a non-existing file.
Best regards,
oz
Pursuant to discussion regarding #4699, turn on
`misc-non-private-member-variables-in-classes` using the
`IgnoreClassesWithAllMemberVariablesBeingPublic` flag (the check treats
structs as classes, so we need this for structs with all-public
members). Updates the style guide notes to match, which should be pretty
minor due to the scoping of test fixtures.
Also fixes some underscore uses in test files on the way. Basically this
is keeping the style for [class data member
naming](https://google.github.io/styleguide/cppguide.html#Variable_Names)
even while making them public.
This changes to an `Error` return to let the driver do the "error: "
prefix, except for one case with `help` that needs more work to change
(I'm not planning on picking up that TODO). It also changes
capitalization, backtick use, and a few minor punctuation things to try
to better match the diagnostic style.
This also adds `Error` matchers so that the changes to command line
testing are clearer.
As discussed in #4530 . This required switching to using
`llvm::IntrusiveRefCntPtr` in a number of places.
---------
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
This is a primarily automated change:
- Search & replace for capitalization
-
`(CARBON_DIAGNOSTIC\((?:\n\s+)?\w+,(?:\n\s+)?\s\w+,(?:\n\s+)?\s")([A-Z])`
- `$1\L$2`
- Search & replace for period
-
`(CARBON_DIAGNOSTIC\((?:\n\s+)?\w+,(?:\n\s+)?\s\w+,(?:\n\s+)?\s"(?:[^)]|\n)+)\.("[,)])`
- `$1$2`
- Limited search & replace for `ERROR: ` -> `error: ` in streamed things
- Leaving a TODO for command_line because there's more cleanup that can
be done there
- Modify diagnostic_consumer.cpp
- ERROR -> error
- WARNING -> warning
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This switches `DCHECK` and `FATAL` as well.
The goal is to reduce the code size impact of these assertions so that
we can keep more of them enabled. Currently, the largest cost I see from
`CHECK` is not the actual check or the cold code itself, but actually
the failure to inline trivial functions due to the presence of the cold
code. This means that our goal isn't to reduce apparent code size in the
final binary but the LLVM IR cost assessed for these routines in the
inliner, which closely correlates with code size but is a bit different.
As discussed in #4283, experimentation shows that a single function call
with a minimal number of arguments is the lowest cost model for these.
This is easily achieved with a format-string API that internally uses
`llvm::formatv`. This PR is essentially the `CHECK` version of #4283.
However, the check macros are substantially harder to make work with
both format strings and streaming because they also take a condition.
Also, unexpectedly, I was very successful at devising a regular
expression based automated rewrite from the streaming to the format
string form with only low 10s of manual fixes. This includes compacting
strings broken up across lines, etc. Given how well that went, I've
prepared this PR which just directly switches to the format string API
and migrate everything to use it.
One nice side-effect is that the format string approach ends up greatly
simplifying the implementation here as well.
This is ... *shockingly* effective. Parsing speeds up by more than 3%
with just this change. And checking speeds up by **8%** with this change
alone:
```
BM_CompileAPIFileDenseDecls<Phase::Parse>/256 86.3µs ± 1% 82.9µs ± 1% -3.94% (p=0.000 n=17+19)
BM_CompileAPIFileDenseDecls<Phase::Parse>/1024 431µs ± 1% 415µs ± 1% -3.76% (p=0.000 n=18+19)
BM_CompileAPIFileDenseDecls<Phase::Parse>/4096 1.77ms ± 1% 1.71ms ± 1% -3.18% (p=0.000 n=18+19)
BM_CompileAPIFileDenseDecls<Phase::Parse>/16384 7.44ms ± 1% 7.17ms ± 2% -3.56% (p=0.000 n=18+20)
BM_CompileAPIFileDenseDecls<Phase::Parse>/65536 30.7ms ± 1% 29.7ms ± 1% -3.15% (p=0.000 n=18+20)
BM_CompileAPIFileDenseDecls<Phase::Parse>/262144 131ms ± 1% 127ms ± 1% -2.81% (p=0.000 n=18+18)
BM_CompileAPIFileDenseDecls<Phase::Check>/256 878µs ± 2% 800µs ± 1% -8.91% (p=0.000 n=19+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/1024 1.88ms ± 2% 1.72ms ± 1% -8.56% (p=0.000 n=19+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/4096 5.78ms ± 2% 5.28ms ± 1% -8.70% (p=0.000 n=20+18)
BM_CompileAPIFileDenseDecls<Phase::Check>/16384 21.9ms ± 1% 20.1ms ± 1% -8.02% (p=0.000 n=18+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/65536 90.4ms ± 2% 83.1ms ± 1% -8.04% (p=0.000 n=19+20)
BM_CompileAPIFileDenseDecls<Phase::Check>/262144 381ms ± 2% 352ms ± 1% -7.79% (p=0.000 n=19+19)
```
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
Consolidates both main libraries into `//testing/base`, and factors out
the exe path handling for benchmarks and unit tests into a common
library to remove duplication. Refactors how that logic is managed to be
cleaner and avoid a confusing bool that came up in code review.
Updates all the tests and benchmarks that use these. I still need to
update other benchmarks to use the same main, but I wanted to keep this
PR somewhat minimal.
This also fixes a bug noticed in passing that the compilation benchmark
didn't have the required dependency on the benchmark library itself,
just the benchmark main library.
This removes the `data_dir` from the driver favoring the installation
abstraction for the both locating the prelude and linking utilities.
With this, an installed toolchain should also be able to compile and
link Carbon successfully, and the build of the examples should exercise
this path almost exactly. (The only difference is using the driver
`cc_binary` directly rather than relying on the symlink from inside the
install tree.)
The install directory contains the BUILD logic for creating an
installable tree of data files and executables for the toolchain, and
a library to facilitate toolchain code accessing the paths to their data
within this installation.
Then adds an installation of LLD in a synthetic LLVM installation, and
teaches the Clang runner to configure this and use it for linking
instead of the system linker.
Currently, the install paths only really manage access to the LLVM
binaries installed and used by the Clang runner for linking, but
eventually other data files like the prelude and runtime libraries will
be fleshed out as well. There are TODOs for moving more things over here
such as the prelude.
One interesting aspect of this is where to put helpers like parts of
LLVM in our install. This PR suggests nesting those files under
`lib/carbon`. While using a `lib` subdirectory isn't a perfect fit for
the FHS (Filesystem Hierarchy Standard), having a single location where
private data is collected is significantly superior to spreading them
across the system. This also matches similar patterns used by Clang
itself and several other language toolchains and standard libraries.
The install directory also provides a natural place for us to build out
packaging rules to create installable packages in various formats, but
that remains future work.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This is just to be able to directly run the driver (e.g., `bazel run
//toolchain/driver:carbon compile foo.carbon`); it shouldn't affect
anything else.
- Adds an empty prelude.carbon file
- Imports that file in any non-Core package file
- Adds --disable-prelude-import to avoid that
- Adds --exclude-dump-file-prefix to be able to hide files from dumping
- Used to hide core files (we can't do this by package name due to lex
dumps, for example)
- Restructures some tests to not rely on `i32`, particularly `alias`
tests (which rely on a name ref) and tests with no prelude.
I'm adding the framework for switching i32 to calling Int32 in the
prelude, but I'm running into a separate error actually switching over.
So that *mostly* works, but isn't quite ready for prime time. However,
maybe the current state of this PR is still useful to review since it
does a lot of the infrastructure work and adds the %Core everywhere?
I wanted to choose one or the other. I think some code has been using
each from early on. We're predominately using `filename`, so
consolidating on that. This conveniently matches the [Google dev doc
style guide](https://developers.google.com/style/word-list#filename)
(which we use for docs) which says "filename: Not file name".
In toolchain:
```
╚╡git grep file_name . | wc -l
35
╚╡git grep filename . | wc -l
489
```
This handles toolchain failures per-file. The intent is to allow placing
both "success" and "fail" tests in the same file, using splits. However,
this PR only adds support and updates existing tests to continue
passing.
Similar to #3705, we actually have a mix of `Make` and `Create` in
factory functions too, so this PR is normalizing on `Make`. It's
intended to be consistent with the naming choice for Carbon factory
functions.
Note, MakeSyntheticBlock is the only one I feel a little weird about
because llvm's own APIs use Create, and this is essentially wrapping
LLVM calls. But the flipside is it also feels like a vague line to draw,
when we also differ from LLVM coding style in other ways.
Rationale: this convention avoids forcing closely-related code to be far
apart in the namespace hierarchy, and vice versa. By the same token, it
makes the namespace hierarchy more consistent with the directory
hierarchy.
Previous code could fail silently, only caught with yaml output
mismatches. This uses ErrorOr to be explicit about the error detection;
recovery isn't supported because we should only print valid yaml (in
tests, at least).
Also moves the test helpers from //toolchain/base to
//toolchain/testing.
Switches tree_test to use the matchers so that, on mismatch, gtest
prints a path to the mismatch (the straight value means it was just
printing "not equal").
Addresses
https://github.com/carbon-language/carbon-lang/pull/3217#discussion_r1323586836
Lex now prints its yaml as:
```
- filename: name
tokens: [ ... ]
```
New support in file_test allows the `filename` marker at the top to
define the default file number for later lines, meaning multi-file
output from lexing is now associated with the appropriate file. Similar
support will probably also apply to lowering, semir, and other places
that print a filename once for the full dump.
This hammers a bit at how line number replacements work in file_test,
allowing stacking them so that lex errors and stdout can both be
line-associated properly. I've tried to make the autoupdate more
frequently work in one pass, now also taking into account the file index
when doing line replacements.
There are still some issues with EndOfFile that it may be good to
discuss: because CHECK lines are appended to the end of the file now,
and the EndOfFile token points at the last line including comments, new
lex tests now take two runs to autoupdate (because without CHECK lines,
the EndOfFile points at a content line, which content is then inserted
after). Note that removing CHECK lines from the test is not a solution:
autoupdate also started inserting blank lines, which breaks this for a
similar reason. One solution here might be to not have EndOfFile
associate with a line or column, which has been a bit of an issue
regardless.
Also fixes a small issue with toolchain's autoupdate script.
A lot of this is more boring "remove unused header", plus some other
minor cleanups. I think the most significant changes were:
- yaml_test_helpers.cpp is doing a switch on an unsigned int, comparing
to enum values.
-
[bugprone-switch-missing-default-case](https://clang.llvm.org/extra/clang-tidy/checks/bugprone/switch-missing-default-case.html)
is unhappy with EnumBase, but correctly identified
yaml_test_helpers.cpp, so I'm opting to address it rather than disabling
it even though it needs NOLINT in several locations as a result, in
addition to what I think are some low-value `default` cases. I'd be fine
going the other way with this too and disabling it globally (I could see
it being noisier in the explorer).
- MarkInitializerFor swaps the argument names between the .h and .cpp. I
think the .cpp had the order as intended.
- There's a new-ish
[performance-enum-size](https://clang.llvm.org/extra/clang-tidy/checks/performance/enum-size.html)
which I'm basically treating as "add int8_t to enums".
My main motivation here is to just clean up as many of these as I can so
that I stop seeing them in vscode.
This also tries to restructure the command line interface to the
toolchain a bit to make it start operating more like a compiler that
could be integrated into a build system rather than primarily as
a testing tool.
1) This switches form a `dump` subcommand to a `compile` subcommand
which has "dump" actions that can be enabled within it.
2) A distinct set of compile _phases_ that match the toolchain
structure:
- `lex` to run the lexer
- `parse` to run the parser
- `check` to fully check that the code is valid
- `lower` to lower to LLVM's IR
- `codegen` to generate executable code
3) The codegen phase has two output formats: textual assembly and
a binary object. These outputs can be configured, with a default for
an object when writing to a file and more firm default for textual
assembly when writing to stdout.
4) Select and expose the use of the LLVM host detection to compute
a default code generation target in the driver so that the command
line interface can reflect this. For example, the `help` output will
include the default target.
5) The `//toolchain/codegen` library APIs have been restructured a bit
to make the code flow a bit more naturally when implementing the new
command line structure. No real changes to the logic though.
There are also some minor tweaks to the command line interface based on
trying to use the shortest names for things that still seem likely to be
learnable for users:
- Switched `target-triple` to just `target`: the "triple" component to
this name is historical and can be confusing. For example, almost all
"triple" strings have more than three components today.
- Switched to just `--output` as now the fact that it is a file can be
configured in the documentation -- it will render as `--output=FILE`.
This also adds support for two custom output filename modes. First, when
no output is specified, we now compute one in the conventional way for
compilers by removing the file extension of the input file and replacing
it with `.o` for an object file output or `.s` for an assembly file
output. This matches the behavior of Clang and GCC for example.
Second, output to stdout is enabled with the special output file name of
`-` since it is no longer the default. This also follows the convention
of most compilers and many other command line tools to use `-` as a file
name to signify using standard in/out pipes.
There are still some rough edges here that I suspect could be improved,
but this seems like a good start of switching over to a complete
argument parser.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Lucile Rose Nihlen <luci.the.rose@gmail.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
These tests are doing string comparisons on output that don't seem to be meaningfully different from a file_test.
I'm tempted to migrate lexer tests in general, but I'm not doing that here since others may find more value in the current approach.
This is just a cleanup. Since we now have a testing directory, I think this is a better home for testonly libraries than //common. (I was thinking about this when I was considering adding more test_raw_ostream deps)
This adds vfs support to the toolchain, allowing Driver to take in-memory inputs in tests. As a consequence, I'm simplifying SourceBuffer: rather than allowing tests to pass in their own memory buffer, I'm using InMemoryFileSystem to push for greater consistency with production code. This does hit a quirk where I need to be careful about null terminator handling because fuzzer imports don't always have one, but that's probably more robust anyways.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This is a convenient test helper for anything that can use injected
streams. Extract this to where it can be used by other tests and add
some basic tests, mostly documenting how it works.
This makes TokenizedBuffer more consistent with ParseTree and SemanticsIR, which also wrap with [] to produce a sequence value.
It also makes it possible in driver.cpp to just prefix the line with a name, so it ends up with:
var_name: [
(content)
]
Noticed this due to bracketing comments on #2443 and trying to think of better answers. With this, we can also say that the [] bracket a variable.
The ParseTree comments say that preorder is "easier to visualize and read". The problem is, both the ParseTree and Semantics need to operate on the postorder traversal: the ParseTree during construction, and the Semantics during processing. As a consequence, understanding the postorder traversal is important, but it's also very hard to decipher when presented preorder. This PR provides a way to see the postorder, with helpful indents to show subtrees.
This retains the preorder printing as an option for people who prefer that. I'm pretty sure it'll be easier to debug tests if we can see the postorder, so I'm making that the default.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This unifies `dump-tokens` and `dump-parse-tree` so that we don't keep writing basically the same code repeatedly.
I'll need to modify the output of SemanticsIR::Print more, and may soon unify printing multiple IRs this way (particularly including the builtins SemanticsIR) but this is intended to offer a starting point.
I may eventually try to unify lit.cfg.py files, but I was thinking about whether that works in various contexts we may run in and eventually decided copying the driver/testdata/lit.cfg.py file would be the easiest solution.
The small size is for the 1m vs 5m time limit -- all these tests _should_ be fast so a lower limit seems consistent, and the 5m timeout was getting in my way when trying to debug *actual* timeouts.
The Carbon::Testing bit is for convenience -- test libraries are generally using it, it seems like the tests should too. Note this reduces the need for `using`.
This does push NodeMatchers into Carbon::Testing -- I don't think this was benefiting from having its own namespace; `using namespace` is discouraged [under style](https://google.github.io/styleguide/cppguide.html#Namespaces), we wouldn't support an equivalent in Carbon, and it feels like it's not helping to avoid name collisions. (also tidy was bugging about it, and while I could NOLINT that, this felt like the better approach)
There are some declaration order changes, and a few test classes switched from `struct` to `class`. However, this PR is mostly adopting `_` naming of private member variables due to the shift in naming style. None of what's here should have behavior impacts, it should just be style.
Note, there are a lot of things that *look* like they could be accessor-named, but I'm not doing that in this change. Happy to do it separately if you want me to do another PR focused on it.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This moves over to the vanilla upstream GoogleTest pulled in the more
expected manner with Bazel. It also adds Abseil and Google Benchmark
libraries in the same fashion (there are cross dependencies here).
As part of this, also introduce a dependency check test that can enforce
basic layering of dependencies. For example, this lets us ensure that
non-test Carbon code only depends on LLVM and Clang despite having other
libraries available. There remains some cleanup to improve the way these
dependency tests work, but this at least ensures we don't regress.
I've also provided workarounds to allow both Carbon code and LLVM code
to freely be used with GoogleTest (and other `std::ostream` based
output code). This is done by extending the code in
`//common/ostream.h`. One downside is that it requires opening the
`llvm` namespace and adding an ADL_found overload there. I think on
balance this is still a win and doesn't make me too nervous.
The new version of GoogleTest requires printing more often from matchers
and so I've also added several printing routines to types that
previously didn't require them. Otherwise, most of the updates are just
using the more conventional upstream style of including the headers and
adding `ostream.h` where it is needed.
I did consider moving code over to use `std::ostream` instead of LLVM's
`raw_ostream`, but the advantages of not doing virtual dispatch still
seem significant, and it also seems good to retain access to LLVM's
formatting utilities built around `raw_ostream` given that we can't pull
arbitrary dependencies into Carbon code outside of test code.
All of this was slightly motivated by requests for newer features in
GoogleTest, but much more-so by my desire to have access to Google
Benchmark and Abseil when writing benchmarks. For example, using
Abseil's random number generator seems extremely helpful when generating
inputs for benchmarks. The growing dependencies between these packages
further motivated me to just pull them all in and ensure they worked
well.
This should clean up our top level directory and the build patterns.
No non-mechanical edits here. Just injecting `toolchain/` and
`TOOLCHAIN_` and then running formatting tools.
Richard SmithandJon Ross-Perkins
Thomas Köppe
ottmar-zittlau
josh11bandJosh L
Jon Meow
Richard Smith