Commit Graph
13 Commits
Author SHA1 Message Date
Chandler CarruthandJon Ross-Perkins d3a5b0eee7 Add utilities for managing a toolchain install, and install and use LLD. (#3993)
The install directory contains the BUILD logic for creating an
installable tree of data files and executables for the toolchain, and
a library to facilitate toolchain code accessing the paths to their data
within this installation.

Then adds an installation of LLD in a synthetic LLVM installation, and
teaches the Clang runner to configure this and use it for linking
instead of the system linker.

Currently, the install paths only really manage access to the LLVM
binaries installed and used by the Clang runner for linking, but
eventually other data files like the prelude and runtime libraries will
be fleshed out as well. There are TODOs for moving more things over here
such as the prelude.

One interesting aspect of this is where to put helpers like parts of
LLVM in our install. This PR suggests nesting those files under
`lib/carbon`. While using a `lib` subdirectory isn't a perfect fit for
the FHS (Filesystem Hierarchy Standard), having a single location where
private data is collected is significantly superior to spreading them
across the system. This also matches similar patterns used by Clang
itself and several other language toolchains and standard libraries.

The install directory also provides a natural place for us to build out
packaging rules to create installable packages in various formats, but
that remains future work.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-06-01 03:19:52 +00:00
Jon Ross-Perkins a3b1c433be Remove legacy repo_name settings (#3772)
I'd kept these in to separate the bazel module update from the BUILD
file changes, then forgot about it. I think all of these can be cleanly
removed now. I think it's something we should clean up for consistency
with the bazel central repository names; I think it's best to reduce
that divergence.

llvm_zlib and llvm_zstd remain because of how llvm depends on the
particular names.
2024-03-13 22:58:56 +00:00
Richard SmithandJon Ross-Perkins bf8697113a Move llvm::Initialize* calls to main. (#3449)
Per their documentation, the `llvm::Initialize*` functions are only
supposed to be called by the main program, not by a library like
toolchain/codegen. Fixes a hang due to a data race in multithreaded
autoupdate.

Add a utility class `Carbon::InitLLVM` to do the common LLVM
initialization shared by all Carbon tools, optionally including
initializing the LLVM targets. Because the LLVM targets add a lot of
binary size, only initialize them for binaries that opt in by depending
on a new target `//common:all_llvm_targets`.

Also fix `//explorer:file_test` and `//explorer:file_test.trace` to
share a binary rather than linking an identical binary twice.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2023-12-07 01:45:47 +00:00
Jonathan B. CoeandChandler Carruth 8c28a0494e Add size="small" to test targets where advised (#3326)
Running `bazel test //...` reported:

```
Test execution time outside of range for MODERATE tests.
Consider setting timeout="short" or size="small".
```

This change adds size="small" to avoid such warnings being reported.

---------

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2023-10-24 06:52:00 +00:00
Jon Ross-Perkins 7a1d54f8e3 Merge filesystem logic into FileTest and update test files. (#3183)
two_files.carbon has odd STDOUT placement, but will pursue that
separately.
2023-09-05 18:40:00 +00:00
Jon Ross-Perkins 605763d62d Add lint fixes to the buildifier setup. (#3109)
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?

`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).

Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
2023-08-22 21:01:42 +00:00
Jon Ross-Perkins b5167b2d69 Implement autoupdate for file_test. (#3043)
I've migrated the toolchain autoupdate scripts here, I just need a
little more time to do the explorer side (which I need to check
performance, that may require multithreading as we do in the current
script). However, this felt substantial enough to share and it means I
can handle autoupdate in more of the toolchain, including preparatory
work for autoupdate on multi-file tests.

Once explorer is done I'll remove the old script.
2023-08-02 21:32:21 +00:00
Jon Ross-Perkins 3f24aa28b7 Introduce flag handling into file_test. (#3030)
I'm integrating absl flag support with a few thoughts here...

1. It simplifies the handling of files in file_test.
    - Removes the need for a separate subset target.
3. Looking forward, I'm planning to add a flag to allow for autoupdate
of golden files.
4. In explorer, there's been confusion about having file_test run tests
"twice" so hopefully it's clearer when it's a separate target with a
different flag on the target.
2023-07-27 23:36:21 +00:00
Jon Ross-Perkins d18c1347d7 Migrate compatible uses to TestRawOstream. (#2891)
Replacing direct raw_string_ostream uses. I figure the wrapper should be used more consistently.

There are still remaining raw_string_ostream uses that weren't compatible -- I'm continuing to look at those, but felt it was cleaner to have this on its own.
2023-06-14 09:31:56 -07:00
Jon Ross-Perkins 6586179c8d Add support for splitting a test file. (#2876)
I'm looking at this as I start thinking about handling `import`. Syntax is based on llvm's `split-file` tool.

The `std::vector` -> `llvm::SmallVector` switch is minor, I'm doing it here because I had to touch everything anyways and I think for tests I'll lean slightly more towards the toolchain's way of doing things versus explorer's.
2023-06-07 08:56:20 -07:00
Jon Ross-Perkins 6b7a522b3f Provide local paths for file tests. (#2830)
The intent of this change is that instead of paths looking like `explorer/testdata/foo/bar.carbon` (repo-relative), they're now just `bar.carbon` (local). The consequence is that paths should be a bit more durable in various environments, and just paths should be shorter and easier to read.

The explorer's prelude is an exception to this since it comes from data, rather than being the test target. Due to the change in approaches, it needs the regex again.

Uses #2829
2023-05-18 10:20:51 -07:00
Jon Ross-Perkins 4247a39acd Add testonly to file_test (#2826)
I don't know why gtest doesn't have testonly set in public, but that's the root of this.
2023-05-15 16:31:57 -07:00
Jon Ross-PerkinsandChandler Carruth 941e60ade6 Add framework for replacing lit with cc_test (#2814)
This is really part of #2811, but is extracted out to allow a little review in parallelism because #2811 expects #2813. Getting this in will allow migration of toolchain tests, whereas #2811 is focused on explorer tests. For explorer test timing information, see #2811.

The syntax being used for matching deliberately mirrors the `FileCheck` setup, partly for compatibility if something changes, partly so there's nothing new to learn, partly so that we don't need to build more test updating.

Individual tests look like:

```
[ RUN      ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon

To test this file alone, run:
  bazel test //explorer/parse_and_execute:file_test.subset --test_arg=explorer/parse_and_execute/testdata/assert/convert.carbon

[       OK ] ParseAndExecuteTestFile.explorer/parse_and_execute/testdata/assert/convert.carbon (202 ms)
```

The printed command line is intended to assist developers in debugging a single test, particularly when sharding the main test. The use of a single `.subset` target means the total number of targets is constant even as the number of test files increases, which may be important for some `bazel` execution environments. I plan to make similar changes to the `glob_sh_run` implementation so that we have consistent setups, i.e. that we no longer create target-per-file scaling risks.

This uses `native_test` to share the test binary, avoiding re-linking if files are individually run.

Investigation did reveal a mistake where STDOUT/STDERR wasn't prefixed on empty output lines; this PR fixes that mistake, so that output is fully covered.

Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
2023-05-12 10:05:08 -07:00