Consolidates both main libraries into `//testing/base`, and factors out
the exe path handling for benchmarks and unit tests into a common
library to remove duplication. Refactors how that logic is managed to be
cleaner and avoid a confusing bool that came up in code review.
Updates all the tests and benchmarks that use these. I still need to
update other benchmarks to use the same main, but I wanted to keep this
PR somewhat minimal.
This also fixes a bug noticed in passing that the compilation benchmark
didn't have the required dependency on the benchmark library itself,
just the benchmark main library.
There was a file that got into the testing filegroups but not the actual
built tarball. The result was that the installation failed to locate
itself correctly.
We also didn't have any testing to catch this. I'd like to add a bit
more end-to-end testing long-term, but for now just add a Python test
that validates the same set of files are in both.
Sadly, building and testing the compressed release is really slow and
not likely worthwhile outside of the actual release, but it would be
good to catch this stuff earlier. I tried switching from `bz2` to `gz`,
which made very little file size difference (so we should do it
anyways), and it is still too slow to do routinely. Instead, I've added
a Starlark macro that builds both the `pkg_tar` and the `py_test` to
validate it for both uncompressed `tar` and compressed `tar.gz`. This
lets us continuously test the `tar` version, and just test the `tar.gz`
in the nightly release run.
Updates the nightly release workflow to both test, run this specific
test, and use `gz`.
Last but not least, removes the `pkg_zip` as I don't think we have a use
case for this yet and some TODOs that should have been removed when the
version got added.
This adds a defined Carbon version to the Bazel build and codebase that
can be used both to implement features like version checks and to report
a meaningful version on the command line. This replaces a hard-coded
string and a TODO in the driver.
As part of this, it adds support for defining the version in Bazel, and
special build flags for overriding relevant parts such as the
pre-release marker used. The exact structure and meaning of our version
string, including the pre-release parts, is implemented here in line
with the draft proposal:
https://docs.google.com/document/d/11S5VAPe5Pm_BZPlajWrqDDVr9qc7-7tS2VshqO0wWkk/edit?resourcekey=0-2YFC9Uvl4puuDnWlr2MmYw
This also introduces a workspace status command to the repository to
extract the git commit SHA and other information when building, and the
logic to stamp that into binaries as part of the version string when
useful. The technique used leverages weak symbols with whole archive
linking to allow a link-time override of unstamped data with stamped
data in the leaf executable. This makes building with `--stamp` a
reasonable default, especially for development builds. The CI system is
explicitly opted out of this as there it has no benefit.
Last but not least, all of these are wired into the install rules so
that we build installable packages with the version number in a
conventional place in the directory and filename.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This also allows us to default construct installs in an error state,
which is useful for cases like fuzzers where we want to default
construct a global, but then re-initilaize it That said, happy to
consider alternative designs here.
Despite this name not being in the runfiles manifest, this works in
Bazel because of how the module and workspace are configured. I should
have realized that as it was used elsewhere as well. This should
consolidate all of our runfiles paths to use the exact same patterns
now. These were the only remaining divergences.
Jon correctly pointed out that this wasn't being used any more.
Originally, the code needed a different tree for the install prefix in
order to test different code paths of detection, but since those are now
explicit in the code it is already using a single tree. But I never
updated the actual BUILD file to stop building and depending on the
extra prefix.
Just spotted these while looking at warnings that seem to fire on our
code are probably are things we'd fix if we saw them. None of these seem
important FWIW.
Also removes a redundant flag that is part of `-Wall`.
I have a follow-up for the high-value warning I spotted that motivated
me to look at all of this. But it's noisy so kept it as a separate PR.
This removes the `data_dir` from the driver favoring the installation
abstraction for the both locating the prelude and linking utilities.
With this, an installed toolchain should also be able to compile and
link Carbon successfully, and the build of the examples should exercise
this path almost exactly. (The only difference is using the driver
`cc_binary` directly rather than relying on the symlink from inside the
install tree.)
This takes the installation layout and replicates it using `rules_pkg`
to build either a tarball or a zip file of the toolchain. Correctly
manages file permissions and symlinks, etc.
There are some big remaining things here:
- Figure out how we want to test this. We can add shell tests maybe?
A bit awkward. Nicer would be to make the `//examples` tree build
using this rather than the more native-bazel install data, however
building these is quite slow and it seems bad to pay that cost
constantly so dedicated testing is probably better. For now, I've
tested these manually.
- Need to add versions to the toolchain and then thread them through
here so they install properly as a versioned release.
But my primary goal for now is just to be able to validate that the
install tree is working outside of Bazel and this does enough for that.
The above will be longer-term things.
The install directory contains the BUILD logic for creating an
installable tree of data files and executables for the toolchain, and
a library to facilitate toolchain code accessing the paths to their data
within this installation.
Then adds an installation of LLD in a synthetic LLVM installation, and
teaches the Clang runner to configure this and use it for linking
instead of the system linker.
Currently, the install paths only really manage access to the LLVM
binaries installed and used by the Clang runner for linking, but
eventually other data files like the prelude and runtime libraries will
be fleshed out as well. There are TODOs for moving more things over here
such as the prelude.
One interesting aspect of this is where to put helpers like parts of
LLVM in our install. This PR suggests nesting those files under
`lib/carbon`. While using a `lib` subdirectory isn't a perfect fit for
the FHS (Filesystem Hierarchy Standard), having a single location where
private data is collected is significantly superior to spreading them
across the system. This also matches similar patterns used by Clang
itself and several other language toolchains and standard libraries.
The install directory also provides a natural place for us to build out
packaging rules to create installable packages in various formats, but
that remains future work.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Chandler Carruth
