This brings some fixes:
- The handling of `zlib` and `zstd` are much cleaner
- Three of our patches are no longer needed
This also includes the fixes from #6562
It also moves us from `zlib` to `zlib-ng` which is a much better basis
for what we want, and likely makes our toolchain faster when generating
debug info at least.
It fixes another API change in terms of which headers provide the
`createInvocation` we use.
Lastly, it cleans up the deps test to correctly recognize the wrappers
for `zlib-ng` and `zstd`, as well as improving the documentation for why
we allow dependencies on them.
When building in Bazel actions, notably building runtimes, using
absolute paths makes the results non-hermetic and generally less
cache-friendly.
This restructures the code to only form an absolute path as part of the
`bazel run` change of working directory. It also tries to make the API
for doing this a bit more clear by taking the `exe_path` and
transforming it internally.
To support this, this PR also generalizes the `RemovingDir` to support
relative paths. While these can be tricky -- the working directory needs
to not change while they exist -- that isn't a reason to fully exclude
them and they're useful for implementing relative-path runtimes, etc.
There are two uses I'm not converting here, that seem to want the
"shortest" behavior. For everything else, I'm going to `zip_equal` since
it's more restrictive.
I wish `zip` were named `zip_shortest`.
This fixes the flakiness caused by reuse of inode values when refreshing
stale cache entries by keeping the relevant directory open even as it is
unlinked from the filesystem.
It does this in two places, as technically we had the same flakiness in
two tests. However, the second test was broken and not testing what it
intended to due to confusing off-by-one naming and a typo. I've tried to
improve the naming, removed the typo, and added the parallel flakiness
fix.
This test was also egregiously slow because we ended up building too
many runtimes and trying to prune stale runtimes while holding a file
lock on _all_ runtimes -- a scenario that is not what the code was
designed for in the first place. Fixing that makes the test go from 10s
to 1s in runtime, and makes it much easier to test for flakiness.
Now appears to pass 100% of the 10k runs I did.
Closes#6168
This layer allows runtimes to be built on-demand but cached in a
consistent and re-usable location on the system. It handles careful
filesystem operations to ensure consistency even in the face of multiple
versions and build configurations.
This addresses a number of TODOs from the initial runtimes building
on-demand, and sets the stage to scale up to more runtimes.
This doesn't switch on-demand runtimes to be on by default, I wanted to
wait and make that change as a separate step.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Chandler Carruth
Jon Ross-Perkins
