mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-11-10 12:28:25 +00:00
Newer versions of `protobuf` depend on Rust to a limited extent. Builds don't use it, and _our_ builds rarely even touch on `protobuf`. But the way CI was running `bazel mod` commands in `action.yml` forced fully evaluating all the module extensions, including Rust. This ends up both hitting end points we didn't allow and being really expensive. We can skip that and keep the more targeted `cquery` that doesn't do anything beyond what our builds need. We also still have a test that the lock file is accurate, so we don't lose any coverage here, it just makes things faster. That said, we _do_ want to be able to hit the Rust dependency if and where we (transitively) need it, and we'll at least need our `prek` checks to be able to resolve them and rebuilding the lock file after an update, so the relevant end points are added to our hardened runner as allowed. The second issue is that `rules_python` in version 2.0 and newer is using a Bazel feature we disable. There is an escape hatch though that we can enable in our `.bazelrc` so it keeps working, and that works even with the current version. Once this lands, the Dependabot update of the Bazel module versions should be able to make more progress. Assisted-by: Antigravity with Claude Opus
144 lines
6.1 KiB
YAML
144 lines
6.1 KiB
YAML
# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
|
|
# Exceptions. See /LICENSE for license information.
|
|
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
|
|
name: Setup build environment (macOS)
|
|
inputs:
|
|
matrix_runner:
|
|
required: true
|
|
remote_cache_upload:
|
|
required: true
|
|
runs:
|
|
using: composite
|
|
steps:
|
|
# Setup Python and related tools with uv.
|
|
- name: Set up uv and Python
|
|
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
|
with:
|
|
enable-cache: true
|
|
version: '0.11.15'
|
|
|
|
- uses: ./.github/actions/build-setup-macos
|
|
if: startsWith(inputs.matrix_runner, 'macos')
|
|
with:
|
|
matrix_runner: ${{ inputs.matrix_runner }}
|
|
|
|
- uses: ./.github/actions/build-setup-ubuntu
|
|
if: startsWith(inputs.matrix_runner, 'ubuntu')
|
|
|
|
# Print the various tool paths and versions to help in debugging.
|
|
- name: Print tool debugging info
|
|
shell: bash
|
|
run: |
|
|
echo '*** PATH'
|
|
echo $PATH
|
|
echo '*** bazelisk'
|
|
which bazelisk
|
|
bazelisk --version
|
|
echo '*** run_bazel.py'
|
|
./scripts/run_bazel.py --version
|
|
echo '*** uv'
|
|
which uv
|
|
uv --version
|
|
uv python list --only-installed
|
|
echo '*** clang'
|
|
which clang
|
|
clang --version
|
|
echo '*** clang++'
|
|
which clang++
|
|
clang++ --version
|
|
echo '*** clang-tidy'
|
|
which clang-tidy
|
|
clang-tidy --version
|
|
|
|
# Add our bazel configuration and print basic info to ease debugging.
|
|
- name: Configure Bazel and print info
|
|
env:
|
|
# Add a cache version for changes that bazel won't otherwise detect,
|
|
# like llvm version changes.
|
|
CACHE_VERSION: 1
|
|
shell: bash
|
|
run: |
|
|
cat >user.bazelrc <<EOF
|
|
# Disable the local disk cache as we use a remote cache and don't want
|
|
# two copies of every output taking up disk space. The only way to
|
|
# disable the disk cache is with an empty string:
|
|
# https://github.com/bazelbuild/bazel/issues/5308
|
|
build --disk_cache=
|
|
|
|
# Enable remote cache for our CI but minimize downloads.
|
|
build --remote_cache=https://storage.googleapis.com/carbon-builds-github-v${CACHE_VERSION}
|
|
build --remote_download_outputs=minimal
|
|
|
|
# Allow passing targets that are incompatible so that our explicit
|
|
# target lists work more like //... wild card patterns in CI. In CI,
|
|
# we're using explicit target lists to prune to a minimal set of
|
|
# dependencies, and so skipping incompatible targets is the expected
|
|
# behavior.
|
|
build --skip_incompatible_explicit_targets
|
|
|
|
# We import a special key into every action in order to key the Bazel
|
|
# remote cache in a way that avoids collisions between different
|
|
# runners. Anything that might change the system external to Bazel but
|
|
# not be fully captured by the sand-boxing of the build should be used
|
|
# as part of the key. We don't need to use the CPU target for example,
|
|
# as that is captured by Bazel's configuration of each action. And the
|
|
# Clang version is incorporated by our Clang toolchain setup. But we
|
|
# do need to capture any differences between GitHub runner OSes that
|
|
# don't impact the Bazel configuration to avoid collisions between
|
|
# those.
|
|
build --action_env=BAZEL_REMOTE_CACHE_KEY=github-action-${{ inputs.matrix_runner }}
|
|
|
|
build ${{ inputs.remote_cache_upload }}
|
|
|
|
# Set an artificially high jobs count. This flag controls the number
|
|
# of concurrency Bazel itself uses, which is essential for actions
|
|
# that are internally blocked on for example downloading results form
|
|
# the cache above. Without setting this high, Bazel will pick a small
|
|
# number based on the available host CPUs and the reality will be a
|
|
# long chain of largely serialized download events with little or no
|
|
# usage of the host machine. Fortunately, local actions are
|
|
# *separately* gated on '--local_*_resources' that will avoid a large
|
|
# jobs value overwhelming the host. There is a bug to make downloads
|
|
# behave completely asynchronously and remove the need for this filed
|
|
# back in 2018 but work seemed to not finish:
|
|
# https://github.com/bazelbuild/bazel/issues/6394
|
|
#
|
|
# There is a new effort (yay!) but until then it seems worth using the
|
|
# workaround of a high jobs value. The biggest downside (increased
|
|
# heap usage) seems like it isn't currently a big loss for our builds.
|
|
#
|
|
# Higher values like 50 have led to CI failures with network errors
|
|
# and IOExceptions, see
|
|
# https://discord.com/channels/655572317891461132/707150492370862090/1151605725576056934
|
|
build --jobs=32
|
|
|
|
# Avoid any cache impact from build stamping in CI.
|
|
build --nostamp
|
|
|
|
# General build options.
|
|
build --verbose_failures
|
|
test --test_output=errors
|
|
EOF
|
|
./scripts/run_bazel.py info
|
|
|
|
- name: Run bazel to sync deps with retry
|
|
shell: bash
|
|
run: |
|
|
# GitHub sometimes has a high failure rate for Bazel's downloads (even
|
|
# from GitHub URLs). Bazel exits with `1` on HTTP errors, which is hard
|
|
# to distinguish from a normal, permanent error.
|
|
#
|
|
# This workaround runs fast commands that should always pass (although
|
|
# they may be broken by an invalid PR). All errors are retried. The hope
|
|
# is that this caches necessary downloads, allowing later commands to
|
|
# more reliably succeed without retrying "permanent" errors.
|
|
#
|
|
# Disable lockfile updates, because some actions want to see
|
|
# differences. This avoids `bazel mod deps`, which evaluates every
|
|
# module extension; without the lockfile, that includes expensive ones
|
|
# that builds don't use, like the Rust crates newer protobuf versions
|
|
# depend on.
|
|
./scripts/run_bazel.py --attempts=5 --retry-all-errors \
|
|
cquery --lockfile_mode=off //... | wc -l
|