Files
carbon-lang/.github/actions/build-setup-common/action.yml
T
Chandler Carruth a1305604f2 Get Bazel and CI ready for updated protobuf and rules_python (#8094)
Newer versions of `protobuf` depend on Rust to a limited extent. Builds
don't use it, and _our_ builds rarely even touch on `protobuf`. But the
way CI was running `bazel mod` commands in `action.yml` forced fully
evaluating all the module extensions, including Rust. This ends up both
hitting end points we didn't allow and being really expensive. We can
skip that and keep the more targeted `cquery` that doesn't do anything
beyond what our builds need. We also still have a test that the lock
file is accurate, so we don't lose any coverage here, it just makes
things faster.

That said, we _do_ want to be able to hit the Rust dependency if and
where we (transitively) need it, and we'll at least need our `prek`
checks to be able to resolve them and rebuilding the lock file after an
update, so the relevant end points are added to our hardened runner as
allowed.

The second issue is that `rules_python` in version 2.0 and newer is
using a Bazel feature we disable. There is an escape hatch though that
we can enable in our `.bazelrc` so it keeps working, and that works even
with the current version.

Once this lands, the Dependabot update of the Bazel module versions
should be able to make more progress.

Assisted-by: Antigravity with Claude Opus
2026-11-05 18:11:31 +00:00

144 lines
6.1 KiB
YAML

# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
# Exceptions. See /LICENSE for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
name: Setup build environment (macOS)
inputs:
matrix_runner:
required: true
remote_cache_upload:
required: true
runs:
using: composite
steps:
# Setup Python and related tools with uv.
- name: Set up uv and Python
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
enable-cache: true
version: '0.11.15'
- uses: ./.github/actions/build-setup-macos
if: startsWith(inputs.matrix_runner, 'macos')
with:
matrix_runner: ${{ inputs.matrix_runner }}
- uses: ./.github/actions/build-setup-ubuntu
if: startsWith(inputs.matrix_runner, 'ubuntu')
# Print the various tool paths and versions to help in debugging.
- name: Print tool debugging info
shell: bash
run: |
echo '*** PATH'
echo $PATH
echo '*** bazelisk'
which bazelisk
bazelisk --version
echo '*** run_bazel.py'
./scripts/run_bazel.py --version
echo '*** uv'
which uv
uv --version
uv python list --only-installed
echo '*** clang'
which clang
clang --version
echo '*** clang++'
which clang++
clang++ --version
echo '*** clang-tidy'
which clang-tidy
clang-tidy --version
# Add our bazel configuration and print basic info to ease debugging.
- name: Configure Bazel and print info
env:
# Add a cache version for changes that bazel won't otherwise detect,
# like llvm version changes.
CACHE_VERSION: 1
shell: bash
run: |
cat >user.bazelrc <<EOF
# Disable the local disk cache as we use a remote cache and don't want
# two copies of every output taking up disk space. The only way to
# disable the disk cache is with an empty string:
# https://github.com/bazelbuild/bazel/issues/5308
build --disk_cache=
# Enable remote cache for our CI but minimize downloads.
build --remote_cache=https://storage.googleapis.com/carbon-builds-github-v${CACHE_VERSION}
build --remote_download_outputs=minimal
# Allow passing targets that are incompatible so that our explicit
# target lists work more like //... wild card patterns in CI. In CI,
# we're using explicit target lists to prune to a minimal set of
# dependencies, and so skipping incompatible targets is the expected
# behavior.
build --skip_incompatible_explicit_targets
# We import a special key into every action in order to key the Bazel
# remote cache in a way that avoids collisions between different
# runners. Anything that might change the system external to Bazel but
# not be fully captured by the sand-boxing of the build should be used
# as part of the key. We don't need to use the CPU target for example,
# as that is captured by Bazel's configuration of each action. And the
# Clang version is incorporated by our Clang toolchain setup. But we
# do need to capture any differences between GitHub runner OSes that
# don't impact the Bazel configuration to avoid collisions between
# those.
build --action_env=BAZEL_REMOTE_CACHE_KEY=github-action-${{ inputs.matrix_runner }}
build ${{ inputs.remote_cache_upload }}
# Set an artificially high jobs count. This flag controls the number
# of concurrency Bazel itself uses, which is essential for actions
# that are internally blocked on for example downloading results form
# the cache above. Without setting this high, Bazel will pick a small
# number based on the available host CPUs and the reality will be a
# long chain of largely serialized download events with little or no
# usage of the host machine. Fortunately, local actions are
# *separately* gated on '--local_*_resources' that will avoid a large
# jobs value overwhelming the host. There is a bug to make downloads
# behave completely asynchronously and remove the need for this filed
# back in 2018 but work seemed to not finish:
# https://github.com/bazelbuild/bazel/issues/6394
#
# There is a new effort (yay!) but until then it seems worth using the
# workaround of a high jobs value. The biggest downside (increased
# heap usage) seems like it isn't currently a big loss for our builds.
#
# Higher values like 50 have led to CI failures with network errors
# and IOExceptions, see
# https://discord.com/channels/655572317891461132/707150492370862090/1151605725576056934
build --jobs=32
# Avoid any cache impact from build stamping in CI.
build --nostamp
# General build options.
build --verbose_failures
test --test_output=errors
EOF
./scripts/run_bazel.py info
- name: Run bazel to sync deps with retry
shell: bash
run: |
# GitHub sometimes has a high failure rate for Bazel's downloads (even
# from GitHub URLs). Bazel exits with `1` on HTTP errors, which is hard
# to distinguish from a normal, permanent error.
#
# This workaround runs fast commands that should always pass (although
# they may be broken by an invalid PR). All errors are retried. The hope
# is that this caches necessary downloads, allowing later commands to
# more reliably succeed without retrying "permanent" errors.
#
# Disable lockfile updates, because some actions want to see
# differences. This avoids `bazel mod deps`, which evaluates every
# module extension; without the lockfile, that includes expensive ones
# that builds don't use, like the Rust crates newer protobuf versions
# depend on.
./scripts/run_bazel.py --attempts=5 --retry-all-errors \
cquery --lockfile_mode=off //... | wc -l