From a1305604f28138821822d5c4d76448198f1bb12c Mon Sep 17 00:00:00 2001 From: Chandler Carruth Date: Thu, 5 Nov 2026 18:11:31 +0000 Subject: [PATCH] Get Bazel and CI ready for updated `protobuf` and `rules_python` (#8094) Newer versions of `protobuf` depend on Rust to a limited extent. Builds don't use it, and _our_ builds rarely even touch on `protobuf`. But the way CI was running `bazel mod` commands in `action.yml` forced fully evaluating all the module extensions, including Rust. This ends up both hitting end points we didn't allow and being really expensive. We can skip that and keep the more targeted `cquery` that doesn't do anything beyond what our builds need. We also still have a test that the lock file is accurate, so we don't lose any coverage here, it just makes things faster. That said, we _do_ want to be able to hit the Rust dependency if and where we (transitively) need it, and we'll at least need our `prek` checks to be able to resolve them and rebuilding the lock file after an update, so the relevant end points are added to our hardened runner as allowed. The second issue is that `rules_python` in version 2.0 and newer is using a Bazel feature we disable. There is an escape hatch though that we can enable in our `.bazelrc` so it keeps working, and that works even with the current version. Once this lands, the Dependabot update of the Bazel module versions should be able to make more progress. Assisted-by: Antigravity with Claude Opus --- .bazelrc | 5 +++++ .github/actions/build-setup-common/action.yml | 7 ++++--- .github/workflows/clangd_tidy.yaml | 3 +++ .github/workflows/nightly_release.yaml | 3 +++ .github/workflows/prek.yaml | 3 +++ .github/workflows/tests.yaml | 3 +++ MODULE.bazel | 5 +++++ 7 files changed, 26 insertions(+), 3 deletions(-) diff --git a/.bazelrc b/.bazelrc index 65aabe6d9c6c..d0e669137df7 100644 --- a/.bazelrc +++ b/.bazelrc @@ -125,6 +125,11 @@ common:linux --define=pfm=1 # Disables `actions.declare_symlink`. Done for cross-environment support. common --allow_unresolved_symlinks=false +# Starting with rules_python 2.0, Python binaries use venvs created with +# `actions.declare_symlink`, which we disable above. Have it create them at +# runtime instead. +common --@rules_python//python/config_settings:venvs_use_declare_symlink=no + # Removes the leading `/proc/self/cwd/` from file paths in the debug info. Some # tools like VS Code don't understand `/proc/self/cwd` in places like terminal # stack dumps, but do understand paths relative to the workspace root. diff --git a/.github/actions/build-setup-common/action.yml b/.github/actions/build-setup-common/action.yml index a60239422aa4..20e4c518738d 100644 --- a/.github/actions/build-setup-common/action.yml +++ b/.github/actions/build-setup-common/action.yml @@ -135,8 +135,9 @@ runs: # more reliably succeed without retrying "permanent" errors. # # Disable lockfile updates, because some actions want to see - # differences. - ./scripts/run_bazel.py --attempts=5 --retry-all-errors \ - mod --lockfile_mode=off deps + # differences. This avoids `bazel mod deps`, which evaluates every + # module extension; without the lockfile, that includes expensive ones + # that builds don't use, like the Rust crates newer protobuf versions + # depend on. ./scripts/run_bazel.py --attempts=5 --retry-all-errors \ cquery --lockfile_mode=off //... | wc -l diff --git a/.github/workflows/clangd_tidy.yaml b/.github/workflows/clangd_tidy.yaml index 3b32ebc2d92e..0a1cbc1979a5 100644 --- a/.github/workflows/clangd_tidy.yaml +++ b/.github/workflows/clangd_tidy.yaml @@ -44,6 +44,7 @@ jobs: formulae.brew.sh:443 github.com:443 go.dev:443 + index.crates.io:443 mirror.bazel.build:443 mirrors.kernel.org:443 nodejs.org:443 @@ -56,6 +57,8 @@ jobs: release-assets.githubusercontent.com:443 releases.astral.sh:443 releases.bazel.build:443 + static.crates.io:443 + static.rust-lang.org:443 storage.googleapis.com:443 uploads.github.com:443 www.googleapis.com:443 diff --git a/.github/workflows/nightly_release.yaml b/.github/workflows/nightly_release.yaml index 4b833ccfb0f5..f24bb1307b6b 100644 --- a/.github/workflows/nightly_release.yaml +++ b/.github/workflows/nightly_release.yaml @@ -55,6 +55,7 @@ jobs: formulae.brew.sh:443 github.com:443 go.dev:443 + index.crates.io:443 mirror.bazel.build:443 mirrors.kernel.org:443 nodejs.org:443 @@ -66,6 +67,8 @@ jobs: registry.npmjs.org:443 release-assets.githubusercontent.com:443 releases.bazel.build:443 + static.crates.io:443 + static.rust-lang.org:443 storage.googleapis.com:443 uploads.github.com:443 www.googleapis.com:443 diff --git a/.github/workflows/prek.yaml b/.github/workflows/prek.yaml index bae6cb99878c..6ccfb0dce975 100644 --- a/.github/workflows/prek.yaml +++ b/.github/workflows/prek.yaml @@ -36,6 +36,7 @@ jobs: formulae.brew.sh:443 github.com:443 go.dev:443 + index.crates.io:443 mirror.bazel.build:443 mirrors.kernel.org:443 nodejs.org:443 @@ -48,6 +49,8 @@ jobs: release-assets.githubusercontent.com:443 releases.astral.sh:443 releases.bazel.build:443 + static.crates.io:443 + static.rust-lang.org:443 storage.googleapis.com:443 uploads.github.com:443 www.googleapis.com:443 diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index 1929ddac6ab2..5afae97ac630 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -67,6 +67,7 @@ jobs: formulae.brew.sh:443 github.com:443 go.dev:443 + index.crates.io:443 mirror.bazel.build:443 mirrors.kernel.org:443 nodejs.org:443 @@ -79,6 +80,8 @@ jobs: release-assets.githubusercontent.com:443 releases.astral.sh:443 releases.bazel.build:443 + static.crates.io:443 + static.rust-lang.org:443 storage.googleapis.com:443 uploads.github.com:443 www.googleapis.com:443 diff --git a/MODULE.bazel b/MODULE.bazel index d1a78504c1c7..a99136082b48 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -20,6 +20,11 @@ of: `MODULE.bazel.lock`; create a PR with those changes in order to include the host platform. +`bazel mod deps` evaluates every module extension, but reuses results recorded +in `MODULE.bazel.lock`. Re-evaluating some extensions can be expensive; for +example, the Rust crates newer protobuf versions depend on require downloading +a Rust toolchain. + For updates, run `scripts/query_module_versions.py` to list the latest package versions. """