Get Bazel and CI ready for updated protobuf and rules_python (#8094)

Newer versions of `protobuf` depend on Rust to a limited extent. Builds
don't use it, and _our_ builds rarely even touch on `protobuf`. But the
way CI was running `bazel mod` commands in `action.yml` forced fully
evaluating all the module extensions, including Rust. This ends up both
hitting end points we didn't allow and being really expensive. We can
skip that and keep the more targeted `cquery` that doesn't do anything
beyond what our builds need. We also still have a test that the lock
file is accurate, so we don't lose any coverage here, it just makes
things faster.

That said, we _do_ want to be able to hit the Rust dependency if and
where we (transitively) need it, and we'll at least need our `prek`
checks to be able to resolve them and rebuilding the lock file after an
update, so the relevant end points are added to our hardened runner as
allowed.

The second issue is that `rules_python` in version 2.0 and newer is
using a Bazel feature we disable. There is an escape hatch though that
we can enable in our `.bazelrc` so it keeps working, and that works even
with the current version.

Once this lands, the Dependabot update of the Bazel module versions
should be able to make more progress.

Assisted-by: Antigravity with Claude Opus

.bazelrc

@@ -125,6 +125,11 @@ common:linux --define=pfm=1
 # Disables `actions.declare_symlink`. Done for cross-environment support.
 common --allow_unresolved_symlinks=false
 
+# Starting with rules_python 2.0, Python binaries use venvs created with
+# `actions.declare_symlink`, which we disable above. Have it create them at
+# runtime instead.
+common --@rules_python//python/config_settings:venvs_use_declare_symlink=no
+
 # Removes the leading `/proc/self/cwd/` from file paths in the debug info. Some
 # tools like VS Code don't understand `/proc/self/cwd` in places like terminal
 # stack dumps, but do understand paths relative to the workspace root.

.github/actions/build-setup-common/action.yml

@@ -135,8 +135,9 @@ runs:
         # more reliably succeed without retrying "permanent" errors.
         #
         # Disable lockfile updates, because some actions want to see
-        # differences.
+        # differences. This avoids `bazel mod deps`, which evaluates every
-        ./scripts/run_bazel.py --attempts=5 --retry-all-errors \
+        # module extension; without the lockfile, that includes expensive ones
-          mod --lockfile_mode=off deps
+        # that builds don't use, like the Rust crates newer protobuf versions
+        # depend on.
         ./scripts/run_bazel.py --attempts=5 --retry-all-errors \
           cquery --lockfile_mode=off //... | wc -l

.github/workflows/clangd_tidy.yaml

@@ -44,6 +44,7 @@ jobs:
             formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
@@ -56,6 +57,8 @@ jobs:
             release-assets.githubusercontent.com:443
             releases.astral.sh:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443

.github/workflows/nightly_release.yaml

@@ -55,6 +55,7 @@ jobs:
             formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
@@ -66,6 +67,8 @@ jobs:
             registry.npmjs.org:443
             release-assets.githubusercontent.com:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443

.github/workflows/prek.yaml

@@ -36,6 +36,7 @@ jobs:
             formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
@@ -48,6 +49,8 @@ jobs:
             release-assets.githubusercontent.com:443
             releases.astral.sh:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443

.github/workflows/tests.yaml

@@ -67,6 +67,7 @@ jobs:
             formulae.brew.sh:443
             github.com:443
             go.dev:443
+            index.crates.io:443
             mirror.bazel.build:443
             mirrors.kernel.org:443
             nodejs.org:443
@@ -79,6 +80,8 @@ jobs:
             release-assets.githubusercontent.com:443
             releases.astral.sh:443
             releases.bazel.build:443
+            static.crates.io:443
+            static.rust-lang.org:443
             storage.googleapis.com:443
             uploads.github.com:443
             www.googleapis.com:443

MODULE.bazel

@@ -20,6 +20,11 @@ of:
         `MODULE.bazel.lock`; create a PR with those changes in order to include
         the host platform.
 
+`bazel mod deps` evaluates every module extension, but reuses results recorded
+in `MODULE.bazel.lock`. Re-evaluating some extensions can be expensive; for
+example, the Rust crates newer protobuf versions depend on require downloading
+a Rust toolchain.
+
 For updates, run `scripts/query_module_versions.py` to list the latest package
 versions.
 """