mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-11-18 15:40:28 +00:00
This adds access to the `target-determinator` tool that computes the possibly impacted set of targets between the current checkout and a specific `git` commit. The tool is downloaded (and cached) with a Python script that wraps it and allows us to easily run it in our CI environment. And finally, switches the CI for pull requests and the merge queue to use this script and run a minimal set of impacted tests rather than all of them. In order to make this as simple as possible, this also merges the previously separate build and test steps of our CI. Note that the CI testing post-submit (push events) continues to use a blanket target pattern so that we have a good backstop in case something outside of what is tracked here changes. The important paths, especially the ones that might be in an interactive critical path, are the PR and merge queue. This has been tested on the `action-test` branch to try to make sure it works. Some example runs for folks to inspect: - `push` event with blanket test: https://github.com/carbon-language/carbon-lang/actions/runs/6070060958/job/16465422569 - A "large" PR that impacts a bunch of toolchain tests: - ubuntu-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070063050/job/16465428107?pr=3188 - macos-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070063050/job/16465428420?pr=3188 - The merge queue for this PR: - ubuntu-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070202665/job/16465825387 - macos-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070202665/job/16465825740 - A "small" PR that impacts one test: - ubuntu-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070659668/job/16467147641?pr=3189 - macos-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070659668/job/16467147933?pr=3189 - The merge queue for the small PR: - ubuntu-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070754584/job/16467427268 - macos-opt: https://github.com/carbon-language/carbon-lang/actions/runs/6070754584/job/16467427736 I have observed one failure while testing with this PR, but I've not yet been able to reproduce it. Every other failure (including empty lists, etc.) I've tried to address. However, we may have to keep an eye on actions after this to make sure there isn't some scenario I've not been able to work through in a test branch. --------- Co-authored-by: Jon Ross-Perkins <jperkins@google.com> Co-authored-by: Geoff Romer <gromer@google.com>
270 lines
11 KiB
YAML
270 lines
11 KiB
YAML
# Part of the Carbon Language project, under the Apache License v2.0 with LLVM
|
|
# Exceptions. See /LICENSE for license information.
|
|
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
|
|
name: test
|
|
|
|
on:
|
|
push:
|
|
branches: [trunk]
|
|
pull_request_target:
|
|
merge_group:
|
|
|
|
# Cancel previous workflows on the PR when there are multiple fast commits.
|
|
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
test:
|
|
strategy:
|
|
matrix:
|
|
# At present, these images are newer than "latest". We use them to test
|
|
# against more recent tooling versions.
|
|
# https://github.com/actions/runner-images
|
|
os: [ubuntu-22.04, macos-12]
|
|
build_mode: [fastbuild, opt]
|
|
runs-on: ${{ matrix.os }}
|
|
|
|
steps:
|
|
# Ubuntu images start with 23GB available, and this adds 14GB more. For
|
|
# comparison, MacOS images have >100GB free.
|
|
- name: Free up disk space (Ubuntu)
|
|
if: matrix.os == 'ubuntu-22.04'
|
|
uses: jlumbroso/free-disk-space@v1.2.0
|
|
with:
|
|
android: true
|
|
dotnet: true
|
|
haskell: true
|
|
# Although we could delete more, if we run into a limit, it provides a
|
|
# little flexibility to get space while trying to shrink the build.
|
|
# There's also support for docker images at head (1.2.0 is still
|
|
# the latest release).
|
|
large-packages: false
|
|
swap-storage: false
|
|
|
|
# Checkout the pull request head or the branch.
|
|
- name: Checkout pull request
|
|
if: github.event_name == 'pull_request_target'
|
|
uses: actions/checkout@v3
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha }}
|
|
|
|
- name: Checkout branch
|
|
if: github.event_name != 'pull_request_target'
|
|
uses: actions/checkout@v3
|
|
|
|
# Tests should only run on applicable paths, but we still need to have an
|
|
# action run for the merge queue. We filter steps based on the paths here,
|
|
# and condition steps on the output.
|
|
- id: filter
|
|
uses: dorny/paths-filter@v2
|
|
with:
|
|
filters: |
|
|
ignore:
|
|
- '**/*.md'
|
|
- 'LICENSE'
|
|
- 'CODEOWNERS'
|
|
- '.git*'
|
|
|
|
# Setup Python and related tools.
|
|
- uses: actions/setup-python@v4
|
|
if: steps.filter.outputs.ignore == 'false'
|
|
with:
|
|
# Match the min version listed in docs/project/contribution_tools.md
|
|
python-version: '3.9'
|
|
|
|
# Use LLVM following:
|
|
# https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md
|
|
# Both 14 and 15 are candidates because GitHub is testing new images.
|
|
- name: Setup LLVM and Clang (macOS)
|
|
if: steps.filter.outputs.ignore == 'false' && matrix.os == 'macos-12'
|
|
run: |
|
|
LLVM_PATH="$(brew --prefix llvm@15)"
|
|
if [[ ! -e "${LLVM_PATH}" ]]; then
|
|
LLVM_PATH="$(brew --prefix llvm@14)"
|
|
fi
|
|
echo "Using ${LLVM_PATH}"
|
|
echo "${LLVM_PATH}/bin" >> $GITHUB_PATH
|
|
echo '*** ls "${LLVM_PATH}"'
|
|
ls "${LLVM_PATH}"
|
|
echo '*** ls "${LLVM_PATH}/bin"'
|
|
ls "${LLVM_PATH}/bin"
|
|
|
|
# Use LLVM following:
|
|
# https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
|
|
# Both 14 and 15 are candidates for forwards compatibility, although 15
|
|
# isn't provided.
|
|
- name: Setup LLVM and Clang (Ubuntu)
|
|
if:
|
|
steps.filter.outputs.ignore == 'false' && matrix.os == 'ubuntu-22.04'
|
|
run: |
|
|
LLVM_PATH="/usr/lib/llvm-15"
|
|
if [[ ! -e "${LLVM_PATH}" ]]; then
|
|
LLVM_PATH="/usr/lib/llvm-14"
|
|
fi
|
|
echo "Using ${LLVM_PATH}"
|
|
echo "${LLVM_PATH}/bin" >> $GITHUB_PATH
|
|
echo '*** ls "${LLVM_PATH}"'
|
|
ls "${LLVM_PATH}"
|
|
echo '*** ls "${LLVM_PATH}/bin"'
|
|
ls "${LLVM_PATH}/bin"
|
|
|
|
# Print the various tool paths and versions to help in debugging.
|
|
- name: Print tool debugging info
|
|
if: steps.filter.outputs.ignore == 'false'
|
|
run: |
|
|
echo '*** PATH'
|
|
echo $PATH
|
|
echo '*** bazelisk'
|
|
which bazelisk
|
|
bazelisk --version
|
|
echo '*** python'
|
|
which python
|
|
python --version
|
|
echo '*** clang'
|
|
which clang
|
|
clang --version
|
|
echo '*** clang++'
|
|
which clang++
|
|
clang++ --version
|
|
|
|
# Extract our access key for our build cache.
|
|
- name: Extract access key
|
|
if: steps.filter.outputs.ignore == 'false'
|
|
env:
|
|
GCP_BUILDS_SERVICE_ACCOUNT: ${{ secrets.GCP_BUILDS_SERVICE_ACCOUNT }}
|
|
run: |
|
|
echo "$GCP_BUILDS_SERVICE_ACCOUNT" \
|
|
| base64 -d > $HOME/gcp-builds-service-account.json
|
|
|
|
# We need to replace the `.` with a `_` for the build cache.
|
|
- name: Setup LLVM and Clang (macOS)
|
|
if: steps.filter.outputs.ignore == 'false' && matrix.os == 'macos-12'
|
|
run: |
|
|
echo "os_for_cache=macos-12" >> $GITHUB_ENV
|
|
- name: Setup LLVM and Clang (Ubuntu)
|
|
if:
|
|
steps.filter.outputs.ignore == 'false' && matrix.os == 'ubuntu-22.04'
|
|
run: |
|
|
echo "os_for_cache=ubuntu-22_04" >> $GITHUB_ENV
|
|
|
|
# Add our bazel configuration and print basic info to ease debugging.
|
|
- name: Configure Bazel and print info
|
|
if: steps.filter.outputs.ignore == 'false'
|
|
env:
|
|
# Add a cache version for changes that bazel won't otherwise detect,
|
|
# like llvm version changes.
|
|
CACHE_VERSION: 1
|
|
run: |
|
|
cat >user.bazelrc <<EOF
|
|
# Enable remote cache for our CI but minimize downloads.
|
|
build --remote_cache=https://storage.googleapis.com/carbon-builds-github-v${CACHE_VERSION}-${{ env.os_for_cache }}
|
|
build --google_credentials=$HOME/gcp-builds-service-account.json
|
|
build --remote_download_minimal
|
|
|
|
# Set an artificially high jobs count. This flag controls the number
|
|
# of concurrency Bazel itself uses, which is essential for actions
|
|
# that are internally blocked on for example downloading results form
|
|
# the cache above. Without setting this high, Bazel will pick a small
|
|
# number based on the available host CPUs and the reality will be a
|
|
# long chain of largely serialized download events with little or no
|
|
# usage of the host machine. Fortunately, local actions are
|
|
# *separately* gated on `--local_*_resources` that will avoid a large
|
|
# jobs value overwhelming the host. There is a bug to make downloads
|
|
# behave completely asynchronously and remove the need for this filed
|
|
# back in 2018 but work seemed to not finish:
|
|
# https://github.com/bazelbuild/bazel/issues/6394
|
|
#
|
|
# There is a new effort (yay!) but until then it seems worth using the
|
|
# workaround of a high jobs value. The biggest downside (increased
|
|
# heap usage) seems like it isn't currently a big loss for our builds.
|
|
build --jobs=200
|
|
|
|
# Set an artificially high jobs count. This flag controls the number
|
|
# of concurrency Bazel itself uses, which is essential for actions
|
|
# that are internally blocked on for example downloading results form
|
|
# the cache above. Without setting this high, Bazel will pick a small
|
|
# number based on the available host CPUs and the reality will be a
|
|
# long chain of largely serialized download events with little or no
|
|
# usage of the host machine. Fortunately, local actions are
|
|
# *separately* gated on `--local_*_resources` that will avoid a large
|
|
# jobs value overwhelming the host. There is a bug to make downloads
|
|
# behave completely asynchronously and remove the need for this filed
|
|
# back in 2018 but work seemed to not finish:
|
|
# https://github.com/bazelbuild/bazel/issues/6394
|
|
#
|
|
# There is a new effort (yay!) but until then it seems worth using the
|
|
# workaround of a high jobs value. The biggest downside (increased
|
|
# heap usage) seems like it isn't currently a big loss for our builds.
|
|
build --jobs=200
|
|
|
|
# General build options.
|
|
build --verbose_failures
|
|
test --test_output=errors
|
|
EOF
|
|
bazelisk info
|
|
|
|
# Just for visibility, print space before and after the build.
|
|
- name: Disk space before build
|
|
if: steps.filter.outputs.ignore == 'false'
|
|
run: df -h
|
|
|
|
# Build and run all targets on branch pushes to ensure we always have a
|
|
# clean tree. We don't expect this to be an interactive path and so don't
|
|
# optimize the latency of this step.
|
|
- name: Test (${{ matrix.build_mode }})
|
|
if:
|
|
steps.filter.outputs.ignore == 'false' && github.event_name == 'push'
|
|
env:
|
|
# 'libtool_check_unique failed to generate' workaround.
|
|
# https://github.com/bazelbuild/bazel/issues/14113#issuecomment-999794586
|
|
BAZEL_USE_CPP_ONLY_TOOLCHAIN: 1
|
|
run: |
|
|
bazelisk test -c ${{ matrix.build_mode }} //...
|
|
|
|
# Compute the set of possible rules impacted by this change using
|
|
# Bazel-based diffing. This lets PRs and the merge queue have a much more
|
|
# efficient test CI action by avoiding even enumerating (and downloading)
|
|
# all of the unaffected Bazel targets.
|
|
- name: Compute impacted pull request targets
|
|
if:
|
|
steps.filter.outputs.ignore == 'false' && github.event_name != 'push'
|
|
env:
|
|
# Compute the base SHA from the different event structures.
|
|
GIT_BASE_SHA:
|
|
${{ github.event_name == 'pull_request_target' &&
|
|
github.event.pull_request.base.sha ||
|
|
github.event.merge_group.base_sha }}
|
|
TARGETS_FILE: ${{ runner.temp }}/targets
|
|
run: |
|
|
# First fetch the relevant base into the git repository.
|
|
git fetch --depth=1 origin $GIT_BASE_SHA
|
|
|
|
# Then use `target-determinator` as wrapped by our script.
|
|
./scripts/target_determinator.py $GIT_BASE_SHA >$TARGETS_FILE
|
|
|
|
# Build and run just the tests impacted by the PR or merge group.
|
|
- name: Test (${{ matrix.build_mode }})
|
|
if:
|
|
steps.filter.outputs.ignore == 'false' && github.event_name != 'push'
|
|
env:
|
|
# 'libtool_check_unique failed to generate' workaround.
|
|
# https://github.com/bazelbuild/bazel/issues/14113#issuecomment-999794586
|
|
BAZEL_USE_CPP_ONLY_TOOLCHAIN: 1
|
|
TARGETS_FILE: ${{ runner.temp }}/targets
|
|
run: |
|
|
# Bazel requires a test target to run the test command. There may be
|
|
# no targets or there may only be non-test targets that we want to
|
|
# build, so simply inject an explicit no-op test target.
|
|
echo "//scripts:no_op_test" >> $TARGETS_FILE
|
|
|
|
bazelisk test -c ${{ matrix.build_mode }} \
|
|
--target_pattern_file=$TARGETS_FILE
|
|
|
|
# See "Disk space before build".
|
|
- name: Disk space after build
|
|
if: steps.filter.outputs.ignore == 'false'
|
|
run: df -h
|