The expensive local actions will be separately gated to not overwhelm
the machine, and currently highly asynchronous actions are a dominant
part of our builds due to downloading cached artifacts. Without a high
concurrency, these are downloaded roughly 2-at-a-time currently.
I've verified that on a small machine without a good cache this doesn't
seem to generate huge amounts of work and local build and test actions
are successfully gated on the local flags.
There is already a Bazel issue tracking this limitation:
https://github.com/bazelbuild/bazel/issues/6394
At present, it looks like tools consume 3GB and fastbuild consumes 20GB
versus 23GB available. We've apparently been on the edge of this, and
are just now getting pushed over.
https://github.com/carbon-language/carbon-lang/actions/runs/5625169220/job/15243450729
shows what this looks like (review actions are run from trunk versions,
not this branch).
Trying to fix merge queue support for PRs that don't need tests, #2971
didn't work because it lacks the matrix configuration. This takes a
different approach from that, instead filtering each step based on
affected paths. This way we're also less likely to see skew in behavior.
@chandlerc requested that leads questions not be marked inactive. These arise when there *isn't* a clear-cut answer, and so seem to make more sense to be effectively long-term.
The \n\n\n is trying to get a blank line between paragraphs. Right now it gets a newline, but the paragraphs sort of blend. I might need more \n's, don't know but I'll keep an eye on messages. The examples that I could find don't have multiple paragraphs.
While we have a generic 'long term' label, it seems redundant for some
issues that are already labeled with something that clearly is
open-ended and not something we should expect to have a bounded
timeline. For example, we want to actively curate a backlog of design
ideas and good first issues for folks to browse and pick up, so we
shouldn't be marking them as inactive after any fixed time frame.
We keep seeing fragility installing software, with both brew (e.g., the recent python issues) and apt.llvm.org (currently flaky).
At this point, images for both ubuntu and macos have versions of llvm-14 that seem to successfully compile:
https://github.com/carbon-language/carbon-lang/actions/runs/3474670746/jobs/5808098087
Although we may want to figure out a way to resume running llvm-15 so that we can see compatibility issues, this seems preferable for baseline testing in order to reduce maintenance churn.
In addition to the above changes, this also configures cancellation more precisely, and stops installing bazel/bazelisk (it should already be preinstalled).
We've been having issues with asan builds on linux. This change should fix all of that. A build run can be found at:
https://github.com/carbon-language/carbon-lang/actions/runs/3093378863/jobs/5005683059
(currently in progress, but I'm expecting it to succeed at this point)
It may be that the issues with asan builds were actually related to caching. That is, maybe the brew build command didn't change enough between v14 and v15 that the cache hits were still an issue. We did notice this with 15.0.0 versus 15.0.1 include paths (that is, bazel wasn't happy using the cached results of a 15.0.0 build due to the skew in include paths). In order to address this, I've added CACHE_VERSION to the remote_cache setup. I've also set up corresponding buckets in Cloud.
However, I'm also switching Linux to llvm-15 and apt. I'd originally been looking at this because the issues were linux-specific, and we've previously had linux-specific issues with Homebrew. Although it may have been the cache all along, I would prefer to keep this setup (if nothing else, it made the caching issues more obvious, even though we were still confused by the include path manifestation).
This removes the separate data files, which have been failing to load properly ([example](https://github.com/carbon-language/carbon-lang/runs/7957047892?check_suite_focus=true)). This one wasn't working previously, but hopefully will with these changes (and it might've also just been a pull_request_target issue, but the silent "Resource not accessible by integration" failures aren't great).
Wiki can either be "require push access" or "everyone" -- apparently there's no other option. I've set it to "everyone" so that contributors can make edits without needing push access. So the options as I see it are:
- Leave wiki as "everyone" can edit, use this for notifications.
- GitHub doesn't give notifications for wiki edits. This is trying a different approach for notifications.
- Grant push access to a larger group (contributors), don't add CODEOWNERS.
- Not sure this is the right choice because of the implications around merges, but again maybe it'd be fine and we can expect the approval requirement to work out.
- Grant push access to contributors, add CODEOWNERS.
- This causes the auto-assignment to CODEOWNERS that we don't want. (details in https://github.com/carbon-language/carbon-lang/pull/1367)
- Create a separate wiki repo so that we can differently handle push access.
- This seems overly complex a solution though.
I'm hoping this approach works.
This is being done because Projects v1 requires repo write access, a serious limitation for letting people use it. Projects v2 isn't a great option because it lacks event support. Labels are pretty stable in GitHub, so this switches to that.
Note this assumes we're fine renaming "decision: accepted" -> "proposal accepted", etc. There are two reasons for this:
1) To make it clear that this is a proposal-specific label, versus something like an issue for leads label.
2) Removing the colon because it was causing trouble with yaml syntax.
This also adds the "proposal draft" label, mainly to complete the taxonomy.
I was considering whether this should be a proposal itself, but it feels like maybe it's not necessary because it's a fairly low-key infrastructure change, and I'm not sure how much people were relying on the project board anyways.
I tested this in a personal repo, basically just poking at https://github.com/jonmeow/test/pull/2
This will allow us to automatically create and maintain specific repos
based on the main repository here, pulling key files like the license
and other infrastructure and pushing them systematically to a narrow
repo. Things like editor plugins that are best packaged and installed
from a separate repos can still be developed in a central place, even
potentially sharing common things like grammars where useful.
Currently this will maintain a Vim plugin repository out of the
`utils/vim` directory, but can be easily expanded for other systems.
I accidentally put in one I'd considered, instead of the one I'd decided was probably the best fit, and the names are so similar I didn't notice.
NOTE: This still isn't totally working, but I think it will when we go public.
This was noticed because macos stopped installing the brew llvm, and they don't have clang-format. However, our tests don't actually need clang-format (and even if they did, we should probably match the pre-commit version), so it seems superfluous to check for.
Keeping brew because there's an advantage to consistency on tool versions, just running it on all platforms instead of linux-only.
Moves common script logic into utils.py (not a great name, but couldn't come up with better). This is in particular to make the buildifier.py script really trivial, allowing that pre-commit to be easily added. However, scripts have also been diverging on how we find bazel, so I'm trying to unify that.
The advantage of reimplementing buildifier's pre-commit is that (a) we can now run buildifier server-side, and (b) we can stop advising installing it manually. Then the only Linux-specific package manager is Cargo, which is only used for watchman, which is optional -- so stop highlighting Linux-specific package managers in the tool instructions.
- brew instead of pyenv to centralize package management
- can't recall why we used pyenv before, may not have been the best choice.
- pyenv has been a burden in updating versions, a single `brew upgrade` works better
- python3.9 because it's the latest and greatest, feels weird going to old versions if we're recommending installs.
- bazel 5.0.0 pre-release due to https://github.com/bazelbuild/bazel/commit/b9fc66d327debcfbdb2964afdba35a1cc8919b81
- earlier versions want `python` to be on the path, which pyenv did but brew does not
For most people this will mean: `rm -rf ~/.pyenv && brew uninstall pyenv && brew install python@3.9 && pip3 install -U pip && pip3 install pre-commit`, plus removing any `pyenv` invocations from the shell `rc` file.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
- Automatically mark issues/PRs `inactive` after 90 days without comments
- Archive PRs (only) after 14 more days
- Adding "long term" for issues which can't be resolved in a few months, but shouldn't be marked `inactive`.
The built-in GitHub action cache is too broken for us to use currently.
Pull request runs will evict trunk runs from the cache easily, which
will remove a useful baseline and make builds generally uncached and
extremely slow.
Instead, move to using Bazel's remote caching functionality with a GCP
storage bucket. This gives us nearly limitless storage and is actually
vastly simpler than the prior arrangement. It should also allow cache
sharing between different build configurations and other benefits.
This cache isn't usable by anyone else sadly, but it should at least
provide a better scaling technique for our CI.
This does require us to move from `pull_request` workflow to
a `pull_request_target` workflow and run the CI within the domain of the
project. This is only really safe for us to do while the project remains
private. When we go public we'll want to make the cache publicly
readable and change this code so that when not running on a branch in
the main repository we use the cache in a read-only mode and only write
back updates for branch runs. This should still be effective but
requires a bit more complexity here and in the remote cache including
setting up public access. For now, going with the simple if slightly
less secure model.
This was necessary when I was testing this on my own PR branch, but
without this rebuilds on trunk won't save a fresh cache for folks to use
on *their* pull commit branches. We just need a fresh key here.
Also add one more flag to our `.bazelrc` to improve the overall build
efficiency.
While the first build with this will be very slow (likely a couple of
hours), the caching layer we are using now should effectively cache this
for subsequent builds so that only the changed files have to be
recompiled. Updates to Clang and LLVM themselves will still result in
a slow rebuild, but normal development should remain fast.
The Bazel disk cache provides a stable, durable, and supported
persistent cache of the intermediate steps of Bazel builds. This is like
`ccache` but covering more steps and more reliable.
It is based on the same fundamental infrastructure that enables
a *remote* build cache, and at some point we may want to enable that so
that our CI can pre-seed a cache for users. But for now, this just
enables local support for this.
For local builds this will ensure that intermediate artifacts are
re-used when changing branches, syncing, across Bazel server restarts or
even upgrades. It should dramatically reduce the need to re-build large
and slow components like all of Clang and LLVM. Largely, those should
only be rebuilt when we update our pinned revision of upstream and then
cached persistently on a given machine. Enabling this by default
requires hard-coding a path, and so I've also added built-in support for
importing a user config file if present so that folks can customize this
(and any other Bazel features) as desired.
This also wires the disk cache up for the GitHub cache in our CI. By
using the designated Bazel disk cache for this, we get a *much* simpler
and *much* more robust solution. So much so that I've re-arranged the
key structure so we can more aggressively re-use old caches. The only
downside is that nothing ever gets removed automatically from the cache.
I've added a layer that will trim any unused file roughly once a day.
For CI, the benefits of this are huge. None of the previous hacks are
needed to locate and preserve the cache. The design of the cache is
deeply durable and so we don't need to aggressively invalidate it. Last
but not least, it focuses the cache on the raw artifacts, which compress
exceptionally well. With the other changes to shrink the build outputs
and this one combined, I expect we will be able to enable all of the
build targets across all four config/os combinations without any
significant pressure on our space quota for GitHub action caches.
Chandler Carruth
Geoff Romer
Jon Ross-Perkins
Adrien Leravat
SADIK KUZU
Jon Meow