Trying to improve robustness against failures such as
[here](https://github.com/carbon-language/carbon-lang/actions/runs/11845529962/job/33011203628):
```
WARNING: Download from https://ftp.gnu.org/gnu/m4/m4-1.4.18.tar.xz failed: class java.net.ConnectException Connection refused
```
Which is coming from the target-determinator invocation:
```
subprocess.CalledProcessError: Command '['/home/runner/.cache/carbon-lang-scripts/target-determinator', '--bazel=/usr/local/bin/bazelisk', 'a720921dfb99c21f08832663c91a1f9e48a1bcc4']' returned non-zero exit status 1.
```
My thought is that the `bazel cquery` should trigger equivalent
downloads (equally though, I'd thought the `bazel mod deps` would do
that, so I don't want to give the impression of confidence).
It's harder to inject this into the target-determinator command line,
since we'd need something that retries by default.
Came up due to [libc++ install
issues](https://discord.com/channels/655572317891461132/655577725347561492/1302023663155023905)
We've discussed clang version verification, and adding that as long as
I'm in here. The more significant bit is the libc++ check, which if it's
not installed should fail like:
```
(tons of output)
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/lib/llvm-16/lib/clang/16/include
/usr/local/include
/usr/include/x86_64-linux-gnu
/usr/include
End of search list.
/usr/local/google/home/jperkins/.cache/bazel/_bazel_jperkins/85deb7d9d96f7e0e80b42618a55969d7/external/_main~clang_toolchain_extension~bazel_cc_toolchain/_temp:6:2: error: "No libc++ install found!"
#error "No libc++ install found!"
^
1 error generated.
ERROR: Analysis of target '//toolchain:toolchain' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.265s, Critical Path: 0.08s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
```
pre-commit runs bazel, and GitHub runners have an old clang by default
(caught by the new check), so I'm installing here for a consistent
version.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Removes the separate language server binary; I'm not sure we need to
provide it. Instead, `carbon language-server` is added as a subcommand.
Moves //language_server to //toolchain/language_server. Splits into a
trivial language_server.h, and a substantive server.h. I wasn't sure
about a better name, but wanted the split similar to check/check.h,
lex/lex.h, etc. At the same time, the class is probably going to be a
little big so not a good fit to through into just a cpp file.
This fixes some style issues with the language server class, but
generally I'm trying to not address things here in order to keep it
simpler.
I expect the change here will try rerunning a couple times, then fail if
it's a permanent failure.
I've seen similar failures, and believe this is flaky. Here's the
specific example that caused me to try a workaround:
https://github.com/carbon-language/carbon-lang/actions/runs/11560505812/job/32177497296
I don't see a difference in the runner information between failing and
passing runs, which might've indicated a canary. That's why I'm doing
this as a trivial retry.
Because clang-tidy is slow (and I'm not sure we can make it really
fast), trying to run it slightly less. Also, I noticed we can shave a
few minutes by disabling apt removal without losing too much free space.
Note that since this removes the old clang-tidy, I'll need to change the
branch protections before merging.
This should free up quite a bit of space and even make our builds faster
by avoiding duplicating every output. The syntax for this flag is
counter intuitive, so I've left a comment explaining it.
Demo site: https://jonmeow.carbon-lang.dev/
I'm trying to keep work under the `/website` subdirectory so that the
misc files don't interfere with unrelated views of the repository. The
`prebuild.py` script does some work to move things around and add
frontmatter, helping the jekyll generation.
I'm using the "just-the-docs" theme because I think it's a decent match
for what we want, and getting jekyll up and running with it wasn't too
difficult. Note #1526 proposed using Docusaurus; I started out there,
but was having trouble getting it working with newer versions. The
plugins in particular I got stuck trying to make work, which sent me
looking for options that we could have working with less customization.
I do lean towards jekyll though, because it's what GH uses so hopefully
we can get a more consistent experience.
Having a website has been approved for a while under #1492, but hasn't
been a priority. I'm mainly doing this because I want to just be able to
point people to carbon-lang.dev and have easy links that way.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Also a small pass on workflow names.
Note, I'm a little concerned that the test/nightly release/pre-commit
endpoints may be fragile. At the same time, it's also where it may be
most useful, to prevent network access by arbitrary test code. I think
this is imperfect, but maybe we can try it out and see if it's much of
an issue.
Note, the discord wiki action is currently broken, this should fix it.
This configures the nightly build to generate release notes and provides
a template for organizing them. The organization is done through
labeling of PRs and categorizing them based on those labels. I've tried
to provide a rough categorization that seems reasonable for folks.
While doing this I looked at our PR labeling and found a few bugs that
were preventing many labels form being attached. I've fixed those and
significantly expanded the coverage of file-based labeling. I've also
added code to do author-based labeling for automated PRs so those can be
separated out from human PRs.
There was a file that got into the testing filegroups but not the actual
built tarball. The result was that the installation failed to locate
itself correctly.
We also didn't have any testing to catch this. I'd like to add a bit
more end-to-end testing long-term, but for now just add a Python test
that validates the same set of files are in both.
Sadly, building and testing the compressed release is really slow and
not likely worthwhile outside of the actual release, but it would be
good to catch this stuff earlier. I tried switching from `bz2` to `gz`,
which made very little file size difference (so we should do it
anyways), and it is still too slow to do routinely. Instead, I've added
a Starlark macro that builds both the `pkg_tar` and the `py_test` to
validate it for both uncompressed `tar` and compressed `tar.gz`. This
lets us continuously test the `tar` version, and just test the `tar.gz`
in the nightly release run.
Updates the nightly release workflow to both test, run this specific
test, and use `gz`.
Last but not least, removes the `pkg_zip` as I don't think we have a use
case for this yet and some TODOs that should have been removed when the
version got added.
There is still a lot more we should try to do here, but this gets us
a decent start and should be enough for some of the developers working
on Carbon to use for simple cases, or tools like Compiler Explorer.
This first factors most of the complex setup for our CI workflow into
local actions that we can reuse in other workflows. It's actually
a fairly nice cleanup I think even without the specific use case of
a nightly release process. These use "composite" actions which is the
cleanest model for factoring out a subsequence of steps from a workflow.
I've tried to keep the inputs fairly minimal and easy to spot.
Next, it adds a cron-scheduled action that builds and creates a nightly
release. It also supports explicitly running the workflow, which can be
useful if the nightly release is broken in some way: a fix can be landed
and then a force run used.
Tested the `tests` workflow using our `action-test` branch:
https://github.com/carbon-language/carbon-lang/actions/runs/9545546278
Also tested the nightly release action by hacking in a `push` trigger
and using *draft* release. If you have admin rights, you can see an
example draft nightly build here:
https://github.com/carbon-language/carbon-lang/releases/tag/untagged-6a85cdce7b3a784a2e6b
Note that the GitHub releases are easily deleted, and even the tags
created in the repository can be cleaned up as needed so it should be
fairly harmless to fix this forward if needed.
This adds a defined Carbon version to the Bazel build and codebase that
can be used both to implement features like version checks and to report
a meaningful version on the command line. This replaces a hard-coded
string and a TODO in the driver.
As part of this, it adds support for defining the version in Bazel, and
special build flags for overriding relevant parts such as the
pre-release marker used. The exact structure and meaning of our version
string, including the pre-release parts, is implemented here in line
with the draft proposal:
https://docs.google.com/document/d/11S5VAPe5Pm_BZPlajWrqDDVr9qc7-7tS2VshqO0wWkk/edit?resourcekey=0-2YFC9Uvl4puuDnWlr2MmYw
This also introduces a workspace status command to the repository to
extract the git commit SHA and other information when building, and the
logic to stamp that into binaries as part of the version string when
useful. The technique used leverages weak symbols with whole archive
linking to allow a link-time override of unstamped data with stamped
data in the leaf executable. This makes building with `--stamp` a
reasonable default, especially for development builds. The CI system is
explicitly opted out of this as there it has no benefit.
Last but not least, all of these are wired into the install rules so
that we build installable packages with the version number in a
conventional place in the directory and filename.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This is coming out of advice from
https://securityscorecards.dev/viewer/?uri=github.com/carbon-language/carbon-lang
Updates action versions and pins by checksum. This is something we
already do in other places, and doing it with actions just seems
consistent.
Trying to add token permissions at the same time.
step-security/harden-runner is being set up to monitor network traffic
for actions, with the idea that we'd set it to block unexpected traffic
in the future.
Note, https://app.stepsecurity.io/secureworkflow generates the pinned
checksums and adds the harden-runner. In some cases it added token
permissions, but we have a couple it doesn't recognize (`gh` executions,
other custom commands, jlumbroso/free-disk-space, reviewdog/action-setup
as examples) so I'm trying to guess my best.
Because these are workflows, testing is limited.
Due to my workflow, I prefer to run `pre-commit` locally when I forget
to after an amend. While reviewdog can be helpful in the UI, it's mostly
making me mark comments as resolved as part of re-pushing. Rather than
continuing with this, maybe opting out is best?
This can't be done directly from a `pull_request` action, because that's
run without write privileges. This is done for security reasons, because
it runs in the context of the pull request branch. So instead, we
perform this in two steps:
- The `pull_request` action runs `pre-commit` and uploads an artifact
containing the diffs and the event information (which is only used to
extract the pull request number).
- A separate `workflow_run` action is triggered when the `pre-commit`
action finishes. This action is privileged, and should be able to
download the artifact and create corresponding suggestions.
Unfortunately, due to the permissions model in play here, the second
half of this appears to only be testable live in production.
For now, we're splitting the pre-commit action into two actions -- one
to run on PRs and one to run when actually merging commits -- so that
the suggestions are only triggered in the former case. It might be
possible to recombine these using data in the `workflow_run` invocation
to tell them apart, but the documentation here isn't very good so I've
made this PR dump out that event information so that we can look at it
and see if it contains the relevant information.
This should also unblock our switch to C++20 and other improvements.
There are three core parts of the change --
1) Updating our infrastructure to fetch and find Clang-16.
2) Updating our documentation to reflect this and help folks with any
system issues they encounter.
The infrastructure change is unfortunately tricky. We can't get Clang 16
easily on GitHub's runner images, and in the past we've had persistent
problems with flakiness when our actions download this much during their
runs. Due to the flakiness, we've previously removed all downloading of
dependencies outside of Bazel itself, and added retry loops around Bazel
specifically to overcome flaky downloads.
This change tries to address these problems by populating the Clang and
LLVM toolchain in a place that we can then cache using the built-in
GitHub action caching infrastructure. This seems like by far the least
likely to flake way of downloading extra things into our runs. And since
these are relatively slow moving dependencies, we should populate this
cache very, very rarely.
For Linux, this downloads the binary release artifact from GitHub,
prunes out large parts of it that we don't need, and then caches this as
a local toolchain. This proves both small and fast.
For macOS, this uses a trick to cache the destination of Homebrew
installs. It unfortunately caches the *entire* Homebrew installation
though, and so it also goes to some lengths to prune and minimize how
much is installed from Homebrew. The result is "only" a 2gb cache image.
Because of the size and slower download and filesystem, the macOS runs
see a 1 - 2 minute slowdown.
We might extend the Linux infrastructure here usefully if we want to
test multiple LLVM versions. We might also extend the macOS version to
get a cheaper way to prune parts of the system and free up disk space,
or to cache other Homebrew installed tools if needed.
Last but not least, this brought to the forefront an issue with our C++
toolchain integration which relied on a specific CMake build option
being set in the LLVM toolchain install. This option isn't used in the
official release artifacts. Instead, switch to a more robust approach to
linking libc++abi statically that shouldn't have these problems.
Beyond the infrastructure changes, this also updates the documentation
to reflect requiring Clang 16 or newer, and adds some extra tips for
folks that are missing this.
The documentation is also updated to address a problem with getting the
right libc++abi files installed to support the more robust linking
strategy. This may reduce the problems we've seen in the past around
libc++abi and linking on other Linux distros as well.
---------
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
This captures the exit code of Bazel and checks for success or permanent
errors on each attempt. It also sleeps a small amount between attempts.
We should be able to increase the retries and sleeps as needed to
minimize flakiness here, and Bazel should even persist incremental
progress efficiently. Hopefully this helps reduce the failure rate of
our CI.
It also changes how we build on a `push` to use a single Bazel clause to
hold this logic.
Managed to get one of the download failures when testing this, and the
retry logic worked but there was a bug in the success logic.
Otherwise seems to work:
- Synthetic failure:
https://github.com/carbon-language/carbon-lang/actions/runs/6872431707/job/18690894069
- Success:
https://github.com/carbon-language/carbon-lang/actions/runs/6872461061
---------
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
We're seeing issues building on the 20230911.1.0 release. I think there
may be a misconfiguration; using `/usr/lib/llvm-15/bin/clang++`, I'm
getting errors such as:
```
external/com_google_absl/absl/base/config.h:56:10: fatal error: 'cstddef' file not found
#include <cstddef>
^~~~~~~~~
```
e.g.:
https://github.com/carbon-language/carbon-lang/actions/runs/6166275499/job/16735504697
(the passing ubuntu run is on a 20230903.1.0 image using llvm-14)
(note if this PR fails testing, it needs to be merged before it would
take effect)
Reported on https://github.com/actions/runner-images/issues/8253
The expensive local actions will be separately gated to not overwhelm
the machine, and currently highly asynchronous actions are a dominant
part of our builds due to downloading cached artifacts. Without a high
concurrency, these are downloaded roughly 2-at-a-time currently.
I've verified that on a small machine without a good cache this doesn't
seem to generate huge amounts of work and local build and test actions
are successfully gated on the local flags.
There is already a Bazel issue tracking this limitation:
https://github.com/bazelbuild/bazel/issues/6394
At present, it looks like tools consume 3GB and fastbuild consumes 20GB
versus 23GB available. We've apparently been on the edge of this, and
are just now getting pushed over.
https://github.com/carbon-language/carbon-lang/actions/runs/5625169220/job/15243450729
shows what this looks like (review actions are run from trunk versions,
not this branch).
Trying to fix merge queue support for PRs that don't need tests, #2971
didn't work because it lacks the matrix configuration. This takes a
different approach from that, instead filtering each step based on
affected paths. This way we're also less likely to see skew in behavior.
Dana Jansens
Jon Ross-Perkins
Richard Smith
Adrien Leravat