I was considering making a "/bazel" alias for this, but I'm really on
the fence about whether that's a good choice. However, I do think this
is good to have to simplify installs.
Fixes#3071 and #3896
Adds scripts/calculate_release_shas.py to print the versions, and
updates tool versions. Consolidates target-determinator logic with the
other tool logic.
This is also intended to make it easier to add more tools.
Python [PEP-585](https://peps.python.org/pep-0585/) replaces a number of
`typing` module types with built-in equivalents and `collections.abc`
versions as of Python 3.9, with the aim of eventually removing the
`typing` module versions of these classes altogether. Since the minimum
required version of Python listed in the [Contribution Tools
document](https://github.com/carbon-language/carbon-lang/blob/trunk/docs/project/contribution_tools.md#main-tools)
is 3.9, the type hints in the various python files in the repo can be
updated to this style of type hint without a need for backwards
compatibility.
Feel free to close if this isn't a desired change at this time!
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Some background information is at https://bazel.build/external/migration
Trying to handle the simple cases first. This adds a requirement for
bazel 7 due to differences in bzlmod handling between 6 and 7 (also
discussed on
[#infra](https://discord.com/channels/655572317891461132/707150492370862090/1184942191412510720)).
Bazel seems to be okay with a partial migration such as ths.
The python import behavior has subtly shifted, so `carbon.` is no longer
part of import paths. There's a version-incompatible change for `@@`.
bzlmod makes repos sometimes show as `name~version`.
`target-determinator` seems to be okay with `@@` after a version update.
Things not moved here are things that basically need more dep work:
- clang_register_toolchains because I need to dive into its format.
- llvm-project because we need something slightly atypical, I need to
make sure patching and the repo work carries over.
- com_google_libprotobuf_mutator is sufficiently atypical that it
doesn't have a module already, but should be one of the easier things to
fix.
- brotli/woff2: I think we should actually consider removing these. But
again, they're not trivial moves.
- treesitter due to toolchain registration, which has shifted a bit.
- rules_nodejs because treesitter depends on it in an awkward way to
migrate.
There was a lot of repetition and unnecessary cruft in our Bazel
toolchain support. Switch to generating all of it with a single macro
that handles everything. This should make no real difference but
dramatically simplifies adding a new CPU.
Use this simplified system and add `aarch64` which is how Arm 64-bit CPU
support shows up on a Linux host.
Also teach the basic scripts to map `aarch64` to `arm64` which is used
in the released artifact strings.
I went down this route because I figured the installer binary should be in `check_deps`. The result:
- Add thin wrapper scripts for `buildozer` and `bazel`, similar to the `buildifier` wrapper that's already there.
- These are mainly to avoid deduplicating logic, and dodging intricacies of Python imports that I don't understand well through instead subprocessing.
- Fix update_roots.py (it was missing a `.parent`, must've been relocated without a re-run at some point)
- Fix fuzzer proto code to be `testonly`
- Update the `check_deps` targets
I'm wondering if my issue all along was parallel script executions (I reproduced the error message locally with `pre-commit run -a`, and I think in my case that's what happened).
Adding the file size to errors for sanity checking, since part of what had me looking again was checksum mismatches, and actually the parallel execution offers a consistent story there (checksum mismatch due to file being modified by another download mid-stream).
Moves common script logic into utils.py (not a great name, but couldn't come up with better). This is in particular to make the buildifier.py script really trivial, allowing that pre-commit to be easily added. However, scripts have also been diverging on how we find bazel, so I'm trying to unify that.
The advantage of reimplementing buildifier's pre-commit is that (a) we can now run buildifier server-side, and (b) we can stop advising installing it manually. Then the only Linux-specific package manager is Cargo, which is only used for watchman, which is optional -- so stop highlighting Linux-specific package managers in the tool instructions.
Jon Ross-Perkins
Jack McCluskeyandJon Ross-Perkins
Chandler Carruth
Jon Meow