- In pre-commit-config, add a run of `--update_list`. I'm doing this because I noticed (on running tests.py) that we'd lost track of a few tests.
- In tests.py, the paths affect bison output where the path to the file is mentioned. This shift should make `--update_goldens` more reliable.
What this does:
- Sets up a `migrate_cpp` tool which currently only runs `clang-tidy`.
- This is intended to have more transformations in the future.
- Sets up a `migrate_cpp.sh` script.
- This copies the original woff2 code into a `carbon` directory and runs the `migrate_cpp` tool on it there.
- Adds the initial `carbon` directory of woff2
- To be clear, this is currently only updated via `clang-tidy`.
- More transformations should be expected in the future.
- Minor related edits. For example:
- Adjust pre-commit to skip the `carbon` directory, because it's third-party code and shouldn't be edited in the same way.
- Adds `@brotli_carbon` as a local repository so that we can "build" outputs.
- Makes clang-tidy from the bootstrap toolchain accessible for BUILD dependencies, as it's then used for `migrate_cpp`.
What this does not do:
- Any actual transformation of C++ code to Carbon
Co-authored-by: Matthew Riley <mdriley@gmail.com>
This restructures the `compile_flags.txt` to use the downloaded libc++ system
headers and avoid needing a virtual include directory to be built. It still
needs _some_ Bazel build to complete before working in order to have the libc++
system headers downloaded and the symlink to the Bazel tree created.
One (very) tricky part of making this work is to work around bugs in Clang's
tooling layer that incorrectly handle `..` path components after traversing
symlinks. To avoid this, we add a custom symlinks (`bazel-execroot` and
`bazel-clang-toolchain`) that hide the relevant traversal of the Bazel layout to
find build artifacts and the downloaded toolchain. These symlinks will be broken
until a build with Bazel downloads the toolchain and creates the basic output
tree structure.
It also adds a `create_compdb.py` script. Running this script improves the
tooling fidelity by taking a few steps:
1. It queries Bazel to find all the relevant files and adds them to a
`compile_commands.json` database that allows `clangd` and other tools to
index the entire project for improved cross-references, etc.
2. It builds all the generated files with Bazel so that they can be included
successfully. This is very fast in my testing, taking only 10s of seconds. It
is also very likely to be cached effectively.
3. It translates the arguments from `compile_flags.txt` to make them
persistently use the built generated files include paths so that nothing
breaks even as different targets are built potentially with different
configurations.
There are still some limitations.
- It still requires running Bazel before anything works, even if a fast run.
- It will require re-running if new generated files are added and needed but not
built.
- It assumes that the standard Bazel symlink names are used and available.
Much of the Python here was written by @geoffromer in #384 -- I've adapted it
here after discussing to try to fill in some of the blanks and use a slightly
different approach to querying Bazel. I use the normal `bazel query` rather than
`bazel aquery`. This, for example, allows the index to reliably cover header
files in header-only libraries more directly (rather than relying on transitive
inclusion). It also seems a bit simpler too parse, but that is a pretty minor
difference.
Co-authored-by: Geoffrey Romer <gromer@google.com>
Notes versus what jsiek wrote:
- This adopts Bazel for building.
- System-local versions of bison/flex are used. I found https://github.com/jmillikin/rules_bison, but those print a lot of warnings (things like -Wsign-compare IIRC) which makes builds hard to read. Plus I think the underlying bison_cc_library rule didn't work, so this would really only get a hermetic bison/flex build (helpful, but didn't seem worth more time).
- I'm adding in a .bazeliskrc to push a somewhat more standard choice of bazel versions. I noticed I was getting unstable versions by default, possible Google-specific, but seemed good to include.
- The `-lpthread` kludge.
- Turn all of the examples into golden tests.
- Including adding a golden test rule.
- Fixed various style guide issues. For example:
- Fixing function names to be CamelCase instead of snake_case (https://google.github.io/styleguide/cppguide.html#Function_Names)
- Removed exception use (https://google.github.io/styleguide/cppguide.html#Exceptions)
- File name fixes (https://github.com/carbon-language/carbon-lang/blob/trunk/docs/project/cpp_style_guide.md#file-names)
- Dropped `using` of `std` names -- I believe this is preferred (maybe we should be explicit about this in the Carbon style guide)
- Switched `enum` uses to `enum class` for ease-of-identification.
- Spent some time breaking out files to hopefully be easier to read/edit pieces, and understand relations between structs.
- Added `code requires` to `syntax.ypp` to address include issues
Possibly other things -- but the fundamental structure is, I believe, unchanged. I put in the golden tests pretty early to ensure I wasn't mutating output/results.
Revs are from `pre-commit autoupdate --freeze` (jlebar isn't providing versions, thus the sha, but this does silence a warning from pre-commit).
The migration to the new prettier location is manual, but noted at https://github.com/prettier/pre-commit
Manually validated with pre-commit run --all-files
The Bazel bits are collected into a directory and given less confusing
names (I hope). Other than names, everything is a direct copy from the
toolchain repository without any edits.
A few files needed to be merged in:
- `.gitignore`
- `.pre-commit-config.yaml`
Subsequent commits will add relevant C++ infrastructure and then the
source code itself.
One implication here is that both proposals/README.md and the website should use basically the same format for their proposal lists. Another subtle change is that the website sidebar was previously sorted by *filename*, and is now sorted by *title*, which seems better because it's something that readers can see.
Enumerating why files change:
- gen_sidebar.py is the centerpiece, with a couple helpful functions for re-use elsewhere
- Delete the old sidebar html includes
- Modify the Makefile to handle gen_sidebar.py reasonably well (let's be honest... I'm not great at Makefiles)
- Move proposal listing out to its own file (proposals.py) for re-use
- Add a few PYTHONPATH things + `__init__.py` files to get modules importing correctly (may be a better way at this, I've hit a wall though).
Adopts new copyright and markdown toc checks.
The new toc check generates the toc header, so that's why all the md files changed (this had felt better to me for the long-term, more auto-generated content)
Caught by black stable change, done by `pre-commit autoupdate`
`pre-commit autoupdate` also wanted to update pre-commit-hooks, but that gave me some pyenv errors so I'm leaving it alone for now.
This had been causing me issues when writing a test for pr-comments.py, which I then renamed so that I could import. It's had me mulling that I should really write tests for other scripts, but the name gets in the way.
The decision files are just an exception, so trying to reach consistency.
python3.6 causes weird venv issues for at least josh11b -- not sure why.
```
$ ls -l /usr/bin/python3.6
-rwxr-xr-x 2 root root 4538776 Jan 2 2019 /usr/bin/python3.6*
$ virtualenv -p python3.6 venv
RuntimeError: failed to find interpreter for Builtin discover of python_spec='python3.6'
```
This catches some common spelling errors, but is *not* an exhaustive spell-checker. The only false positive is "rouge" in Gemfile.lock, which is why I'm excluding that file.
True positives are in the change, and it does catch the "langauge" typo I keep making, which zygoloid fixed for me in a few spots. False negatives are:
- virmc
- Announcemented
- propsal
For comparison:
- cspell (https://www.npmjs.com/package/cspell) is too aggressive and has a small dictionary
- Caught the false negatives, but many false positives, including 'LLVM', 'roadmap', 'Carruth'
- https://github.com/lorenzwalthert/precommit has a spellcheck, but config looks ruby-specific
Added here:
- .pre-commit-config.yaml
- docs/project/contribution_tools.md
- src/scripts/pre-commit-toc.js
- CONTRIBUTING.md
- proposals/* (just to switch TOC formats)
The rest consists of auto-generated fixes.
The switch from doctoc to markdown-toc is because doctoc wants to put a TOC in every file, whereas markdown-toc only does it if you have an insertion point.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Jon Meow
Richard Smith
Chandler CarruthandGeoffrey Romer
josh11b