carbon-lang/docs/project/contribution_tools.md

Contribution tools

The Carbon language project has a number of tools used to assist in preparing contributions.

Table of contents

• Tool setup flow
• Package managers
  • Linux and MacOS
    • Homebrew
    • Python using pyenv
  • Linux only
    • go get
    • Cargo (optional)
• Main tools
  • Bazel and Bazelisk
  • Bison and Flex
  • buildifier
  • Clang and LLVM
  • Ninja
  • pre-commit
  • gql
  • PyGitHub
• Optional tools
  • Carbon-maintained
    • new_proposal.py
    • pr_comments.py
  • GitHub
    • gh CLI
    • GitHub Desktop
  • rs-git-fsmonitor and Watchman
  • Vim
    • vim-prettier
  • Atom
  • pre-commit enabled tools
    • black
    • codespell
    • markdown-toc
    • Prettier

Tool setup flow

In order to set up a machine and git repository for developing on Carbon, a typical tool setup flow is:

1. Install package managers.
2. Install main tools and any desired optional tools.
3. Set up the git repository:
   • In GitHub, create a fork for development at https://github.com/carbon-language/carbon-lang.
   • gh repository clone USER/carbon-lang, or otherwise clone the fork.
   • cd carbon-lang to go into the cloned fork's directory.
   • git submodule update --init to sync submodules if you'll be building c++ code or working on the compiler.
   • git config core.fsmonitor rs-git-fsmonitor to set up rs-git-fsmonitor in the clone.
   • pre-commit install to set up pre-commit in the clone.
4. Validate your installation by invoking `bazel test //...:all' from the project root. All tests should pass.

Package managers

Instructions for installing tools can be helpful for installing tooling. These instructions will try to rely on a minimum of managers.

Linux and MacOS

Homebrew

Homebrew is a package manager, and can help install several tools that we recommend. See the installation instructions.

To get the latest version of brew packages, it will be necessary to periodically run brew upgrade.

Python using pyenv

We strongly recommend using pyenv to manage Python and Python's pip package manager. pip should typically be used for Python package installation rather than other package managers.

These can be installed together through brew:

brew install pyenv
pyenv install 3.8.5
pyenv global 3.8.5

# Add 'eval "$(pyenv init -)"' to your shell rc file, for example zshrc.
echo 'eval "$(pyenv init -)"' >> ~/.zshrc
# Load the shell rc file changes.
exec $SHELL

To get the latest version of pip packages, it will be necessary to periodically run pip list --outdated, then pip install -U <package> to upgrade desired packages. Keep in mind when upgrading that version dependencies may mean packages should be outdated, and not be upgraded.

Linux only

Linux-specific package managers are typically used for packages which work through brew on MacOS, but not on Linux.

Installation instructions assume Debian- or Ubuntu-based Linux distributions with apt available.

go get

go get is Go's package manager.

Our recommended way of installing is:

apt install golang

To get the latest version of go packages, it will be necessary to periodically re-run the original go get ... command used to install the package.

Cargo (optional)

Rust's Cargo package manager is used to install a couple tools on Linux. See the installation instructions.

To get the latest version of cargo packages, it will be necessary to periodically re-run the original cargo install ... command used.

Main tools

These tools are key for contributions, primarily focused on validating contributions.

Bazel and Bazelisk

Bazel is Carbon's standard build system. Bazelisk is recommended for installing Bazel.

Our recommended way of installing is:

brew install bazelisk

Bison and Flex

Bison and Flex are used by executable semantics. Although we may switch to a hemertic toolchain later, an install is currently required.

Our recommended way of installing is:

brew install bison flex

On MacOS, it will be necessary to explicitly add the installed paths to the PATH environment variable so that the brew-installed versions are used instead of Xcode-installed versions. Read brew output for instructions.

buildifier

Buildifier is a tool for formatting Bazel BUILD files, and is distributing separately from Bazel.

Our recommended way of installing is:

• Linux:

  go get github.com/bazelbuild/buildtools/buildifier
  

• MacOS:

  brew install buildifier
  

Clang and LLVM

Clang and LLVM are used to compile and link Carbon, and are provided through git submodules. A complete toolchain will be built and cached as part of standard bazel execution. This can be very slow on less powerful computers or laptops (30 minutes to an hour). However, it should only happen when either Bazel or LLVM's submodule is updated, which we try to minimize. If you need to force a rebuild of the toolchain, you can use bazel sync --configure.

Ninja

Ninja is used to build Clang and LLVM.

Our recommended way of installing is:

brew install ninja

pre-commit

We use pre-commit to run various checks. This will automatically run important checks, including formatting.

To set up pre-commit, see the installation instructions, or:

pip install pre-commit

# From within each carbon-language git repository:
pre-commit install

When you have changes to commit to git, a standard pre-commit workflow can look like:

# Let pre-commit fix style issues.
pre-commit run
# Add modifications made by pre-commit.
git add .
# Commit the changes
git commit

When modifying or adding pre-commit hooks, please run pre-commit run --all-files to see what changes.

gql

pip install gql

PyGitHub

pip install PyGitHub

Optional tools

Carbon-maintained

new_proposal.py

new_proposal.py is a helper for generating the PR and proposal file for a new proposal. It's documented in the proposal template.

NOTE: This requires installing the gh CLI.

pr_comments.py

pr_comments.py is a helper for scanning comments in GitHub. It's particularly intended to help find threads which need to be resolved.

Options can be seen with -h. A couple key options to be aware of are:

• --long: Prints long output, with the full comment.
• --comments-after LOGIN: Only print threads where the final comment is not from the given user. For example, use when looking for threads that you still need to respond to.
• --comments-from LOGIN: Only print threads with comments from the given user. For example, use when looking for threads that you've commented on.

GitHub

gh CLI

The gh CLI supports some GitHub queries, and is used by some scripts.

To install gh, run:

brew install github/gh/gh

GitHub Desktop

GitHub Desktop provides a UI for managing git repositories. See the page for installation instructions.

rs-git-fsmonitor and Watchman

rs-git-fsmonitor is a file system monitor that uses Watchman to speed up git on large repositories, such as carbon-lang when submodules are synced.

Our recommended way of installing is:

• Linux:

  brew install watchman
  cargo install --git https://github.com/jgavris/rs-git-fsmonitor.git
  
  # Configure the git repository to use fsmonitor.
  git config core.fsmonitor rs-git-fsmonitor
  

• MacOS:

  brew tap jgavris/rs-git-fsmonitor \
    https://github.com/jgavris/rs-git-fsmonitor.git
  brew install rs-git-fsmonitor
  
  # Configure the git repository to use fsmonitor.
  git config core.fsmonitor rs-git-fsmonitor
  

Vim

vim-prettier

vim-prettier is a vim integration for Prettier.

If you use vim-prettier, the .prettierrc.yaml should still apply as long as config_precedence is set to the default file-override. However, we may need to add additional settings where the vim-prettier default diverges from prettier, as we notice them.

Atom

Atom is an IDE that's mainly mentioned here for its Markdown support. See the page for installation instructions.

Some packages that may be helpful are:

• markdown-preview-enhanced: An improvement over the default-installed markdown-preview package. If using this:
  • Disable markdown-preview.
  • In the package settings, turn off 'Break On Single New Line' -- this is consistent with carbon-lang.dev and Prettier.
• prettier-atom: For Prettier integration.
• python-black: For black integration.

pre-commit enabled tools

If you're using pre-commit, it will run these tools. Installing and running them manually is optional, but may be helpful.

black

We use Black to format Python code. Although Prettier is used for most languages, it doesn't support Python.

codespell

We use codespell to spellcheck common errors. This won't catch every error; we're trying to balance true and false positives.

markdown-toc

We use markdown-toc to provide GitHub-compatible tables of contents for some documents.

If run manually, specify --bullets=- to use Prettier-compatible bullets, or always run Prettier after markdown-toc.

Prettier

We use Prettier for formatting. There is an rc file for configuration.