Implement the toolchain side of proposal #7254, removing the `:!`
binding
syntax for generic and template parameters in favor of the keywords
`generic`,
`template`, and `runtime` plus contextual defaults for phase.
For valid programs this is semantics-preserving: each binding resolves
to the
same phase, and produces the same SemIR, as it did under `:!`/`:`. The
parser
derives a binding's phase from its syntactic context plus any explicit
phase
keyword; new diagnostics and error recovery for misused keywords are
described
below.
Implementation details for each component:
- Lexer: remove the `:!` (`ColonExclaim`) token, move its virtual
parse-node
budget onto `:`, and add the `generic` and `runtime` keywords.
- Parser: thread a `BindingContext` (`ExplicitParam`, `DeducedParam`, or
`CompileTimeEntityParam`) from declaration introducers down through
parameter
lists to each binding pattern, using a one-token lookahead to
distinguish a
name-qualifier parameter list from a declaration's own final list.
Parameters
of a compile-time entity (`class`, `interface`, `constraint`, `choice`,
`alias`, `export`, `namespace`) and deduced `[]` parameters default to
checked
generic; explicit function parameters and local bindings default to
runtime.
`HandleBindingPattern` resolves the phase from that context plus the
keyword: a
`generic` keyword needs no node of its own (the phase is carried by the
binding's node kind), while a `runtime` keyword is preserved as a
`RuntimeBindingName` node so `check` can name it in a diagnostic. A
phase
keyword that is merely redundant with the contextual default is
diagnosed
here, without invalidating the parse tree.
- Check: a phase keyword that is invalid for its context (for example
`runtime`
on a checked-generic parameter) is diagnosed here, and recovers by
building an
error binding that still introduces the name so that later uses of it do
not
produce cascading errors.
The removed `:!` syntax is now rejected as an ordinary parse error.
The `form`/`:?`/`->?` ("extended types") portion of proposal #7254 is
left for a
separate change.
Assisted-by: Claude Code
Copy MultiplexExternalSemaSource.h and MultiplexExternalSemaSource.cpp
from https://github.com/llvm/llvm-project/pull/204458 into
third_party/llvm, and apply a few minor changes to allow them to compile
and pass precommit checks.
This allows
`0011-Add-empty-constructor-and-GetSources-method-to-Multi.patch` to be
removed, which brings Carbon closer to being able to compile on an
unmodified LLVM toolchain.
Implements proposal #7016: `self` moves from the deduced implicit list
(`fn F[self: Self]()`) to the front of the explicit list. Its type may
be written explicitly (`fn F(self: Self)`) or omitted, in which case it
defaults to `Self` (`fn F(self)`, `fn F(ref self)`); `self` in the
implicit list is rejected.
Throughout checking, `self` is modeled as the first explicit parameter.
Because a method is just a function whose first parameter is `self`, it
can also be called as an ordinary function with the receiver passed
explicitly (`Type.M(obj, ...)`), not only as `obj.M(...)`. A new
`SemIR::CallArgParamPatterns` helper chooses the parameters matched
against the explicit arguments, excluding a leading `self` only when it
is supplied as a method-call receiver; arity checking, conversion, and
generic deduction use it. The resulting SemIR and lowering are
unchanged: `self` is still `call_param0`, and witnesses, thunks, and
vtables are unaffected.
An omitted `self` type is parsed as a `SelfBindingPattern` node with no
type expression; checking synthesizes the `Self` type so it behaves
exactly like `self: Self`. However, the exact spelling used must match
between a forward declaration and a definition, following #3763's rules
around declaration matching.
Generated functions, thunks, and C++ interop import/export build `self`
as the first explicit parameter, and the `self`-type override (e.g.
Derived->Base for a virtual override) applies to the explicit `self`.
Placement is validated by new diagnostics: `SelfInImplicitParamList`,
`SelfNotFirstParam`, and `SelfOutsideParamList`. The benchmark source
generator and the documentation adopt the `(self)` shorthand; the
prelude, the examples, and the test data are migrated in the following
commits.
Assisted-by: Claude Code with Claude Opus 4.7
---------
Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
This is (far) from robust -- particularly with repeated compiles in the
same address space. But it appears to be sufficient in the short term,
and we already have the relevant TODOs to factor this upstream into
something that we can use here.
This also somehow uncovered a bug in how we were logging failed commands
-- the failed commands are destroyed when we destroy the driver object
(and its diagnostics object), so we simply cannot do the logging _after_
flushing diagnostics. That's probably ok, and just doing this in the
other order makes the code simpler.
Assisted-by: Antigravity with Gemini
Roll LLVM to `6811a83c81500ee373adfc0d9978ff9625a4cf1c`.
This includes https://github.com/llvm/llvm-project/pull/183831 which
moved the functionality of `finish()` on `DiagnosticConsumer`s into the
destructors, and removed the `finish()` method. So, our callers to
`finish()` are migrated to cause the destructor to run at that time
instead.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Also consolidate on using `//bazel/cc_rules:defs.bzl` where appropriate.
Also update a couple of Bazel modules deps of `@rules_cc` to the latest
versions.
Previously this was kept in `//toolchain/install` so it would be near to
the code that actually defines the installation layout. However, that
creates somewhat unfortunate dependency cycles between
`//toolchain/install` and other directories. Exacerbating this, a
subsequent PR is likely to add dependencies on it from
`//toolchain/base` itself that suggests that is the correct layering.
This PR just moves the code mechanically with as few other edits as
possible.
This clarifies that the CC1 logic is directly extracted from Clang.
There are probably some other places in the toolchain we should extract
code like this where we're replicating and customizing logic from LLVM,
but wanted to start here.
Every test that used `addr` before #6283 should be using `ref` after
this PR. In most cases that was done in #6283, but this PR transitions a
few that I missed in that first pass. In addition, #6283 cloned the old
`addr` tests from `foo.carbon` to `foo_addr.carbon` in order to maintain
test coverage during the transition; this PR removes those cloned tests.
Bazel's WORKSPACE file is deprecated, and we haven't worked on the woff2
example in ages. Rather than investing time into keeping it around,
remove it and we can revive it when we're ready.
This uses a header-only extraction of the Boost unordered hashtable
project to allow a trivial Bazel build and for us to benchmark against
it effectively.
I'd kept these in to separate the bazel module update from the BUILD
file changes, then forgot about it. I think all of these can be cleanly
removed now. I think it's something we should clean up for consistency
with the bazel central repository names; I think it's best to reduce
that divergence.
llvm_zlib and llvm_zstd remain because of how llvm depends on the
particular names.
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
Also make minor updates to the skeletal design in
docs/design/name_lookup.md following #2113, as there are no longer any prelude names that are made available to unqualified name lookup by default.
Add `type` to the keyword list in
docs/design/lexical_conventions/words.md, following #2360.
This does some more work to the run_clang_tidy.py wrapper script, and runs an example pass.
"again" because it's really the proto fuzzer changes that broke it, it had been working before.
"mostly" because there's still an issue within the proto fuzzer that it can't find "port/protobuf.h", i.e. https://github.com/google/libprotobuf-mutator/tree/master/port, but I'm still hesitant to add an include path there.
The most significant change here is that explorer now uses the chosen spelling
rather than the old `Bool` spelling. Also update a few documentation examples
and some skeletal design docs to use the chosen spelling.
* finished fuzzer and added fuzzverter util
* fixed typo
* renamed cmd line params
* fixed libproto_mutator download path
* small fixes
* small fixes
* small fixes
* renamed sample corpus proto
* small fixes
* try building on github with LIBCPP_DEBUG enabled
* temporarily marked proto fuzzer as a manual test
* code review
* use a dedicated proto-fuzzer feature to work around LIBCPP_DEBUG=1 crash in proto code
* code review comments, added README.md
* minor fixes to the text
* Update bazel/cc_toolchains/clang_cc_toolchain_config.bzl
Co-authored-by: Jon Meow <jperkins@google.com>
* use Carbon source representation for "empty Main()" instead of text format proto representation
* fixed typo
* made FuzzerUtil produce the full carbon source (proto converted + Main if needed) to decrease code duplication a bit
* typo
* switched to text proto format per code review
* Update executable_semantics/prelude.h
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* review comments
* removed unnecessary file mode variables
* Update executable_semantics/fuzzing/fuzzverter.cpp
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* Update executable_semantics/fuzzing/README.md
Co-authored-by: Jon Meow <jperkins@google.com>
* code review comments
* Update executable_semantics/syntax/BUILD
Co-authored-by: Jon Meow <jperkins@google.com>
* buildifier
Co-authored-by: Jon Meow <jperkins@google.com>
Per https://github.com/bazelbuild/rules_cc this still isn't necessary. There's no build-time enforcement, so usage is inconsistent/incorrect. Rather than letting this linger, remove it pending Bazel tooling enforcing it.
This replaces GetAstMatcher with AddMatcher because cxxForRangeStmt is a StatementMatcher. addMatcher has multiple definitions (https://clang.llvm.org/doxygen/classclang_1_1ast__matchers_1_1MatchFinder.html) and so this approach allows using the right addMatcher without writing per-call overloads.
To handle the `var`, I'm considering something like moving VarDecl logic into a VarMatcherBase so that I can just use CXXForRangeStmt's getLoopVariable. The problem is a for-range statement has multiple VarDecls, and getLoopVariable may be the easiest way to identify the real one.
Range-based for loops remain messed up -- I'm just disabling the test here and turning a blind eye.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
TODOs in the test for known issues.
I may switch the approach to getting the variable type (on examination, this isn't working quite as well as I'd thought) but for now I think it's okay. I had an earlier approach though that may work better overall -- I'd been thinking this would work better, but as you can see in the null check for type information, I think I missed a key point.
Anyways, what'd really been vexing me was `int i, j` which I think I handle passably well now. There's obviously room for improvement, but given I've been going at this for a couple days now, I thought it best to checkpoint where I was.
This also includes some related framework changes to fix bumps I was running into. Overall the tool should operate a bit more smoothly with these changes. There are still issues with overlapping replacements, but I think it's primarily with range-based for loops which I just need to take some time to fix.
This might need some customization, e.g. `woff2_decompress.impl.carbon` (as a file with a `main`) might be better as `woff2_decompress.carbon`, but I thought at least putting on `.carbon` extensions would help perspectives on files.
This restructures the script to make it easier to access standard info (e.g. cpp_files) without passing everywhere.
This picks up a newer version of LLVM and the LLVM Bazel integration.
The big change here is that we can configure the LLVM targets that are
built, which allows us to dramatically reduce the build costs by
focusing on a couple of CPUs for the time being.
There are a few API updates needed as well.
This also rotates the cache version so we start with a clean Bazel cache
from here. Otherwise we'd potentially pay the cost of carrying around
stale bits of LLVM endlessly.
What this does:
- Sets up a `migrate_cpp` tool which currently only runs `clang-tidy`.
- This is intended to have more transformations in the future.
- Sets up a `migrate_cpp.sh` script.
- This copies the original woff2 code into a `carbon` directory and runs the `migrate_cpp` tool on it there.
- Adds the initial `carbon` directory of woff2
- To be clear, this is currently only updated via `clang-tidy`.
- More transformations should be expected in the future.
- Minor related edits. For example:
- Adjust pre-commit to skip the `carbon` directory, because it's third-party code and shouldn't be edited in the same way.
- Adds `@brotli_carbon` as a local repository so that we can "build" outputs.
- Makes clang-tidy from the bootstrap toolchain accessible for BUILD dependencies, as it's then used for `migrate_cpp`.
What this does not do:
- Any actual transformation of C++ code to Carbon
Co-authored-by: Matthew Riley <mdriley@gmail.com>
This is to support converting the code to Carbon. My theory with the setup is:
- Have the code available to build in C++ under `third_party/<project>/original`.
- Created a `third_party/<project>/carbon` for the converted version.
Having the existing code building should, I think, make it easier to run analysis on said code. Using a submodule means we should be aiming to keep it pristine, for easy comparison / updates.
The Bazel bits are collected into a directory and given less confusing
names (I hope). Other than names, everything is a direct copy from the
toolchain repository without any edits.
A few files needed to be merged in:
- `.gitignore`
- `.pre-commit-config.yaml`
Subsequent commits will add relevant C++ infrastructure and then the
source code itself.
Chandler Carruth
Nicholas Bishop
Dana JansensandChandler Carruth
Jon Ross-Perkins
Geoff Romer
Richard Smith
josh11b
Yen-Jen Wang
Josh Soref
pk19604014andJon Meow
Jon Meow
