The use of zlib-ng seemed to work previously, but may have only done so
because of some combination of an implementation detail of
`strip_include_prefix` in older versions of Bazel and some happenstance
of a system installed `zlib.h` being possible to find. With Bazel 7 it
started breaking on my machine which doesn't have a system `zlib.h`.
Using the "normal" zlib and the bzlmod rigging for it is simple and
seems to work.
Also adds the Linux AArch64 component of the Bazel lock file which was
necessary to test this on my Arm Linux machine.
We have `StringLiteral`s in multiple other `Carbon` sub-namespaces.
Rename to a more specific name to avoid collisions.
We should likely also rename `Carbon::IntId` -> `Carbon::IntValueId` and
`Carbon::RealId` -> `Carbon::RealValueId`, but this collision is
prioritized because it was blocking work on typed parse nodes which
introduces a `Carbon::Parse::StringLiteralId`.
Building on #3505, the toolchain and llvm rules require a little more
special-casing to get them to work well. This also moves
libprotobuffer_mutator, but that one's more minor. The migration
encounters more quirks in repo naming as seen by various queries.
This changes some of the toolchain work that was recently done for bazel
7 in #3496, dropping a bzl file I'd suggested to add, instead using
`:all` for toolchain registration. (somewhat as an improvement, somewhat
just to avoid a `load`)
Remaining in the WORKSPACE are example code repos and tree sitter rules.
Neither of these are part of the main toolchain builds, and so will
probably be lower impact if there's a good solution for them.
Some background information is at https://bazel.build/external/migration
Trying to handle the simple cases first. This adds a requirement for
bazel 7 due to differences in bzlmod handling between 6 and 7 (also
discussed on
[#infra](https://discord.com/channels/655572317891461132/707150492370862090/1184942191412510720)).
Bazel seems to be okay with a partial migration such as ths.
The python import behavior has subtly shifted, so `carbon.` is no longer
part of import paths. There's a version-incompatible change for `@@`.
bzlmod makes repos sometimes show as `name~version`.
`target-determinator` seems to be okay with `@@` after a version update.
Things not moved here are things that basically need more dep work:
- clang_register_toolchains because I need to dive into its format.
- llvm-project because we need something slightly atypical, I need to
make sure patching and the repo work carries over.
- com_google_libprotobuf_mutator is sufficiently atypical that it
doesn't have a module already, but should be one of the easier things to
fix.
- brotli/woff2: I think we should actually consider removing these. But
again, they're not trivial moves.
- treesitter due to toolchain registration, which has shifted a bit.
- rules_nodejs because treesitter depends on it in an awkward way to
migrate.
This builds out a little infrastructure for one name scope to `extend`
another. We'll need more refinement here to cover other cases, but this
should provide some foundation for that future work.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This builds on the series of changes to NodeKinds, aiming to simplify
the NodeKind implementation a little, also making it clearer that
there's a single associated token for each parse node (or, for
placeholders/invalid parses, not validated).
Note that prior to the relevant changes, there were nodes with multiple
tokens. This change is also locking in the approach of one token per
parse node, by refactoring macros to stop supporting multiple.
Note #3486 rewrites the macro behavior, and is already approved: so this
PR is only for the changed enforcement during error. Also, #3493 already
changed several things to allow any token while this PR was awaiting
review, but this still changes enforcement for `For` and `If`.
This was brought up on
[#toolchain](https://discord.com/channels/655572317891461132/655578254970716160/1182066616456970251),
and I think this any-on-error approach gets at least some support. We
could try setting it to the introducer, but it's quite possible we want
it to be something like the token which led to the parse error, rather
than a static token. That leads to a conclusion that, most typically,
we'll expect arbitrary tokens when error conditions may lead to tokens
which aren't the expected token.
A couple related, recent `CARBON_IF_ERROR` crash fixes can be found in
#3404 and #3424. Something like #3404 would've been needed regardless
because `namespace` didn't have `CARBON_IF_ERROR` before, although I
might've missed the underlying issue with declarations because only
`namespace` had a relevant test (that is, if #3404 had added
`CARBON_ANY_TOKEN_ON_ERROR`, I wouldn't have had a crash in #3462).
#3424 would've been avoided with this change because there was a
`CARBON_IF_ERROR`, and it was just too restrictive.
This leaves a single state for each in the expr loop. I was trying to
think through ways to have per-token states, but they felt sort of
bulky.
Note this is more verbose: but I think the long-term is going to be that
when we start wanting to add handlers, we're going to need to switch to
different names based on the token found. As a consequence, the parse
state logic will end up diverging a little, and we'll just want to align
towards boilerplate handlers.
Short-term, this opens up a path for saying that each parse node
corresponds to precisely one token in success states, and separates out
what were becoming big handler functions in check.
Another incremental change to diagnostic formatting. I simply recurse
over all the tokens in the subtree of a parse node and construct a
`DiagnosticLocation` that covers all of the tokens.
I believe it's nicer for the user to be directed at the entire chunk of
source where the error is occurring rather then just pointing at the
bracketing/terminator tokens, but let me know if you all agree.
StructFieldUnknown was used previously for invalid parses. But we have
added other, more common ways of talking about those; so this is
removing the special-case.
Adds a `LazyImportRef` instruction. Versus `CrossRef`, this is intended
to represent an instruction which cannot be used directly, and must be
replaced when it comes up due to name lookup. The intent is to use this
to avoid recursive loading of imported IR instructions.
Note, under this model, when `ResolveIfLazyImportRef` is called, it
essentially needs to load both inst and type information to a sufficient
point where any further attempts would hit name lookup again. That will
probably be complex, and the current implementation is just touching the
surface of the issue. I was heading down this route because it would
mean we have a limited number of points that need to consider whether
they're going to talk about a `LazyImportRef`.
I'm considering whether `CrossRef` should be dropped in favor of more
specific `Builtin` special-casing, due to the divergence of desired
behaviors. This could mean dropping the `builtins` IR since it's not
looking useful right now.
Modify `NameScope` to track whether the scope is associated with a load
error. This is to handle cases where one or more imports failed, so we
do not want to issue warnings for related scopes.
The 0-size on `ValueStore` comes up due to the changes to `NameScope`,
which make it too large for the default handling. After discussion with
zygoloid, the thought was we might want to try reserving a roughly
correct value based on parse node counts, but the stack default wasn't
buying much.
Fixes a bug where the implicit import used the package name instead of
the invalid identifier.
Both of these flag flips require work to address. Their default changed
in bazel 7. Note adding these probably breaks versions of bazel older
than 6 -- but an update requirement seems okay in that case, since 6 is
about a year old now.
This is enough to support calling methods that take a `Base` or `Base*`
as their `self`. But name lookup doesn't look in the base class yet, so
base class methods aren't actually found.
This remains structurally valid although we don't use this operator or
have a design for it, this at least fixes a fuzzer-found crash.
---------
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
This is also doing the parse node split, allowing lower reliance in
formatter on the tokenized buffer (something that I may be touching more
due to import handling).
Since we switched to using the scope stack instead of the decl state
stack to hold information about the containing definition in #3460 , we
can now pop the decl state at the end of the declaration instead of the
end of the body of the definition.
(Split out of #3410)
Consistently use `ParenExpr` solely for parenthesized single
expressions, and use more syntax-oriented terminology for states and
nodes that might represent either a `ParenExpr` or a tuple literal.
Building on #3463. The PushState+PopState to construct a state feels
worth cleanup. The rest is just kind of making it easier to do without
adding another PushState overload.
Building on #3462, trying to make the flow easier to see, also making a
little more use of macros for boilerplate modifier handling.
---------
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
BinaryOperatorAdd had been added early on as a proof-of-concept for
toolchain design for parsing -> checking flow. However, it doesn't
reflect the interface direction for operators, and now other portions of
the toolchain demonstrate the relevant logic. Instead, it's just a bit
of an outlier versus other instructions which have evolved over time.
Choosing to make the constructor explicit in the test, rather than
NOLINT, because it seems to better reflect how our code is usually
written (and may be more likely to trip an issue).
Per their documentation, the `llvm::Initialize*` functions are only
supposed to be called by the main program, not by a library like
toolchain/codegen. Fixes a hang due to a data race in multithreaded
autoupdate.
Add a utility class `Carbon::InitLLVM` to do the common LLVM
initialization shared by all Carbon tools, optionally including
initializing the LLVM targets. Because the LLVM targets add a lot of
binary size, only initialize them for binaries that opt in by depending
on a new target `//common:all_llvm_targets`.
Also fix `//explorer:file_test` and `//explorer:file_test.trace` to
share a binary rather than linking an identical binary twice.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Chandler Carruth
josh11bandJon Ross-Perkins
Richard Smith
Geoff Romer
Christopher Di Bella
Uriel García
Jacob Schneider