Note, I'm annotating the lookup partly so that the reason the conflict
comes up is clear, partly so that there's actually a diagnostic line
associated with the root cause as more tests get packed into a single
file.
I'd kept these in to separate the bazel module update from the BUILD
file changes, then forgot about it. I think all of these can be cleanly
removed now. I think it's something we should clean up for consistency
with the bazel central repository names; I think it's best to reduce
that divergence.
llvm_zlib and llvm_zstd remain because of how llvm depends on the
particular names.
Right now, ConstantValueStore defaults to having unknown values use
NotConstant. This generally works for the current IR, but with imports
we're expecting sparse entries which are generally unknown -- and
distinguishing between NotConstant and simply unset would be helpful. As
a consequence, add Invalid.
We discussed whether to simply have ConstantValueStore default to
Invalid going forward, or to make the default flexible. The upside to
the former is consistency, the upside to the latter is that it should
result in fewer Sets when operating on the current IR (which will more
frequently have known non-constant values). This PR offers both
approaches in separate commits, but I somewhat lean towards the latter
for fewer array resizes.
Note, a totally different approach would be to use a different class
(not ConstantValueStore) for imported IRs -- then the default of Invalid
versus NotConstant would be type-dependent. However, I expect we're
going to want to do at least somewhat consistent lookups, and using the
same ConstantValueStore for both cases allows avoiding a virtual
interface or templating. Also, I'm hoping to only maintain the
ConstantValueStore for an imported IR as part of Context (not File),
which would mean the SmallVector storage overhead is ephemeral,
mitigating one of the potential advantages of using a different type for
imported IRs.
This is one of the nicer improvements to boiler-plate as we benefit from
concepts simplifying the constraints on the templates, defaulted `!=`
finding `==`, and the `<=>` operator for a single definition of all
relational operators.
I've checked and the `!=` usages do seem to be effective at finding the
`==` definitions, but we still need both the RHS- and LHS-based
overloads of `==` as far as I can tell.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This is primarily being done for performance reasons, removing hash
lookups. The increased memory consumption is accepted.
Refactors LexicalLookup out to its own structure.
Namespaces are copied, which means also adding their name to the
underlying instruction. It happened not to be done previously; the name
was only in name lookup.
Since the only import supported right now is the default import,
functionality is limited; in the future I'll need to deal with namespace
vs package conflicts.
Tests of namespace imports are under "namespace" -- I figured this would
be best for scaling as more instructions get support.
This also improves some debugging-related output that I was trying to
use while trying to build the support.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This involves a number of supporting changes:
* The `parse_node;` member of instruction types may now have any type
derived from `Parse::NodeId` and is no longer required to have that
exact type.
* `Parse::Node::Invalid` is now a singleton object of a separate type
that is convertible to `Parse::NodeId` and its descendants. This
replaces the `Invalid` member of its descendants, and avoids having to
write long `NodeIdOneOf<...>` types when initializing variables to
invalid.
* `IndexBase` now allows `==` and `!=` comparisons between its derived
classes and types that are convertible to those types.
* A number of functions in the check stage have been changed to preserve
more type information instead of using `Parse::NodeId`.
* `NodeIdForKind<K>` (also known as `KId`) now has a `Kind` member so it
may be used to declare `NodeIdOneOf<T, U>` types without #including
`parse/typed_nodes.h`.
* `NodeIdForKind<K>` (also known as `KId`) may be implicitly converted
to `NodeIdOneOf<T, U>` if `T::Kind == K` or `U::Kind == K` (executing a
TODO).
Many of the `parse_node` members were not converted since they would
have required more extensive changes. They have been marked with "TODO"
comments.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
We have `StringLiteral`s in multiple other `Carbon` sub-namespaces.
Rename to a more specific name to avoid collisions.
We should likely also rename `Carbon::IntId` -> `Carbon::IntValueId` and
`Carbon::RealId` -> `Carbon::RealValueId`, but this collision is
prioritized because it was blocking work on typed parse nodes which
introduces a `Carbon::Parse::StringLiteralId`.
Adds a `LazyImportRef` instruction. Versus `CrossRef`, this is intended
to represent an instruction which cannot be used directly, and must be
replaced when it comes up due to name lookup. The intent is to use this
to avoid recursive loading of imported IR instructions.
Note, under this model, when `ResolveIfLazyImportRef` is called, it
essentially needs to load both inst and type information to a sufficient
point where any further attempts would hit name lookup again. That will
probably be complex, and the current implementation is just touching the
surface of the issue. I was heading down this route because it would
mean we have a limited number of points that need to consider whether
they're going to talk about a `LazyImportRef`.
I'm considering whether `CrossRef` should be dropped in favor of more
specific `Builtin` special-casing, due to the divergence of desired
behaviors. This could mean dropping the `builtins` IR since it's not
looking useful right now.
Modify `NameScope` to track whether the scope is associated with a load
error. This is to handle cases where one or more imports failed, so we
do not want to issue warnings for related scopes.
The 0-size on `ValueStore` comes up due to the changes to `NameScope`,
which make it too large for the default handling. After discussion with
zygoloid, the thought was we might want to try reserving a roughly
correct value based on parse node counts, but the stack default wasn't
buying much.
Fixes a bug where the implicit import used the package name instead of
the invalid identifier.
There's a trade-off here of explicitness in the use versus repetition,
but I'm hoping the Id offers sufficient info (also, some Ids already
relied on this, so this builds consistency). The forward declarations
I'm mixed on, but they are difficult to avoid if heading down this route
due to interdependencies between ids and types which contain ids.
This reflects how we're naming classes that derive from these classes,
and matches usage for each existing `Id` and `Index` type, except:
- `Parse::NodeId` previously inherited from `ComparableIndexBase`, and
is no longer comparable.
- `SemIR::MemberIndex` previously inherited from `IndexBase`, and is now
comparable.
Making `Parse::NodeId` non-comparable reflects that it's intended to be
an opaque identifier for a node and that the ordering is an
implementation detail rather than part of the intended public interface.
`PostorderIterator` and `SiblingIterator` still rely on the numerical
meaning of `NodeId`s, but that's OK since they're part of the node
implementation.
I was suggesting this because `FloatingPoint` is pretty long. `int` and
`float` should be familiar abbreviations. `unsigned` should be familiar
to developers too, but `UnsignedInt` still feels usefully clearer for
the additional chars.
These are manual fixes; mostly from clang-tidy, some from clangd (which
notes unused includes).
In typed_insts, adding inlline due to misc-definitions-in-headers. Per
discussion, clang-tidy is wrong, but inline silences it.
For parameter name skew in definition versus declaration, I'm just using
the name from the definition.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Add a `NameId` that is effectively just a wrapper around a `StringId`,
with
some additional predefined values for names that don't correspond to
strings, such as the name of `self` or the function's return slot.
Following up on discussion yesterday regarding this split.
Note, I'm expecting #3341 to do IdentifierId -> NameId in SemIR. It
might be worth adding NameId creation directly to StringStore if you're
content with this setup though.
Advantages:
- Allows lexing/parsing in parallel, since they are modifying fully
separate ValueStores.
- Allows SemIR to reliably be stored hermetically.
Disadvantages:
- Creates overlapping storage of duplicate strings when multiple files
are compiled together.
- Prevents Ids from being uniquely compared cross-file.
Per discussion, the decision is that the advantages are more important.
The looser ownership remains because both SemIR checking and
metaprogramming may still generate things we would want to deduplicate.
It would be somewhat odd if TokenizedBuffer owned something that
checking modified.
Provides an adapter for the llvm::yaml API because it otherwise needs a
bunch of const/non-const definitions, and the traits are difficult to
diagnose issues with. The current approach is pretty simple to use, even
if it's not super efficient (which, yaml output is more of a debugging
thing so I'm not really expecting it to be an issue).
Changes the format of yaml output to provide more index information,
just as reminders when seeing something like `node+0`. Note this would
create more churn in deltas if we were reliant on the output yaml in
tests, but we aren't so it should be okay.
This standardizes on having ValueStore and related structures provide
printing, removing the handlers in file.cpp.
The `[]` is provided for empty sequences versus if there was simply
nothing, in which case it would be a sequence when non-empty, and a null
value when empty. Consistently (and explicitly) providing sequences
feels easier to understand.
The changes to the output yaml are overall more terse. My hope is that
this is an improvement for most readers.
Also fixes printing of APInt, defaulting to unsigned for consistency
with Carbon's use.
Finishing what #3316 started, add more bespoke ValueStore-like
structures to File. With this, the things which previously had somewhat
boilerplate Add/Get functions are now all on side classes, giving a
uniform style of API for calling.
Note, I was on the fence about making things public on ValueStore. If
it's preferred that I make some things there protected I certainly can,
there's just a trade-off that may mean more distinct child/wrapper
types.
Building on #3313, start using ValueStore on File. Functions and classes
are straightforward. Types here I present as a borderline case where
maybe we want a more bespoke API, but maybe this is okay? Most other
things probably need a slightly different API, which although I might do
that for a consistent interface, felt more out-of-scope for this change.
This replaces the printing that was removed from SemIR's raw dump. It's
separate because (for example) lexing generates shared values, and so
reviewing them is not specific to any particular phase.
Building on #3311, change SemIR to use the SharedValueStore. Since this
removes hermeticity, raw output no longer prints ints, reals, and
strings. TokenizedBuffer accessors are modified to return IDs because
values are often passed through in semantics without needing to read
them.
I would've put SharedValueStores on Context, except for the
GetArrayBoundValue convenience method. I felt awkward removing that, so
it's on File, at least for now. That's then used by the formatter and
Lower too. The flipside of this is that TokenizedBuffer has a
SharedValueStores only for printing, so maybe that's similar enough to
what File is doing.
This doesn't start shifting other SemIR members to ValueStore, but that
seems like a next step.
This updates lexing to use the data. I'll do checking separately, just
to split changes.
Note the ValueStore structure is also set up such that SemIR::File can
use it for other fields.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Previous code could fail silently, only caught with yaml output
mismatches. This uses ErrorOr to be explicit about the error detection;
recovery isn't supported because we should only print valid yaml (in
tests, at least).
Also moves the test helpers from //toolchain/base to
//toolchain/testing.
Switches tree_test to use the matchers so that, on mismatch, gtest
prints a path to the mismatch (the straight value means it was just
printing "not equal").
Addresses
https://github.com/carbon-language/carbon-lang/pull/3217#discussion_r1323586836
Building on #3214 and #3215, updates sem_ir yaml to be:
```
- filename: name
sem_ir: [ ... ]
```
Also, changes the textual format from `package { ... }` to `file
<filename> { ... }`. My thought on packages there is:
```
file "foo.carbon" {
package MyPackage
...
}
```
The reason for putting the file first is that it's easier if we put what
we're grouping on first, whereas the package is an "annotation" on the
file.
With the toolchain splitting namespaces, ostream.h's `operator<<`
templates aren't reliably found with name lookup, likely due to the loss
of associated namespaces (zygoloid commented on this at
https://github.com/carbon-language/carbon-lang/pull/3161#discussion_r1307941999).
This is especially a barrier to moving the lex files into `Carbon::Lex`;
versus other parts of the toolchain, they contain more printable types
which are used cross-namespace, including `Carbon::Testing`. As a
consequence, I'm looking at migrating ostream.h to a more reliable
approach that doesn't rely as much on everything being in the `Carbon`
namespace.
A lot of this is more boring "remove unused header", plus some other
minor cleanups. I think the most significant changes were:
- yaml_test_helpers.cpp is doing a switch on an unsigned int, comparing
to enum values.
-
[bugprone-switch-missing-default-case](https://clang.llvm.org/extra/clang-tidy/checks/bugprone/switch-missing-default-case.html)
is unhappy with EnumBase, but correctly identified
yaml_test_helpers.cpp, so I'm opting to address it rather than disabling
it even though it needs NOLINT in several locations as a result, in
addition to what I think are some low-value `default` cases. I'd be fine
going the other way with this too and disabling it globally (I could see
it being noisier in the explorer).
- MarkInitializerFor swaps the argument names between the .h and .cpp. I
think the .cpp had the order as intended.
- There's a new-ish
[performance-enum-size](https://clang.llvm.org/extra/clang-tidy/checks/performance/enum-size.html)
which I'm basically treating as "add int8_t to enums".
My main motivation here is to just clean up as many of these as I can so
that I stop seeing them in vscode.
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
Jon Ross-Perkins
Chandler CarruthandJon Ross-Perkins
josh11bandRichard Smith