Some background information is at https://bazel.build/external/migration
Trying to handle the simple cases first. This adds a requirement for
bazel 7 due to differences in bzlmod handling between 6 and 7 (also
discussed on
[#infra](https://discord.com/channels/655572317891461132/707150492370862090/1184942191412510720)).
Bazel seems to be okay with a partial migration such as ths.
The python import behavior has subtly shifted, so `carbon.` is no longer
part of import paths. There's a version-incompatible change for `@@`.
bzlmod makes repos sometimes show as `name~version`.
`target-determinator` seems to be okay with `@@` after a version update.
Things not moved here are things that basically need more dep work:
- clang_register_toolchains because I need to dive into its format.
- llvm-project because we need something slightly atypical, I need to
make sure patching and the repo work carries over.
- com_google_libprotobuf_mutator is sufficiently atypical that it
doesn't have a module already, but should be one of the easier things to
fix.
- brotli/woff2: I think we should actually consider removing these. But
again, they're not trivial moves.
- treesitter due to toolchain registration, which has shifted a bit.
- rules_nodejs because treesitter depends on it in an awkward way to
migrate.
Running `bazel test //...` reported:
```
Test execution time outside of range for MODERATE tests.
Consider setting timeout="short" or size="small".
```
This change adds size="small" to avoid such warnings being reported.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Rationale: this convention avoids forcing closely-related code to be far
apart in the namespace hierarchy, and vice versa. By the same token, it
makes the namespace hierarchy more consistent with the directory
hierarchy.
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
This shifts explorer's file_test to use ExplorerMain in order to be able
to pass arguments similar to how the command line would. It also means
that various output wrapping done by the command line is shared, no
longer copied by file_test.
In order to help make this work, I plugged vfs::FileSystem into
explorer, similar to how we're doing this on the toolchain side (might
try to share more code later). I was having trouble getting an overlay
working, I think due to how paths are specified -- but due to the issues
in fixing this, I'm just loading the prelude into the InMemoryFilesystem
for now.
Under this approach, I'm unifying more of the file versus string
handling. I think we should shift towards an approach where, like
toolchain code, anyone trying to use Explorer should use a vfs
implementation to supply code. This should reduce the number of distinct
code paths which we're maintaining/testing.
Short-term, this allows the trace testdata to be merged into file_test
with less special casing, using ARGS:. Long-term, it means that the
file_test knows the ARGS to pass to generate checks to match against,
which is the direction I'm planning for autoupdate.
This will be more correct if a user file is named prelude.carbon, and
reduces the explorer runtime by about 10% by removing a string
comparison from the check for whether trace output is enabled.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Breaking apart namespaces from the main declaration list is, I think,
the most scalable approach to this. If we have more divergence in
supported declarations it gives a fairly straightforward way for
splitting, and roughly mirrors how class_body splits.
Fixes#2980
This is just a cleanup. Since we now have a testing directory, I think this is a better home for testonly libraries than //common. (I was thinking about this when I was considering adding more test_raw_ostream deps)
Currently ParseAndExecute-style logic is done in main.cpp and a test. #2799 is adding another test that needs it, as is #2811.
Also more clearly marks fuzzing as testing.
This is being extracted out of #2799 in order to try unblocking progress while review continues.
Removes `__continuation`, `__await`, and `__run`.
In part here, the discussion was that while the feature had been useful for validating the early explorer design, it's no longer needed for that role as the explorer is now quite robust. Continuations have been experimental and, at this point, don't have an owner pushing to a proposal.
The triggering factor is that, as we push to address fuzzer issues, I ran into a crash bug in this code; basically, `fn Main() -> i32 { __await; return 0; }`. When I mentioned this, the reaction seemed to trend towards removal of the feature.
Rename value categories to expression categories based on [Discord discussion](https://discord.com/channels/655572317891461132/753021843459538996/1092924035517665332) regarding naming and behavior.
>* let expression -> value expression
>* var expression -> reference expression
>* located expression -> initializing expression
>So:
>- "value expressions" produce values (with no associated location). "reference expressions" produce a location of an existing value. "initializing expressions" take a location and initialize it.
>- A let binding is initialized by a value expression, because lets represent values (with category conversions performed as needed, but if a conversion is performed from a different category of expression, the value of the object is pinned for the lifetime of the let).
>- A var binding is initialized by an initializing expression, without performing a copy (with category conversions performed as needed, calling a copy constructor if the initializer is a different expression category).
>- The & operator requires a reference expression, and it's an error to give it other kinds.
>- The left-hand side of . requires a value expression when calling a function with a non-addr receiver, and requires a reference expression when calling a function with an addr receiver (it's an error to give it a value expression, and for an initializing expression, a temporary is materialized).
Changes
* Rename "value category" to "expression category"
* Rename Var and Let value categories to Value, Reference, and Initializing expression
* Rename `lvalue` to `location` (most of the time)
This PR is making two main changes to the Explorer and Toolchain:
- Replace the `is` keyword in `where SomeType is SomeInterface` with `impls`, so it is `where SomeType impls SomeInterface`
- Rewrite uses of the "impls" to something else to avoid, frequently "`impl` declarations" or "implementations", to avoid confusion with the `impls` keyword.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
The strategy that we use for now to support template instantiation is to check the impl declaration as if it were a generic, but to defer all checking of the impl definition until we see a use in which all template parameters have arguments. At that point, we clone the impl definition and type-check the whole thing, with constant values set on the template parameters corresponding to the given arguments.
No caching of template instantiations is performed yet; each time we form a reference to a template instantiation, we instantiate it afresh. We also don't implement the name lookup rule from #949 yet; lookups during template instantiation look only in the actual type and not in the constraint.
Depends on #2699
### Features
* Add virtual `destructor`s support (Closes#2521)
* Check virtual override for virtual destructors
* Error if attempting to `Delete` a class that does not have virtual destructors from a base class pointer
### Implementation
* Update parser to support virtual override introducers for destructors
* Check virtual override for class destructor and add to class vtable if necessary
* Add corresponding tests
### Notes
Contrary to initial implementation, this implementation leverages the `Address` structure and implements a new `Address::DowncastedAddress()` method to get address from child most class from a base class address. This avoids the need to use `GetAllocationId` and its issues when it comes to having multiple values for an `AllocationId`.
### Next work
Following this PR, we need to:
* Check when using `Delete` that the class was allocated with `New` (WIP)
* Drop the old `GetAllocationId(Value*)` in favor of a better system (WIP)
This is intended to address currently flaky timeouts that are likely caused by the size of the prelude. I'm addressing a performance bottleneck in AnalyzeProgram with trace output. Trying to omit prelude traces reduces most trace output significantly, and I think it'll scale better as the prelude size increases.
The basic mechanics here are:
- In order to consistently track whether tracing is on, I've added a TraceStream class, explorer/interpreter/trace_stream.h.
- The AST now has a num_prelude_declarations field, so that it's provided where the boundary is.
- In order to mark where we try to skip prelude output, I've added calls to set_in_prelude in type_checker.
- In exec_program, I just use num_prelude_declarations directly to skip over.
- Everywhere checks TraceStream::is_enabled before printing, similar to the std::optional check that was previously used.
This does add some timing output in order to better diagnose where slowness is coming from, when tracing. It also adds "verbose" targets to make it easier to get the trace output.
So for example, here's a timing for zero.carbon:
```
Timings:
- Parse: 13ms
- AddPrelude: 25ms
- AnalyzeProgram: 116ms
- ExecProgram: 12ms
```
If I make a small change to just not set skipping_prelude (essentially getting back to current output):
```
- Parse: 13ms
- AddPrelude: 25ms
- AnalyzeProgram: 2359ms
- ExecProgram: 57ms
```
Thus in this trivial example, I'm eliminating about 95% of the execution time.
Note this approach could still be refined in a few ways:
- We could add a flag to allow overriding in_prelude. It should be a small amount of work after this change. But it's a little consistent with how parser_debug works, that it won't print prelude output by default (unless there's an error).
- Execution could skip messages involving initialization of globals declared in the prelude. This is a little noisy right now, but I don't think it's significant for performance because ExecProgram is tiny.
- Once files are more separated, we should be able to change the num_prelude_declarations/set_in_prelude approach.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The design of choice types expects the declaration of an alternative to match the usage: if an alternative is declared as `None`, then it should be used as `None` not `None()`, and if it is declared as `None()` then it should be used as `None()` not `None`. Update explorer to match.
Also clean up the handling of choice types and alternative values a little in general, by moving away from identifying choice types and alternatives as strings and towards identifying them symbolically.
Closes#2422
Support for global variables is still missing; they're a bit more tricky because they use a pattern to introduce their name.
Prior to this change, explorer heavily relied on name comparisons to determine whether two declarations declare the same entity. Some of those instances are fixed in this PR, but more remain to be fixed, and some TODOs are added for some harder-to-fix instances.
First step towards permitting declarations within namespaces. Supports only functions within namespaces for now, with no way to call those functions except from within other such functions. Unqualified lookups within a function in a namespace look in that namespace first.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Add support for user-defined assignment, as well as compound assignment and increment, following the design direction in pending proposal #2511.
Some of this isn't fully testable yet: because explorer doesn't properly support `impl` specialization, the blanket `impl`s in the prelude prevent types from customizing assignment.
Add basic support for `match_first` declarations to explorer, as described in https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/generics/details.md#prioritization-rule. The concrete syntax here is not yet approved, so `match_first` is renamed to `__match_first` for now, but the semantics are necessary to implement other approved features in explorer so we're intentionally getting a little ahead of the design here.
Also make minor updates to the skeletal design in
docs/design/name_lookup.md following #2113, as there are no longer any prelude names that are made available to unqualified name lookup by default.
Add `type` to the keyword list in
docs/design/lexical_conventions/words.md, following #2360.
Features:
* Add basic support for `impl` virtual methods (override virtual method)
* Error on invalid declaration for `impl` and `virtual`, covering simple use cases
* Add `abstract` fn parser-only support
Changes:
* Modify parser to handle new function specifiers, resolve conflicts
* Group `virtual_override` and `FN`, and group `impl_kind` and `IMPL` to avoid ambiguities with around `impl` token parsing
* Add new `VirtualOverride` enum for function declarations, and matching `virt_override() -> VirtualOverride` getter
* Update function declaration logic
Depends on #2462
Features:
* Add `virtual` virtual override keyword for functions
* Support `virtual` class methods using dynamic dispatch
Changes:
* Add `vtable` in `NominalClassType`,
* Add `NominalClassValue**` in class values pointing to descendant-most class
* Resolve virtual methods during member lookup
Limitations:
* Does not include yet `impl`, `abstract` virtual override keywords, or the complete logic for virtual function declaration
Depends on #2460
Relates to #1881
Relates to #2493
Protobufs code hits a warning with the latest system headers on macOS.
I figured this may have been fixed so I updated protobufs and Bazel to
the latest releases. This generally cleaned things up.
However, it actually added *more* warnings. This clearly isn't a really
well tested path. In fact, we already have a disabled warning that we'd
like for Carbon code because LLVM isn't clean for that warning.
So I've switched our warning strategy to a more durable approach of
suppressing all warnings for external repository headers and source
files. This lets us re-enable the missing warning and should fix the
protobuf warning that started me down this twisty path.
Sadly, we *have* to update to Bazel 6 in order to have the necessary
flag to use this approach to suppressing warnings, so I couldn't do this
as two PRs cleanly. =/ That's why I've bundled both the Bazel (and
protobuf) updates with the warning strategy change.
Last but not least, I've fixed several unused parameters in Carbon's
code that our warnings now catch.
Relates to https://github.com/carbon-language/carbon-lang/issues/1881
- Add support for `.base` field in structs for [parent class initialization](https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/classes.md#constructors)
- Disabling base class initialization without `.base`
- Support class constructors (`Create() -> Self`) for base classes
- Direct access to base class(es) attributes with `object.var` remains unaffected
Changes:
- Add `TypeChecker::FieldTypesWithBase` to help assessing if a struct with `base` fields can be converted to a class
- Add a new `base_type()` attribute+getter to `NominalClassDeclaration` to as a first step to allow resolving parametrized classes
- Add a new `base` attribute+getter to `NominalClassValue` that contains the base class `NominalClassValue`. It is currently used mainly to get and set members of a class object.
- Add `Interpreter::ConvertClassWithBase` to build `NominalClassValue` from a init struct, that contains `.base` fields with either `NominalClassValue` or `StructValue`
- Add `FindClassField` to find a field in a class or its base classes
- Remove superfluous `ClassDeclaration::base()` in favor of `ClassDeclaration::base_type()`
Limitations;
- Though some work is done in that direction, parametrized base class where time is not know at the declaration site are not supported. Namely the example below does not compile
```
base class A(T:! Type) {}
class B(T:! Type) extends A(T) {}
```
But this one is functional already
```
base class A(T:! Type) {}
class B extends A(i32) {}
```
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
In particular, this means that a type can implement `ImplicitAs(Type)` and have values of that type behave like types.
This implies that `()` and `{}` are no longer types. They are now values whose type is the result of converting `()` or `{}` to type `Type`, as has been discussed recently and seems to be the supported direction. This fixes various cases where these types were previously mishandled.
Note, not trying to address every last error, just some obvious/easy ones.
```
/usr/local/google/home/jperkins/dev/carbon-lang/common/string_helpers.cpp:200:13: warning: prefer transparent functors 'less_equal<>' [modernize-use-transparent-functors]
auto le = std::less_equal<const char*>();
^
/usr/local/google/home/jperkins/dev/carbon-lang/toolchain/semantics/nodes/function.h:22:45: warning: pass by value and use std::move [modernize-pass-by-value]
Function(ParseTree::Node node, NodeId id, llvm::SmallVector<NodeRef> body)
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/ast/declaration.cpp:230:14: warning: static member accessed through instance [readability-static-accessed-through-instance]
return cast<SelfDeclaration>(declaration).name();
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1104:33: warning: std::move of the variable 'impl' of the trivially-copyable type 'ConstraintType::ImplConstraint' has no effect [performance-move-const-arg]
impl_constraints_.push_back(std::move(impl));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1145:36: warning: std::move of the variable 'rewrite' of the trivially-copyable type 'ConstraintType::RewriteConstraint' has no effect [performance-move-const-arg]
rewrite_constraints_.push_back(std::move(rewrite));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1156:32: warning: std::move of the variable 'context' of the trivially-copyable type 'ConstraintType::LookupContext' has no effect [performance-move-const-arg]
lookup_contexts_.push_back(std::move(context));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/fuzzverter.cpp:74:42: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
/*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/proto_to_carbon_test.cpp:34:55: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
const ErrorOr<AST> ast = Carbon::Parse(&arena, f, /*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/proto_to_carbon_test.cpp:42:41: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
&arena, f, source_from_proto, /*trace=*/false);
^
./explorer/syntax/parse.h:25:59: note: 'parser_debug' declared here
std::string_view file_contents, bool parser_debug)
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/syntax/parse_test.cpp:27:60: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
ParseFromString(&arena, "file.carbon", FileContents, /*trace=*/false);
^
./explorer/syntax/parse.h:25:59: note: 'parser_debug' declared here
std::string_view file_contents, bool parser_debug)
^
/usr/local/google/home/jperkins/dev/carbon-lang/migrate_cpp/cpp_refactoring/var_decl.cpp:57:58: warning: string concatenation results in allocation of unnecessary temporary strings; consider using 'operator+=' or 'string::append()' instead [performance-inefficient-string-concatenation]
segments.push_back({type_loc_class, qual_str + " " + range_str});
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/ast_to_proto_test.cpp:105:55: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
const ErrorOr<AST> ast = Carbon::Parse(&arena, f, /*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
```
Implements basic support for rewrite constraints as proposed in #2173.
Support for associated constants and complex constraints in general is also made more robust: argument deduction now properly computes and substitutes witnesses, and interface declarations track a more correct description of the constraint that they introduce than the one we previously built.
Instead of forming a `SymbolicWitness` that contains an `Expression` when we can't directly resolve a witness to an `impl`, form different kinds of `Witness` values for the various situations:
- A `BindingWitness` witnesses that a type implements a constraint by reference to an `ImplBinding`.
- A `ConstraintWitness` witnesses that a type implements a constraint by reference to witnesses for each of the impl constraints within the constraint.
- A `ConstraintImplWitness` witnesses that a type implements a constraint by reference to a larger constraint which contains that constraint as an impl constraint.
Remove `SymbolicWitness` values and `InstantiateImpl` expressions, which are now unused. In order to remove the final usage of `SymbolicWitness`, I fixed a TODO to give `Self` the proper type within an interface declaration. I don't think this is an observable change.
No functional change intended.
* Multiplication and division have the same priority.
* A new builtin interface DivWith is added.
* In some tests expecting a compilation error (syntax error), the error
message now says it is expecting SLASH or binary *.
libprotobuf_mutator and abseil have warnings that are already fixed in newer versions.
bison outputs code that hits new warnings; there I'm just silencing it because testing a newer bison version is more difficult.
Fixes#1650Fixes#1660
Jon Ross-Perkins
Jonathan B. CoeandChandler Carruth
Geoff Romer
Richard SmithandJon Ross-Perkins
josh11b
kshokhin
Adrien Leravat
Priyananda Shenoy
pmqtt
Junhee Cho
