With the toolchain splitting namespaces, ostream.h's `operator<<`
templates aren't reliably found with name lookup, likely due to the loss
of associated namespaces (zygoloid commented on this at
https://github.com/carbon-language/carbon-lang/pull/3161#discussion_r1307941999).
This is especially a barrier to moving the lex files into `Carbon::Lex`;
versus other parts of the toolchain, they contain more printable types
which are used cross-namespace, including `Carbon::Testing`. As a
consequence, I'm looking at migrating ostream.h to a more reliable
approach that doesn't rely as much on everything being in the `Carbon`
namespace.
This function hid the `Value::Print` function from the base class, and
provided different output. As far as I can tell, the only caller is the
implementation of `Value::Print`.
#2569 added PrintAsID to //common/ostream.h, but given it's
explorer-specific behavior, I don't think it's the right home for it.
Noticed this while pondering better ostream interfaces.
The different approach to Names avoids the issues with trying to define
a static member (or also member function) of the templated instance of
Carbon::Internal::EnumBase from a non-enclosing namespace such as
Carbon::SemIR.
Note, I'm trying to do this from the cpp file. An alternative might be
to do `inline constexpr llvm::StringLiteral Names[]` in the .h file, but
I think concerns had been raised about that needing deduplication.
This PR removes `PrintDepth` from statement and declaration.
Implements `PrintIndent` for better indented formatting along with
various changes to make printing of statements and declarations better.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
Defines methods into `TraceStream` for adding line prefixes. Instead of
directly using string literals.
Example usage,
```
trace_stream_->Start() << "declaring ... " << ... ;
```
will result in,
```
->> declaring ...
```
Updates the trace output for type checking to be more consistent like
the rest of trace output.
This PR also includes few changes in the lit tests related to trace
output.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
See arena.h for discussion of what canonicalization means in this
context. This is primarily intended to support implementing a memo table
of template instantiations to resolve#2951, but could be useful for
other purposes as well.
Additional changes:
- Pass `VTable` constructor parameters by pointer, to avoid the need to
define `operator==` and `hash_value` for it.
- Clean up the recurring pattern of allocating identical `NamedElement`s
on the stack and heap. Instead we always allocate it on the heap and
pass it by pointer.
- Add `Print()` and `Dump()` to `Bindings` as a debugging convenience.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Adds the following information about name resolution to the trace
output:
* Name resolution process
```
** resolving stmt | decl `<stmt | decl>` (<source_location>)
...
** finished resolving stmt | decl `<stmt | decl>` (<source_location>)
```
* When a name is added to the static scope
```
--- declared `<name>` as `<entity>` in `<scope>` (<source_location>)
```
* When a name is resolved
```
--- resolved `<name>` as `<result>` in `<scope>` (<source_location>)
```
* Marking a name declared/usable
```
--- marked `<name>` <usable | declared but not usable> in `<scope>`
```
This will be more correct if a user file is named prelude.carbon, and
reduces the explorer runtime by about 10% by removing a string
comparison from the check for whether trace output is enabled.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Prevent copies when initializing value expression from reference
expression. This is based on
https://github.com/carbon-language/carbon-lang/pull/2006, which
introduces expression categories, and how it is possible to convert
to/from those different categories. Continuation of
https://github.com/carbon-language/carbon-lang/pull/2907
## Functional changes
* Initializing a value expression from a reference expression takes its
value without a copy
* Reading from the value expression causes an error if the value changed
from the time it was initialized
* In this situation, prevents a copy both for variable definitions, and
call parameter bindings
## Main implementation changes
* Add new `ExpressionCategoryAction`, which evaluates an expression and
returns an `ExpressionValue` containing its category and address (if
any), in addition to the resulting `Value*`
* `ExpressionAction`s now invokes `ExpressionCategoryAction` and unwraps
the returned `ExpressionValue`
* `RuntimeScope::BindAndPin` method, and corresponding when attempting
to read a `value_node`.
## Next work
* Avoid unnecessary copies from value expression to value expression,
after ensuring that even value expression temporaries are registered for
destruction (https://github.com/Pixep/carbon-lang/pull/9)
This fixes a bug where Explorer would not detect when:
- an implementation used a method to implement a class function in an interface
- an implementation used a class function to implement a method in an interface
- an implementation method used `addr self` when the interface method did not
- an interface method used `addr self` when the implementation method did not
at type checking time. This would then cause a crash at runtime.
Closes#2857
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Add partial support for initializing expressions for variable declaration. This is based on https://github.com/carbon-language/carbon-lang/pull/2006, which introduces expression categories, and how it is possible to convert to/from those different categories.
## Functional changes
* Initializing expressions initialize directly the provided storage when used to initialize a variable.
* Allows initializing expressions to avoid a copy when using `[var|let] name: type = call_expression(...)` by initializing `name` in-place.
* Support `returned var: ...` and `return <expr>`
* Support nested initializing expressions
## Main implementation changes
* Updated PatternMatch logic to handle expression categories
* Updated `VariableDefinition` interpreter statement to allocate and pass a location to initializing expressions
* Update statement actions to allow passing an allocation, used by return expr or returned var
* Modified the RuntimeScope API to be one step closer to the memory model we want to have
* Remove `GetAllocationId` and older `Bind` which don't apply
* New set of tests to highlight those different situations
* Added a new intrinsic to print the allocation stack (and make sure we behave correctly, beyond visible side effects)
## Next work
* Dedicated `Action` to retrieve expression category information in the interpreter (https://github.com/carbon-language/carbon-lang/pull/2927)
* Avoid copies when initializing value expression from reference expression and prevent mutations for the duration of the "pinning" (https://github.com/carbon-language/carbon-lang/pull/2927)
* Avoid unnecessary copies from value expression to value expression, after ensuring that even value expression temporaries are registered for destruction.
* Avoid unnecessary copies when binding function arguments
There are a few implicit conversions that are implemented by an `impl` of `ImplicitAs` that delegates to code in explorer:
- Converting between tuple types
- Converting from tuples of types to `type`
- Converting from tuples of values to an array type
- Converting between struct types
- Converting from a struct type to a class type
These conversions can all rely on performing more conversions for elements or subobjects, but previously those inner conversions could only be performed if they were built into explorer. This change instead uses the full implicit conversion machinery in explorer to perform these conversions, including searching for a user-defined `impl` of `ImplicitAs` when necessary.
For example, this permits a conversion from `{.a: T}` to `{.a: U}`, or from `(T, T)` to `(U, U)`, or from `(T, T)` to `[U; 2]` when there is a user-defined conversion from `T` to `U`.
Depends on #2878
This is just a cleanup. Since we now have a testing directory, I think this is a better home for testonly libraries than //common. (I was thinking about this when I was considering adding more test_raw_ostream deps)
When a template has an argument that involves a generic parameter, we're supposed to delay instantiation until we know the concrete value, but explorer is not set up to do that yet, so for now we instead instantiate the template with the symbolic argument. When that happens, bring the constraints on the generic parameter into scope so they can be used inside the template instantiation.
This requires adding a new search over a value for the generic parameters that appear within it; a `VisitNestedValues` visitor is added to visit all the `Value`s nested with a value, and also convert an existing place where we were doing the same thing in a way that was incorrect (but harmlessly incorrect for now) to use it.
This is needed by #2881, which needs implementations of `ImplicitAs` for nested types when instantiating a builtin impl of `ImplicitAs` for an aggregate type.
Co-authored-by: Geoff Romer <gromer@google.com>
adds trailing comma to single element tuple
example:
```carbon
package ExplorerTest api;
fn Main() -> i32
{
var a: (i32,) = (1, 2);
return 0;
}
```
change in error message:
```diff
-type error in initializer of variable: '(i32, i32)' is not implicitly convertible to '(i32)'
+type error in initializer of variable: '(i32, i32)' is not implicitly convertible to '(i32,)'
```
Removes `__continuation`, `__await`, and `__run`.
In part here, the discussion was that while the feature had been useful for validating the early explorer design, it's no longer needed for that role as the explorer is now quite robust. Continuations have been experimental and, at this point, don't have an owner pushing to a proposal.
The triggering factor is that, as we push to address fuzzer issues, I ran into a crash bug in this code; basically, `fn Main() -> i32 { __await; return 0; }`. When I mentioned this, the reaction seemed to trend towards removal of the feature.
running clang-tidy locally got a bunch of warning.
I have ignored `bugprone-unchecked-optional-access` for now, because there are large number of occurrences.
Note the dest_class case didn't actually check dyn_cast success until after using the pointer. That's the case I noticed first, but also cleaning up a couple other small related things.
Rename value categories to expression categories based on [Discord discussion](https://discord.com/channels/655572317891461132/753021843459538996/1092924035517665332) regarding naming and behavior.
>* let expression -> value expression
>* var expression -> reference expression
>* located expression -> initializing expression
>So:
>- "value expressions" produce values (with no associated location). "reference expressions" produce a location of an existing value. "initializing expressions" take a location and initialize it.
>- A let binding is initialized by a value expression, because lets represent values (with category conversions performed as needed, but if a conversion is performed from a different category of expression, the value of the object is pinned for the lifetime of the let).
>- A var binding is initialized by an initializing expression, without performing a copy (with category conversions performed as needed, calling a copy constructor if the initializer is a different expression category).
>- The & operator requires a reference expression, and it's an error to give it other kinds.
>- The left-hand side of . requires a value expression when calling a function with a non-addr receiver, and requires a reference expression when calling a function with an addr receiver (it's an error to give it a value expression, and for an initializing expression, a temporary is materialized).
Changes
* Rename "value category" to "expression category"
* Rename Var and Let value categories to Value, Reference, and Initializing expression
* Rename `lvalue` to `location` (most of the time)
This keeps all of our C++ code written in C++ source files, and avoids the increasing size of the RTTI generation script we'd started to see in #2699.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This PR is making two main changes to the Explorer and Toolchain:
- Replace the `is` keyword in `where SomeType is SomeInterface` with `impls`, so it is `where SomeType impls SomeInterface`
- Rewrite uses of the "impls" to something else to avoid, frequently "`impl` declarations" or "implementations", to avoid confusion with the `impls` keyword.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
The strategy that we use for now to support template instantiation is to check the impl declaration as if it were a generic, but to defer all checking of the impl definition until we see a use in which all template parameters have arguments. At that point, we clone the impl definition and type-check the whole thing, with constant values set on the template parameters corresponding to the given arguments.
No caching of template instantiations is performed yet; each time we form a reference to a template instantiation, we instantiate it afresh. We also don't implement the name lookup rule from #949 yet; lookups during template instantiation look only in the actual type and not in the constraint.
Depends on #2699
### Features
* Add virtual `destructor`s support (Closes#2521)
* Check virtual override for virtual destructors
* Error if attempting to `Delete` a class that does not have virtual destructors from a base class pointer
### Implementation
* Update parser to support virtual override introducers for destructors
* Check virtual override for class destructor and add to class vtable if necessary
* Add corresponding tests
### Notes
Contrary to initial implementation, this implementation leverages the `Address` structure and implements a new `Address::DowncastedAddress()` method to get address from child most class from a base class address. This avoids the need to use `GetAllocationId` and its issues when it comes to having multiple values for an `AllocationId`.
### Next work
Following this PR, we need to:
* Check when using `Delete` that the class was allocated with `New` (WIP)
* Drop the old `GetAllocationId(Value*)` in favor of a better system (WIP)
This is intended to be used for template instantiation, but for now takes no stance as to what it's cloning.
This is tested by parsing and cloning all of our test files, and checking that the result of converting each AST to proto is the same as the result of cloning and then converting each AST to proto.
This is in preparation for template instantiation being triggered during substitution, and being able to fail.
Fix rule-of-three violation (missing assignment operator) in `Error` that got in the way of using it to hold an error temporarily in a failed transformation.
As described in [the generics design](https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/generics/details.md#type-structure-of-an-impl-declaration), `impl` declarations are prioritized by type structure. Given two `impl` declarations that match a `type as interface` query, the one that describes the longest prefix of the query without using placeholders is preferred.
We implement this by putting all impls in a total order, first by type structure equivalence classes and then by lexical order. When matching an impl, we walk this total order, and stop once we find a match and reach the end of its equivalence class.
Equivalence classes are determined by finding the locations of the "holes" (the positions where deduced parameters appear) within the type structure, viewed as a tree. Two impls are in the same equivalence class if their holes are in the same place, and equivalence classes are ordered based on a reverse lexicographical ordering of their holes.
Explorer doesn't keep the `Bindings` list for a parameterized type in any particular order, but the type structure rule requires that we consider them in lexical order. In order to support this, we now track an index on the declared parameters of each generic. This is a simple numbering of enclosing generic parameters, both on that generic and on all lexically enclosing generics.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
These are used by the AST in lots of ways, and this resolves various layering issues.
This means that `AllocationId` also lives in ast/, but is managed by interpreter/. A better layering here would be desirable, but this seems good enough for the time being.
google-readability-function-size and readability-function-size were _both_ triggering on TypeCheckExp. It looks like the Google version may be a subset of the general version, so I've disabled the Google version while keeping the general version and adding a NOLINT for it.
I manually removed the `const` in cases like `Nonnull<const VTable* const>` based on the readability-const-return-type warning. i.e., where a return type is a pointer, the `const` isn't meaningful and the tidy check was warning about that.
Added a NOLINT for misc-definitions-in-headers on IsRecursivelyTransformable. I think that's the right choice for the `constexpr`, the warning didn't feel accurate and may be getting confused by the templating.
I changed the structure of `carbon_files` in the fuzzer because the `new` was causing a warning about exceptions. However, also disabling bugprone-exception-escape because it's what was flagging this and it's not really a helpful warning.
Other changes were automated.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This is intended to address currently flaky timeouts that are likely caused by the size of the prelude. I'm addressing a performance bottleneck in AnalyzeProgram with trace output. Trying to omit prelude traces reduces most trace output significantly, and I think it'll scale better as the prelude size increases.
The basic mechanics here are:
- In order to consistently track whether tracing is on, I've added a TraceStream class, explorer/interpreter/trace_stream.h.
- The AST now has a num_prelude_declarations field, so that it's provided where the boundary is.
- In order to mark where we try to skip prelude output, I've added calls to set_in_prelude in type_checker.
- In exec_program, I just use num_prelude_declarations directly to skip over.
- Everywhere checks TraceStream::is_enabled before printing, similar to the std::optional check that was previously used.
This does add some timing output in order to better diagnose where slowness is coming from, when tracing. It also adds "verbose" targets to make it easier to get the trace output.
So for example, here's a timing for zero.carbon:
```
Timings:
- Parse: 13ms
- AddPrelude: 25ms
- AnalyzeProgram: 116ms
- ExecProgram: 12ms
```
If I make a small change to just not set skipping_prelude (essentially getting back to current output):
```
- Parse: 13ms
- AddPrelude: 25ms
- AnalyzeProgram: 2359ms
- ExecProgram: 57ms
```
Thus in this trivial example, I'm eliminating about 95% of the execution time.
Note this approach could still be refined in a few ways:
- We could add a flag to allow overriding in_prelude. It should be a small amount of work after this change. But it's a little consistent with how parser_debug works, that it won't print prelude output by default (unless there's an error).
- Execution could skip messages involving initialization of globals declared in the prelude. This is a little noisy right now, but I don't think it's significant for performance because ExecProgram is tiny.
- Once files are more separated, we should be able to change the num_prelude_declarations/set_in_prelude approach.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The design of choice types expects the declaration of an alternative to match the usage: if an alternative is declared as `None`, then it should be used as `None` not `None()`, and if it is declared as `None()` then it should be used as `None()` not `None`. Update explorer to match.
Also clean up the handling of choice types and alternative values a little in general, by moving away from identifying choice types and alternatives as strings and towards identifying them symbolically.
Closes#2422
Support for global variables is still missing; they're a bit more tricky because they use a pattern to introduce their name.
Prior to this change, explorer heavily relied on name comparisons to determine whether two declarations declare the same entity. Some of those instances are fixed in this PR, but more remain to be fixed, and some TODOs are added for some harder-to-fix instances.
Name lookup into namespaces needs to be resolved early, as part of name resolution, so that we can properly diagnose references to entities before they are fully declared. Make name resolution set a target `value_node` on simple member accesses that name namespace members, and in type-checking rewrite those member accesses into `IdentifierExpression`s that directly reference the namespace member.
Jon Ross-Perkins
Richard Smith
Prabhat SachdevaandJon Ross-Perkins
josh11b
Adrien Leravat
kshokhin
Manmeet Singh
