With the toolchain splitting namespaces, ostream.h's `operator<<`
templates aren't reliably found with name lookup, likely due to the loss
of associated namespaces (zygoloid commented on this at
https://github.com/carbon-language/carbon-lang/pull/3161#discussion_r1307941999).
This is especially a barrier to moving the lex files into `Carbon::Lex`;
versus other parts of the toolchain, they contain more printable types
which are used cross-namespace, including `Carbon::Testing`. As a
consequence, I'm looking at migrating ostream.h to a more reliable
approach that doesn't rely as much on everything being in the `Carbon`
namespace.
Refactors `CallDestructor` and `CallFunction` to both call a new method
`BindSelfIfPresent`, which includes support for binding `addr self` if
specified. Fixes the associated unit test.
Closes#2802.
#2569 added PrintAsID to //common/ostream.h, but given it's
explorer-specific behavior, I don't think it's the right home for it.
Noticed this while pondering better ostream interfaces.
This PR removes `PrintDepth` from statement and declaration.
Implements `PrintIndent` for better indented formatting along with
various changes to make printing of statements and declarations better.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
I was initially looking at llvm::seq, but then looking through uses it
seemed like enumerate/zip would be best for these, allowing less direct
indexing and more range-based looping.
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
Makes the trace output for pattern match more consistent with the rest
of the trace by using `Match()` and `Indent()` prefix methods and
wrapping the code in the trace with backticks.
Instead of manually putting symbols before and after heading, this
introduces two methods `Heading` and `SubHeading` inside `TraceStream`.
Both methods take `llvm::StringRef` as parameter, formats the given text
as follows and adds it into the output stream.
**Heading**
```
* * * * * * * * * * heading * * * * * * * * * *
-------------------------------------------------
```
**Sub heading**
```
- - - - - sub heading - - - - -
---------------------------------
```
Note: both methods assert that tracing should be enabled.
Defines methods into `TraceStream` for adding line prefixes. Instead of
directly using string literals.
Example usage,
```
trace_stream_->Start() << "declaring ... " << ... ;
```
will result in,
```
->> declaring ...
```
Updates the trace output for type checking to be more consistent like
the rest of trace output.
This PR also includes few changes in the lit tests related to trace
output.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
See arena.h for discussion of what canonicalization means in this
context. This is primarily intended to support implementing a memo table
of template instantiations to resolve#2951, but could be useful for
other purposes as well.
Additional changes:
- Pass `VTable` constructor parameters by pointer, to avoid the need to
define `operator==` and `hash_value` for it.
- Clean up the recurring pattern of allocating identical `NamedElement`s
on the stack and heap. Instead we always allocate it on the heap and
pass it by pointer.
- Add `Print()` and `Dump()` to `Bindings` as a debugging convenience.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Associated constants were getting returned directly, crashing because
the value InterpProgram received was an AssociatedConstant instead of an
IntValue. We have a similar Convert call in ReturnVar, so I _think_ this
is the right approach.
#### Functional changes
* Avoid unnecessary copies when a value binding is created from a value
expression in call parameters
* Ensure the result of the value expression bound is destroyed
* Provide storage to initializing expressions used in call parameters
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Add state (stack and heap information) to the trace stream only when
there is a change in state, this would prevent consecutive repetition of
state information in the trace output.
Checks it based on change in size of the stack and heap, added
`std::pair<int, int> stack_heap_sizes` in `Interpreter` to store size of
stack and heap for comparison.
Adds the following information about name resolution to the trace
output:
* Name resolution process
```
** resolving stmt | decl `<stmt | decl>` (<source_location>)
...
** finished resolving stmt | decl `<stmt | decl>` (<source_location>)
```
* When a name is added to the static scope
```
--- declared `<name>` as `<entity>` in `<scope>` (<source_location>)
```
* When a name is resolved
```
--- resolved `<name>` as `<result>` in `<scope>` (<source_location>)
```
* Marking a name declared/usable
```
--- marked `<name>` <usable | declared but not usable> in `<scope>`
```
This will be more correct if a user file is named prelude.carbon, and
reduces the explorer runtime by about 10% by removing a string
comparison from the check for whether trace output is enabled.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Prevent copies when initializing value expression from reference
expression. This is based on
https://github.com/carbon-language/carbon-lang/pull/2006, which
introduces expression categories, and how it is possible to convert
to/from those different categories. Continuation of
https://github.com/carbon-language/carbon-lang/pull/2907
## Functional changes
* Initializing a value expression from a reference expression takes its
value without a copy
* Reading from the value expression causes an error if the value changed
from the time it was initialized
* In this situation, prevents a copy both for variable definitions, and
call parameter bindings
## Main implementation changes
* Add new `ExpressionCategoryAction`, which evaluates an expression and
returns an `ExpressionValue` containing its category and address (if
any), in addition to the resulting `Value*`
* `ExpressionAction`s now invokes `ExpressionCategoryAction` and unwraps
the returned `ExpressionValue`
* `RuntimeScope::BindAndPin` method, and corresponding when attempting
to read a `value_node`.
## Next work
* Avoid unnecessary copies from value expression to value expression,
after ensuring that even value expression temporaries are registered for
destruction (https://github.com/Pixep/carbon-lang/pull/9)
To filter based on file contexts, you can use `-trace_file_context=...`
flag along with `--trace_file=...` flag. These are the following options
you can pass to the `-trace_file_context=...` flag (you can also pass
them as list by separating them with comma).
1. `main`: Include trace output for file containing the main function.
2. `prelude`: Include trace output for prelude.
3. `import`: Include trace output for imports.
4. `include`: Include trace output for all.
If the flag isn't used or non of the options are passed, by default only
main file context is traced.
File contexts are distinguished based upon the source location by
passing the source location to `TraceStream::is_enabled(..)` as
arguments. If no arguments are passed, file context is
`FileContext::Unknown` and for now it will add the trace from unknown
file context.
Note: `import` option currently doesn't work as imports aren't supported
yet.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Reports an error (`value of generic binding T is not known`) instead of
performing an invalid cast when attempting to index into a tuple. Can
occur when trying to cast a generic constant in the interface position
of an implementation to `type`.
Closes#2938
This fixes a bug where Explorer would not detect when:
- an implementation used a method to implement a class function in an interface
- an implementation used a class function to implement a method in an interface
- an implementation method used `addr self` when the interface method did not
- an interface method used `addr self` when the implementation method did not
at type checking time. This would then cause a crash at runtime.
Closes#2857
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Add partial support for initializing expressions for variable declaration. This is based on https://github.com/carbon-language/carbon-lang/pull/2006, which introduces expression categories, and how it is possible to convert to/from those different categories.
## Functional changes
* Initializing expressions initialize directly the provided storage when used to initialize a variable.
* Allows initializing expressions to avoid a copy when using `[var|let] name: type = call_expression(...)` by initializing `name` in-place.
* Support `returned var: ...` and `return <expr>`
* Support nested initializing expressions
## Main implementation changes
* Updated PatternMatch logic to handle expression categories
* Updated `VariableDefinition` interpreter statement to allocate and pass a location to initializing expressions
* Update statement actions to allow passing an allocation, used by return expr or returned var
* Modified the RuntimeScope API to be one step closer to the memory model we want to have
* Remove `GetAllocationId` and older `Bind` which don't apply
* New set of tests to highlight those different situations
* Added a new intrinsic to print the allocation stack (and make sure we behave correctly, beyond visible side effects)
## Next work
* Dedicated `Action` to retrieve expression category information in the interpreter (https://github.com/carbon-language/carbon-lang/pull/2927)
* Avoid copies when initializing value expression from reference expression and prevent mutations for the duration of the "pinning" (https://github.com/carbon-language/carbon-lang/pull/2927)
* Avoid unnecessary copies from value expression to value expression, after ensuring that even value expression temporaries are registered for destruction.
* Avoid unnecessary copies when binding function arguments
The crash occurred because of an alias target of a mixin. While
eventually, we should probably have some ability to alias mixins, this
isn't yet setup in the explorer and doesn't seem like a current
priority.
We got here because the mixin type checking made it far enough to not
reject this within the type checker, but the next step wasn't prepared
for this to come out of the type checker. The simplest fix seems to be
to reflect that it *can* escape the type checker, but still isn't (yet)
a valid alias target.
Test case added.
There are a few implicit conversions that are implemented by an `impl` of `ImplicitAs` that delegates to code in explorer:
- Converting between tuple types
- Converting from tuples of types to `type`
- Converting from tuples of values to an array type
- Converting between struct types
- Converting from a struct type to a class type
These conversions can all rely on performing more conversions for elements or subobjects, but previously those inner conversions could only be performed if they were built into explorer. This change instead uses the full implicit conversion machinery in explorer to perform these conversions, including searching for a user-defined `impl` of `ImplicitAs` when necessary.
For example, this permits a conversion from `{.a: T}` to `{.a: U}`, or from `(T, T)` to `(U, U)`, or from `(T, T)` to `[U; 2]` when there is a user-defined conversion from `T` to `U`.
Depends on #2878
Within `explorer/common/trace_stream.h`, implemented RAII Type `SetProgramPhase`.
This RAII type simplifies the process by automatically setting the desired program phase upon construction and restoring the previous phase upon destruction.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
When a template has an argument that involves a generic parameter, we're supposed to delay instantiation until we know the concrete value, but explorer is not set up to do that yet, so for now we instead instantiate the template with the symbolic argument. When that happens, bring the constraints on the generic parameter into scope so they can be used inside the template instantiation.
This requires adding a new search over a value for the generic parameters that appear within it; a `VisitNestedValues` visitor is added to visit all the `Value`s nested with a value, and also convert an existing place where we were doing the same thing in a way that was incorrect (but harmlessly incorrect for now) to use it.
This is needed by #2881, which needs implementations of `ImplicitAs` for nested types when instantiating a builtin impl of `ImplicitAs` for an aggregate type.
Co-authored-by: Geoff Romer <gromer@google.com>
By implementing these improvements, users will have the ability to choose specific parts of the trace output.
Currently, when executing a file using the explorer with the --trace_file=- or --trace_file=filename.txt flag, the resulting output is an extensive and verbose log containing all the information.
In this PR, I have introduced the `ProgramPhase` enum class, which have distinct phases encountered during the compilation of a program in the explorer. Each member of this enum class corresponds to a specific phase, signifying the relevant information to be included in the trace output.
The phases covered by the `ProgramPhase` enum class are as follows:
1. Printing the source program
2. Name resolution
3. Control flow resolution
4. Type checking
5. Unformed variable resolution
6. Printing declarations
7. Printing the timings
8. Printing whole output.
These phases can be selected by passing the following compiler flags along with `--trace_file=-`.
`-trace_source_program`, `-trace_name_resolution`, `-trace_control_flow_resolution`, `-trace_type_checking`, `-trace_unformed_variables_resolution`, `-trace_declarations`, `-trace_execution`, `-trace_timing` and `-trace_all`. If none of these flags is passed only execution trace will be added to the output.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Fix misidentification of a potential cycle in the case where the inner match is missing labels from the outer match, and the inner match is strictly more complex when considering only its labels. We previously ignored labels in the outer match that are absent in the inner match, but the existence of any such label should cause us to treat the inner match as not being strictly more complex.
Per #257, we should be treating unformedness as all-or-nothing, rather than being a per-field or per-array-element property. Previously we initialized an array with no explicit initializer as containing a sequence of uninitialized values, but that led to crashes when attempting to access those values, as the checks for reading an uninitialized value only expected values to be uninitialized at the top level.
Also, we had existing tests that attempt to store to an element of an uninitialized array. We now detect that and treat it as UB during evaluation, rather than crashing due to trying to perform field access into an uninitialized value.
Finally, many of these problems can be detected statically, but the resolve_unformed pass wasn't catching them because it missed a few expression and declaration forms. Support for those cases has been added too. This causes the pass to recurse more often, and in particular our existing recursion test started hitting a stack overflow after this, so resolve_unformed now uses `RunWithExtraStack`. In passing, remove the need to explicitly tell `RunWithExtraStack` the return type, and infer it as the return type of the callable instead.
The old version wasn't compatible with ASan's instrumentation. Instead,
use a builtin when available, and even in the fallback disable inlining
and sanitizing as they can't do anything useful in this routine.
This fixes a confusing internal ASan error seen by some folks since the
stack detection was added.
There was a warning "Unchecked access to optional value" in this piece of code inside `action.cpp`
```cpp
if (this->scope().has_value()) {
out << " " << *this->scope();a
}
```
Fixed it by accessing `scope_` directly rather than using `this->scope()`.
Adds stack space handling similar to Clang's approach, but with more support for forwarding return values. Refactors ParseAndExecute for better sharing, and for centralization of the InitStackSpace handling.
Fixes#2795
Isolating test execution time (no build time included):
- Linux, `lit` test-per-file: Elapsed time: 28.214s, Critical Path: 13.97s
- Linux, `cc_test`-per-file: Elapsed time: 11.534s, Critical Path: 6.05s
- Linux, merged `cc_test` with 50 shards: Elapsed time: 11.677s, Critical Path: 11.17s
- Mac, `lit` test-per-file: Elapsed time: 295.686s, Critical Path: 20.00s
- Mac, `cc_test`-per-file: Elapsed time: 55.788s, Critical Path: 3.81s
- Mac, merged `cc_test` with 50 shards: Elapsed time: 16.269s, Critical Path: 7.54s
In GH actions:
- [Before](https://github.com/carbon-language/carbon-lang/actions/runs/4866602695/jobs/8678306144?pr=2799):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 20m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 15m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 36m
- test / test (macos-12, opt) (pull_request_target) Successful in 21m
- [After](https://github.com/carbon-language/carbon-lang/actions/runs/4875154751/jobs/8697004066?pr=2811):
- test / test (ubuntu-22.04, fastbuild) (pull_request_target) Successful in 10m
- test / test (ubuntu-22.04, opt) (pull_request_target) Successful in 9m
- test / test (macos-12, fastbuild) (pull_request_target) Successful in 12m
- test / test (macos-12, opt) (pull_request_target) Successful in 9m
I'm still leaving a handful of `lit` tests to test end-to-end binary execution. This is why testdata directories are split (`lit` tests next to the `explorer` binary, the `cc_test`s next to `ParseAndExecute`).
Currently ParseAndExecute-style logic is done in main.cpp and a test. #2799 is adding another test that needs it, as is #2811.
Also more clearly marks fuzzing as testing.
This is being extracted out of #2799 in order to try unblocking progress while review continues.
Jon Ross-Perkins
Lucile Rose Nihlen
Prabhat SachdevaandJon Ross-Perkins
kshokhin
josh11bandGeoff Romer
Adrien LeravatandJon Ross-Perkins
Chandler Carruth