This library is designed around supporting the kinds of use cases we
expect in the toolchain and other Carbon tools. It supports subcommands,
options, and prints help.
There is still a decent chunk of work to be done to finish polishing
this, but it should give us a solid starting point.
For details about the library and a brief roadmap, see the main comment
in `command_line.h` which provides a comprehensive overview of the
library and a roadmap of the remaining work.
Porting the driver to this went fairly well, but did require some
changes. That port is separated into a follow-up PR #2979.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Lucile Rose Nihlen <luci.the.rose@gmail.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Putting fuzzer files under //testing to emphasize the testonly aspect
(consolidates bazel and common subdirectories). The attributes on
explorer_fuzzer are also a little skewed from what's desirable; it's
been working okay, but this should still be a refinement.
I think there's more we can do here, but this seemed like a good
checkpoint to make sure the path I'm going down is roughly what you
expected. There's one actual edit in if expression structure to match
the increased enforcement.
Replacing direct raw_string_ostream uses. I figure the wrapper should be used more consistently.
There are still remaining raw_string_ostream uses that weren't compatible -- I'm continuing to look at those, but felt it was cleaner to have this on its own.
This is just a cleanup. Since we now have a testing directory, I think this is a better home for testonly libraries than //common. (I was thinking about this when I was considering adding more test_raw_ostream deps)
This is a convenient test helper for anything that can use injected
streams. Extract this to where it can be used by other tests and add
some basic tests, mostly documenting how it works.
protobuf deps have become complexly structured, but AFAICT this is still the preferred dep. Also fixes some the fuzzer include checking, but that's a trivial misconfig.
I'm partly doing this because the current setup would be difficult to share with the toolchain. e.g., ProtoToCarbon isn't explorer-specific, but the only way to run it via CLI is the explorer's fuzzverter. I want a separate tool.
This change:
- Adds a //common/fuzzing:proto_to_carbon tool.
- The rest of fuzzverter is now just //explorer/fuzzing:ast_to_proto.
- The change simplifies overall handling and removes a LLVM CLI dependency.
- Stops allowing unknown fields in the proto.
- This has mostly led to forgetting to remove fuzzer inputs that were for removed features.
- Moves more non-explorer-specific bits to //common/fuzzing.
- Cleans up remaining pieces in //explorer/fuzzing
- Merges the //explorer/fuzzing proto tests, which deduplicates AstToString copies.
- These tests also had duplicate dependencies, etc -- and all complete in ~6s.
- Updates and fixes regen_corpus which was previously broken by other changes.
- Updates the README to reflect changes.
- Removes obsolete proto-fuzzer build configuration (AFAICT this is no longer needed).
Removes `__continuation`, `__await`, and `__run`.
In part here, the discussion was that while the feature had been useful for validating the early explorer design, it's no longer needed for that role as the explorer is now quite robust. Continuations have been experimental and, at this point, don't have an owner pushing to a proposal.
The triggering factor is that, as we push to address fuzzer issues, I ran into a crash bug in this code; basically, `fn Main() -> i32 { __await; return 0; }`. When I mentioned this, the reaction seemed to trend towards removal of the feature.
This keeps all of our C++ code written in C++ source files, and avoids the increasing size of the RTTI generation script we'd started to see in #2699.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This PR is making two main changes to the Explorer and Toolchain:
- Replace the `is` keyword in `where SomeType is SomeInterface` with `impls`, so it is `where SomeType impls SomeInterface`
- Rewrite uses of the "impls" to something else to avoid, frequently "`impl` declarations" or "implementations", to avoid confusion with the `impls` keyword.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
The strategy that we use for now to support template instantiation is to check the impl declaration as if it were a generic, but to defer all checking of the impl definition until we see a use in which all template parameters have arguments. At that point, we clone the impl definition and type-check the whole thing, with constant values set on the template parameters corresponding to the given arguments.
No caching of template instantiations is performed yet; each time we form a reference to a template instantiation, we instantiate it afresh. We also don't implement the name lookup rule from #949 yet; lookups during template instantiation look only in the actual type and not in the constraint.
Depends on #2699
This is intended to be used for template instantiation, but for now takes no stance as to what it's cloning.
This is tested by parsing and cloning all of our test files, and checking that the result of converting each AST to proto is the same as the result of cloning and then converting each AST to proto.
This is in preparation for template instantiation being triggered during substitution, and being able to fail.
Fix rule-of-three violation (missing assignment operator) in `Error` that got in the way of using it to hold an error temporarily in a failed transformation.
This was bugging me after I saw all the strings; it feels like this is why we have EnumBase on the toolchain side.
I've included the move of EnumBase to //common because I figured it's reasonable to evaluate together; if we don't want EnumBase in this case, it doesn't make sense to move.
Just opening files in vscode and seeing what clangd flags.
Some edits to ostream.h to stop it from getting flagged (the usage pattern means it's not always obviously used).
The design of choice types expects the declaration of an alternative to match the usage: if an alternative is declared as `None`, then it should be used as `None` not `None()`, and if it is declared as `None()` then it should be used as `None()` not `None`. Update explorer to match.
Also clean up the handling of choice types and alternative values a little in general, by moving away from identifying choice types and alternatives as strings and towards identifying them symbolically.
Closes#2422
Support for global variables is still missing; they're a bit more tricky because they use a pattern to introduce their name.
Prior to this change, explorer heavily relied on name comparisons to determine whether two declarations declare the same entity. Some of those instances are fixed in this PR, but more remain to be fixed, and some TODOs are added for some harder-to-fix instances.
First step towards permitting declarations within namespaces. Supports only functions within namespaces for now, with no way to call those functions except from within other such functions. Unqualified lookups within a function in a namespace look in that namespace first.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Add support for user-defined assignment, as well as compound assignment and increment, following the design direction in pending proposal #2511.
Some of this isn't fully testable yet: because explorer doesn't properly support `impl` specialization, the blanket `impl`s in the prelude prevent types from customizing assignment.
Add basic support for `match_first` declarations to explorer, as described in https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/generics/details.md#prioritization-rule. The concrete syntax here is not yet approved, so `match_first` is renamed to `__match_first` for now, but the semantics are necessary to implement other approved features in explorer so we're intentionally getting a little ahead of the design here.
Also make minor updates to the skeletal design in
docs/design/name_lookup.md following #2113, as there are no longer any prelude names that are made available to unqualified name lookup by default.
Add `type` to the keyword list in
docs/design/lexical_conventions/words.md, following #2360.
At present, CHECK/FATAL print their own stack trace. This switches to just using std::abort for the stack trace, as well as the CHECK printing more completely.
This has a few consequences:
1) I'm now buffering the FATAL strings in order to print it later.
2) We now print the bug report message and program arguments on failure. This is part of pretty printing and was elided before.
3) We can now have pretty printing on FATAL, e.g. to show the stacks we're building in the parser.
Per recent discussion, in `... where .A = B`, require that `B` is implicitly convertible to the type of `A` immediately, rather than treating that as part of the criteria that a type must satisfy to satisfy the resulting constraint. Extend the implementation of implicit conversion so that conversion of a type to a constraint checks that the type satisfies the constraint.
As part of implementing this, stop duplicating rewrite constraints as equality constraints. Instead, when checking that a constraint is satisfied, check both its equality constraints and its rewrite constraints. This fixes an infinite recursion that would otherwise be caused by this change, and is also a necessary prerequisite for applying rewrite constraints to equality constraints, where we would otherwise collapse the implied equality constraints to a tautological `V == V` constraint.
In passing, make ErrorBuilder support building the error message more incrementally and use that to improve diagnostics for mismatched values with equality constraints.
Note, not trying to address every last error, just some obvious/easy ones.
```
/usr/local/google/home/jperkins/dev/carbon-lang/common/string_helpers.cpp:200:13: warning: prefer transparent functors 'less_equal<>' [modernize-use-transparent-functors]
auto le = std::less_equal<const char*>();
^
/usr/local/google/home/jperkins/dev/carbon-lang/toolchain/semantics/nodes/function.h:22:45: warning: pass by value and use std::move [modernize-pass-by-value]
Function(ParseTree::Node node, NodeId id, llvm::SmallVector<NodeRef> body)
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/ast/declaration.cpp:230:14: warning: static member accessed through instance [readability-static-accessed-through-instance]
return cast<SelfDeclaration>(declaration).name();
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1104:33: warning: std::move of the variable 'impl' of the trivially-copyable type 'ConstraintType::ImplConstraint' has no effect [performance-move-const-arg]
impl_constraints_.push_back(std::move(impl));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1145:36: warning: std::move of the variable 'rewrite' of the trivially-copyable type 'ConstraintType::RewriteConstraint' has no effect [performance-move-const-arg]
rewrite_constraints_.push_back(std::move(rewrite));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1156:32: warning: std::move of the variable 'context' of the trivially-copyable type 'ConstraintType::LookupContext' has no effect [performance-move-const-arg]
lookup_contexts_.push_back(std::move(context));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/fuzzverter.cpp:74:42: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
/*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/proto_to_carbon_test.cpp:34:55: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
const ErrorOr<AST> ast = Carbon::Parse(&arena, f, /*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/proto_to_carbon_test.cpp:42:41: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
&arena, f, source_from_proto, /*trace=*/false);
^
./explorer/syntax/parse.h:25:59: note: 'parser_debug' declared here
std::string_view file_contents, bool parser_debug)
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/syntax/parse_test.cpp:27:60: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
ParseFromString(&arena, "file.carbon", FileContents, /*trace=*/false);
^
./explorer/syntax/parse.h:25:59: note: 'parser_debug' declared here
std::string_view file_contents, bool parser_debug)
^
/usr/local/google/home/jperkins/dev/carbon-lang/migrate_cpp/cpp_refactoring/var_decl.cpp:57:58: warning: string concatenation results in allocation of unnecessary temporary strings; consider using 'operator+=' or 'string::append()' instead [performance-inefficient-string-concatenation]
segments.push_back({type_loc_class, qual_str + " " + range_str});
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/ast_to_proto_test.cpp:105:55: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
const ErrorOr<AST> ast = Carbon::Parse(&arena, f, /*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
```
This does some more work to the run_clang_tidy.py wrapper script, and runs an example pass.
"again" because it's really the proto fuzzer changes that broke it, it had been working before.
"mostly" because there's still an issue within the proto fuzzer that it can't find "port/protobuf.h", i.e. https://github.com/google/libprotobuf-mutator/tree/master/port, but I'm still hesitant to add an include path there.
Add checking that a type only satisfies a constraint if it satisfies all of that constraint's equality and rewrite constraints.
Enforce the rule that `=` must be used in impls when specifying associated constant values rather than `==`.
Turn off the pre-#2173 single-step equality behavior. This is getting somewhat ahead of the approved design, but it's a one-line change to restore the old behavior.
Fix `CARBON_CHECK` to handle top-level `,`s in its argument, such as may happen in template argument lists, as this change introduces such a check.
Implements basic support for rewrite constraints as proposed in #2173.
Support for associated constants and complex constraints in general is also made more robust: argument deduction now properly computes and substitutes witnesses, and interface declarations track a more correct description of the constraint that they introduce than the one we previously built.
1) toolchain and explorer use the same working dir logic, share it
2) explorer's carbon.cpp and main_bin.cpp use the same relative path logic, share it
3) You can append a longer path in one call with the right kind of iterator
4) Fix what's maybe a bug in passing `relative_prelude_path.str()` to `cl::init`
5) Collapse Main and ExplorerMain to avoid passing more parameters between
* Multiplication and division have the same priority.
* A new builtin interface DivWith is added.
* In some tests expecting a compilation error (syntax error), the error
message now says it is expecting SLASH or binary *.
* Replay changes from pre-force-push mixin branch
* Base MixinPseudoType off of the new InterfaceType
* Add more test cases.
Also removed an unnecessary check that would have already been
handled by the parser.
* WIP detect member clashes during mixing mixins
* Implement fuzzer changes
* Implement member name clash check when mixing mixins
* Modify parser and lexer for experimental mixin feature
* Add comments
* Update explorer/testdata/mixin/simple-mix-in-mixin.carbon
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
* Update explorer/testdata/mixin/use-mixin-method-in-class-method.carbon
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
* Make code review changes
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
josh11b
CanftIn
kshokhin
Priyananda Shenoy
pmqtt
Junhee Cho
micttyl
Darshal Shettyandjosh11b
Vineeth B V