With the toolchain splitting namespaces, ostream.h's `operator<<`
templates aren't reliably found with name lookup, likely due to the loss
of associated namespaces (zygoloid commented on this at
https://github.com/carbon-language/carbon-lang/pull/3161#discussion_r1307941999).
This is especially a barrier to moving the lex files into `Carbon::Lex`;
versus other parts of the toolchain, they contain more printable types
which are used cross-namespace, including `Carbon::Testing`. As a
consequence, I'm looking at migrating ostream.h to a more reliable
approach that doesn't rely as much on everything being in the `Carbon`
namespace.
#2569 added PrintAsID to //common/ostream.h, but given it's
explorer-specific behavior, I don't think it's the right home for it.
Noticed this while pondering better ostream interfaces.
Continuing on #3070.
I moved ParseTree::Node to just Parse::Node, versus Parse::Tree::Node.
Other name changes are just removing "Parse" or "Parser" prefixes.
In EnumBase, I'm directly defining operator<< because the ostream.h
approach just isn't working, not for either of Parse::State nor
Parse::NodeKind. Errors look like:
```toolchain/parser/parser_context.cpp:449:34: error: invalid operands to binary expression ('llvm::raw_ostream' and 'const Carbon::Parse::State')
output << "\t" << i << ".\t" << entry.state;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~
```
The expected template in `Carbon::` is not in the error list; I only see
the:
```
./common/ostream.h:112:6: note: candidate template ignored: requirement 'std::is_base_of_v<std::ostream, llvm::raw_ostream>' was not satisfied [with S = llvm::raw_ostream, T = Carbon::Parse::State]
auto operator<<(S& standard_out, const T& value) -> S& {
^
```
I'm still prodding at this, but not seeing an obvious fix.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The different approach to Names avoids the issues with trying to define
a static member (or also member function) of the templated instance of
Carbon::Internal::EnumBase from a non-enclosing namespace such as
Carbon::SemIR.
Note, I'm trying to do this from the cpp file. An alternative might be
to do `inline constexpr llvm::StringLiteral Names[]` in the .h file, but
I think concerns had been raised about that needing deduplication.
Splits IR files into SemIR, and logic files into Check. These will be
split into separate directories as part of a later move; the namespaces
are being done first in order to vet the switch, and hopefully make
conflicts a little easier to manage due to the substantial renames.
A lot of this is just automated removal of Semantics prefixes from
names, adding namespace references where needed. A few special-cases
are:
- SemanticsIR -> SemIR::File
- A few things were discussed, like Unit, CompileUnit, or CompiledUnit.
Unit was too vague for chandlerc, and I thought CompileUnit might lead
to incorrect inferences (CompilationUnit would be more precise, but
typically written as SemIR::CompilationUnit which is pretty long). File
seemed to be a short name that we could agree on.
- SemanticsIRFormatter -> SemIR::Formatter
- FormatSemanticsIR -> SemIR::FormatFile
- SemanticsFileTest -> CheckFileTest
- It remains in the Testing namespace, where just "FileTest" might be
too broad a name.
- SemanticsDeclarationNameStack::Context ->
Check::DeclarationNameStack::NameContext
- This avoids a Check::Context name shadowing.
Changes check_internal.h to include ostream.h to improve finding of
Print/operator<< (otherwise it didn't compile).
This is part of #3070
The main motivation for this is to get python loads in using the
`native-py` lint fix. However, enabling that made me wonder, maybe we
should fix in general?
`native-cc` is delayed, but not wholly cancelled (and `native-py`
picking up might indicate `native-cc` won't be too far behind). There's
also some automated fixes for `.append` and dict sorting -- this felt
okay to me, maybe not something to eagerly add but probably not worth
stopping buildifier from fixing (I've noticed the warnings in the past
and had been ignoring them).
Running everything does mean that load orders are sorted automatically
now, which I think is a positive. Most generally, I think these fixes
aren't _harmful_, and having them done automatically seems beneficial:
my biggest concern about `native-py` and `native-cc` was actually that
regressions wouldn't be caught, but this addresses that issue
automatically.
This library is designed around supporting the kinds of use cases we
expect in the toolchain and other Carbon tools. It supports subcommands,
options, and prints help.
There is still a decent chunk of work to be done to finish polishing
this, but it should give us a solid starting point.
For details about the library and a brief roadmap, see the main comment
in `command_line.h` which provides a comprehensive overview of the
library and a roadmap of the remaining work.
Porting the driver to this went fairly well, but did require some
changes. That port is separated into a follow-up PR #2979.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Lucile Rose Nihlen <luci.the.rose@gmail.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Putting fuzzer files under //testing to emphasize the testonly aspect
(consolidates bazel and common subdirectories). The attributes on
explorer_fuzzer are also a little skewed from what's desirable; it's
been working okay, but this should still be a refinement.
I think there's more we can do here, but this seemed like a good
checkpoint to make sure the path I'm going down is roughly what you
expected. There's one actual edit in if expression structure to match
the increased enforcement.
Replacing direct raw_string_ostream uses. I figure the wrapper should be used more consistently.
There are still remaining raw_string_ostream uses that weren't compatible -- I'm continuing to look at those, but felt it was cleaner to have this on its own.
This is just a cleanup. Since we now have a testing directory, I think this is a better home for testonly libraries than //common. (I was thinking about this when I was considering adding more test_raw_ostream deps)
This is a convenient test helper for anything that can use injected
streams. Extract this to where it can be used by other tests and add
some basic tests, mostly documenting how it works.
protobuf deps have become complexly structured, but AFAICT this is still the preferred dep. Also fixes some the fuzzer include checking, but that's a trivial misconfig.
I'm partly doing this because the current setup would be difficult to share with the toolchain. e.g., ProtoToCarbon isn't explorer-specific, but the only way to run it via CLI is the explorer's fuzzverter. I want a separate tool.
This change:
- Adds a //common/fuzzing:proto_to_carbon tool.
- The rest of fuzzverter is now just //explorer/fuzzing:ast_to_proto.
- The change simplifies overall handling and removes a LLVM CLI dependency.
- Stops allowing unknown fields in the proto.
- This has mostly led to forgetting to remove fuzzer inputs that were for removed features.
- Moves more non-explorer-specific bits to //common/fuzzing.
- Cleans up remaining pieces in //explorer/fuzzing
- Merges the //explorer/fuzzing proto tests, which deduplicates AstToString copies.
- These tests also had duplicate dependencies, etc -- and all complete in ~6s.
- Updates and fixes regen_corpus which was previously broken by other changes.
- Updates the README to reflect changes.
- Removes obsolete proto-fuzzer build configuration (AFAICT this is no longer needed).
Removes `__continuation`, `__await`, and `__run`.
In part here, the discussion was that while the feature had been useful for validating the early explorer design, it's no longer needed for that role as the explorer is now quite robust. Continuations have been experimental and, at this point, don't have an owner pushing to a proposal.
The triggering factor is that, as we push to address fuzzer issues, I ran into a crash bug in this code; basically, `fn Main() -> i32 { __await; return 0; }`. When I mentioned this, the reaction seemed to trend towards removal of the feature.
This keeps all of our C++ code written in C++ source files, and avoids the increasing size of the RTTI generation script we'd started to see in #2699.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This PR is making two main changes to the Explorer and Toolchain:
- Replace the `is` keyword in `where SomeType is SomeInterface` with `impls`, so it is `where SomeType impls SomeInterface`
- Rewrite uses of the "impls" to something else to avoid, frequently "`impl` declarations" or "implementations", to avoid confusion with the `impls` keyword.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
The strategy that we use for now to support template instantiation is to check the impl declaration as if it were a generic, but to defer all checking of the impl definition until we see a use in which all template parameters have arguments. At that point, we clone the impl definition and type-check the whole thing, with constant values set on the template parameters corresponding to the given arguments.
No caching of template instantiations is performed yet; each time we form a reference to a template instantiation, we instantiate it afresh. We also don't implement the name lookup rule from #949 yet; lookups during template instantiation look only in the actual type and not in the constraint.
Depends on #2699
This is intended to be used for template instantiation, but for now takes no stance as to what it's cloning.
This is tested by parsing and cloning all of our test files, and checking that the result of converting each AST to proto is the same as the result of cloning and then converting each AST to proto.
This is in preparation for template instantiation being triggered during substitution, and being able to fail.
Fix rule-of-three violation (missing assignment operator) in `Error` that got in the way of using it to hold an error temporarily in a failed transformation.
This was bugging me after I saw all the strings; it feels like this is why we have EnumBase on the toolchain side.
I've included the move of EnumBase to //common because I figured it's reasonable to evaluate together; if we don't want EnumBase in this case, it doesn't make sense to move.
Just opening files in vscode and seeing what clangd flags.
Some edits to ostream.h to stop it from getting flagged (the usage pattern means it's not always obviously used).
The design of choice types expects the declaration of an alternative to match the usage: if an alternative is declared as `None`, then it should be used as `None` not `None()`, and if it is declared as `None()` then it should be used as `None()` not `None`. Update explorer to match.
Also clean up the handling of choice types and alternative values a little in general, by moving away from identifying choice types and alternatives as strings and towards identifying them symbolically.
Closes#2422
Support for global variables is still missing; they're a bit more tricky because they use a pattern to introduce their name.
Prior to this change, explorer heavily relied on name comparisons to determine whether two declarations declare the same entity. Some of those instances are fixed in this PR, but more remain to be fixed, and some TODOs are added for some harder-to-fix instances.
First step towards permitting declarations within namespaces. Supports only functions within namespaces for now, with no way to call those functions except from within other such functions. Unqualified lookups within a function in a namespace look in that namespace first.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Add support for user-defined assignment, as well as compound assignment and increment, following the design direction in pending proposal #2511.
Some of this isn't fully testable yet: because explorer doesn't properly support `impl` specialization, the blanket `impl`s in the prelude prevent types from customizing assignment.
Add basic support for `match_first` declarations to explorer, as described in https://github.com/carbon-language/carbon-lang/blob/trunk/docs/design/generics/details.md#prioritization-rule. The concrete syntax here is not yet approved, so `match_first` is renamed to `__match_first` for now, but the semantics are necessary to implement other approved features in explorer so we're intentionally getting a little ahead of the design here.
Also make minor updates to the skeletal design in
docs/design/name_lookup.md following #2113, as there are no longer any prelude names that are made available to unqualified name lookup by default.
Add `type` to the keyword list in
docs/design/lexical_conventions/words.md, following #2360.
At present, CHECK/FATAL print their own stack trace. This switches to just using std::abort for the stack trace, as well as the CHECK printing more completely.
This has a few consequences:
1) I'm now buffering the FATAL strings in order to print it later.
2) We now print the bug report message and program arguments on failure. This is part of pretty printing and was elided before.
3) We can now have pretty printing on FATAL, e.g. to show the stacks we're building in the parser.
Per recent discussion, in `... where .A = B`, require that `B` is implicitly convertible to the type of `A` immediately, rather than treating that as part of the criteria that a type must satisfy to satisfy the resulting constraint. Extend the implementation of implicit conversion so that conversion of a type to a constraint checks that the type satisfies the constraint.
As part of implementing this, stop duplicating rewrite constraints as equality constraints. Instead, when checking that a constraint is satisfied, check both its equality constraints and its rewrite constraints. This fixes an infinite recursion that would otherwise be caused by this change, and is also a necessary prerequisite for applying rewrite constraints to equality constraints, where we would otherwise collapse the implied equality constraints to a tautological `V == V` constraint.
In passing, make ErrorBuilder support building the error message more incrementally and use that to improve diagnostics for mismatched values with equality constraints.
Note, not trying to address every last error, just some obvious/easy ones.
```
/usr/local/google/home/jperkins/dev/carbon-lang/common/string_helpers.cpp:200:13: warning: prefer transparent functors 'less_equal<>' [modernize-use-transparent-functors]
auto le = std::less_equal<const char*>();
^
/usr/local/google/home/jperkins/dev/carbon-lang/toolchain/semantics/nodes/function.h:22:45: warning: pass by value and use std::move [modernize-pass-by-value]
Function(ParseTree::Node node, NodeId id, llvm::SmallVector<NodeRef> body)
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/ast/declaration.cpp:230:14: warning: static member accessed through instance [readability-static-accessed-through-instance]
return cast<SelfDeclaration>(declaration).name();
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1104:33: warning: std::move of the variable 'impl' of the trivially-copyable type 'ConstraintType::ImplConstraint' has no effect [performance-move-const-arg]
impl_constraints_.push_back(std::move(impl));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1145:36: warning: std::move of the variable 'rewrite' of the trivially-copyable type 'ConstraintType::RewriteConstraint' has no effect [performance-move-const-arg]
rewrite_constraints_.push_back(std::move(rewrite));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/interpreter/type_checker.cpp:1156:32: warning: std::move of the variable 'context' of the trivially-copyable type 'ConstraintType::LookupContext' has no effect [performance-move-const-arg]
lookup_contexts_.push_back(std::move(context));
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/fuzzverter.cpp:74:42: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
/*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/proto_to_carbon_test.cpp:34:55: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
const ErrorOr<AST> ast = Carbon::Parse(&arena, f, /*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/proto_to_carbon_test.cpp:42:41: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
&arena, f, source_from_proto, /*trace=*/false);
^
./explorer/syntax/parse.h:25:59: note: 'parser_debug' declared here
std::string_view file_contents, bool parser_debug)
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/syntax/parse_test.cpp:27:60: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
ParseFromString(&arena, "file.carbon", FileContents, /*trace=*/false);
^
./explorer/syntax/parse.h:25:59: note: 'parser_debug' declared here
std::string_view file_contents, bool parser_debug)
^
/usr/local/google/home/jperkins/dev/carbon-lang/migrate_cpp/cpp_refactoring/var_decl.cpp:57:58: warning: string concatenation results in allocation of unnecessary temporary strings; consider using 'operator+=' or 'string::append()' instead [performance-inefficient-string-concatenation]
segments.push_back({type_loc_class, qual_str + " " + range_str});
^
/usr/local/google/home/jperkins/dev/carbon-lang/explorer/fuzzing/ast_to_proto_test.cpp:105:55: warning: argument name 'trace' in comment does not match parameter name 'parser_debug' [bugprone-argument-comment]
const ErrorOr<AST> ast = Carbon::Parse(&arena, f, /*trace=*/false);
^
./explorer/syntax/parse.h:19:17: note: 'parser_debug' declared here
bool parser_debug) -> ErrorOr<Carbon::AST>;
^
```
This does some more work to the run_clang_tidy.py wrapper script, and runs an example pass.
"again" because it's really the proto fuzzer changes that broke it, it had been working before.
"mostly" because there's still an issue within the proto fuzzer that it can't find "port/protobuf.h", i.e. https://github.com/google/libprotobuf-mutator/tree/master/port, but I'm still hesitant to add an include path there.
Add checking that a type only satisfies a constraint if it satisfies all of that constraint's equality and rewrite constraints.
Enforce the rule that `=` must be used in impls when specifying associated constant values rather than `==`.
Turn off the pre-#2173 single-step equality behavior. This is getting somewhat ahead of the approved design, but it's a one-line change to restore the old behavior.
Fix `CARBON_CHECK` to handle top-level `,`s in its argument, such as may happen in template argument lists, as this change introduces such a check.
Implements basic support for rewrite constraints as proposed in #2173.
Support for associated constants and complex constraints in general is also made more robust: argument deduction now properly computes and substitutes witnesses, and interface declarations track a more correct description of the constraint that they introduce than the one we previously built.
1) toolchain and explorer use the same working dir logic, share it
2) explorer's carbon.cpp and main_bin.cpp use the same relative path logic, share it
3) You can append a longer path in one call with the right kind of iterator
4) Fix what's maybe a bug in passing `relative_prelude_path.str()` to `cl::init`
5) Collapse Main and ExplorerMain to avoid passing more parameters between
Jon Ross-Perkins
josh11b
CanftIn
kshokhin
Priyananda Shenoy