Right now //explorer/fuzzing:explorer_fuzzer takes my machine 80s to run, just because of the corpus size. The corpus is actually pretty small compared to the toolchain fuzzer, so reducing the corpus size doesn't feel quite right.
This adds support for sharding fuzz tests, and with 8 shards each is closer to 10s. This should put it closer to the noise of other explorer tests in terms of runtime.
Unfortunately I'm not seeing a sharding flag in the llvm library, which seems fair. However, that's why I'm working around it by creating separate test targets per shard, then a suite to merge them back together. The use of `shard_count` for this is idiomatic for bazel rules; I'm using it so that switching implementations should be low-impact if that's ever needed.
Co-authored-by: Adrien Leravat <Pixep@users.noreply.github.com>
These instructions have fallen out of sync with the note that we've stopped recommending brew for linux (due to build issues). Rather than having instructions that only work for MacOS, take the bare minimum and just point at contribution_tools.md.
chandlerc, I believe you'd specifically requested the instructions here as part of #1390. An alternative approach would be to remove this entirely, and just say "See our contribution tools page for instructions on how to build locally."
i-khadra's change #2636 is what made me notice this issue. They're trying to add Windows notes to the README, but this change reflects my leaning it's better to let contribution_tools.md explain setup.
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This is starting to build out actual lowering logic, for a really simple `fn Main() -> i32 { return 0; }`
Notes for achieving this:
- In semantics, currently function names are bound separate from the signature. When emitting IR, this turns out to be inconvenient because we want to know the name when we process the declaration and the definition. This change addresses that by merging the name into the FunctionDeclaration node, which is also accessible from the definition. It removes the separate BindName. This should be the cause of all the test changes in semantics, because the IR generated changes.
- Add a "Lowering" class which I'm using to hold the llvm builder state. This class now has minimal support for the SemanticsIR generated by the above example.
- In the "Lowering", values from expressions are stored in a DenseMap. I'll keep thinking about whether there's a cleaner way to achieve this, and I'd call it a temporary solution for now. However, this is how the `0` in `return 0` gets properly associated across SemanticsIR instructions, and it'll frequently be an issue in less trivial cases.
The toolchain misinterprets escaped whitespace characters as unescaped when trimming trailing whitespace on a line. This PR adds a variable to track the length of the string after escaping the most recent escape, making sure we don't backup past that character, whatever it is.
I did try the approach mentioned in #2132 of not backing up `end_of_regular_text` number of characters, but this caused problems on lines like <kbd>tab</kbd> <kbd>space</kbd> <kbd>tab</kbd> (those characters literally, not escaped) where it would leave the first tab since that is processed in a different iteration of that loop.
I added a test case for this bug. I kept getting mixed up in the test output which one was the expected value, so I also changed a variable name there for clarity.
Fixes#2132.
Adjusts handling of class constants (`static const`) to use CamelCase. This probably better reflects how we use it in C++ code, treating as appropriate for CamelCase instead of under_score.
Fixes adding_children to be preorder in caller (not sure why this wasn't automated).
No automated changes.
Just opening files in vscode and seeing what clangd flags.
Some edits to ostream.h to stop it from getting flagged (the usage pattern means it's not always obviously used).
This fixes some bugs in each, where the fixes had only been made on one
side of the switch or the other. Also don't forget to instantiate
deduced generic arguments in a call when we read them out of the AST.
This is intended to address currently flaky timeouts that are likely caused by the size of the prelude. I'm addressing a performance bottleneck in AnalyzeProgram with trace output. Trying to omit prelude traces reduces most trace output significantly, and I think it'll scale better as the prelude size increases.
The basic mechanics here are:
- In order to consistently track whether tracing is on, I've added a TraceStream class, explorer/interpreter/trace_stream.h.
- The AST now has a num_prelude_declarations field, so that it's provided where the boundary is.
- In order to mark where we try to skip prelude output, I've added calls to set_in_prelude in type_checker.
- In exec_program, I just use num_prelude_declarations directly to skip over.
- Everywhere checks TraceStream::is_enabled before printing, similar to the std::optional check that was previously used.
This does add some timing output in order to better diagnose where slowness is coming from, when tracing. It also adds "verbose" targets to make it easier to get the trace output.
So for example, here's a timing for zero.carbon:
```
Timings:
- Parse: 13ms
- AddPrelude: 25ms
- AnalyzeProgram: 116ms
- ExecProgram: 12ms
```
If I make a small change to just not set skipping_prelude (essentially getting back to current output):
```
- Parse: 13ms
- AddPrelude: 25ms
- AnalyzeProgram: 2359ms
- ExecProgram: 57ms
```
Thus in this trivial example, I'm eliminating about 95% of the execution time.
Note this approach could still be refined in a few ways:
- We could add a flag to allow overriding in_prelude. It should be a small amount of work after this change. But it's a little consistent with how parser_debug works, that it won't print prelude output by default (unless there's an error).
- Execution could skip messages involving initialization of globals declared in the prelude. This is a little noisy right now, but I don't think it's significant for performance because ExecProgram is tiny.
- Once files are more separated, we should be able to change the num_prelude_declarations/set_in_prelude approach.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This starts handling return types on functions, and comparing types with `return` statements.
Note, errors remain poor because the type literal is currently associated with a builtin, losing the parse_node that specified it. This means we don't have the original source location to associate with, even though it may be helpful to point at the type in source. We could point at the signature overall, but my leaning is that we wouldn't want that long-term, so TODOs for now and may want to change a little about how the parse node is tracked once things are a little further along.
The intent here is that changes to prelude.carbon shouldn't break every test that expects some error from prelude.carbon; that would be too fragile. As a consequence, this effectively ignores the line number in prelude.carbon.
This is a little complex because we don't know which line in the original source file is actually causing the error, just that there is an error. Also, the previous look-behind approach required a fixed-with prefix, whereas we want a little more than that in order to capture the filename for comparison.
This would be hard to do with extra_check_replacement because the path to bazel.runfiles is complex to calculate. As a consequence, this is basically all new code.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Both cases need a label for each level of struct and tuple type appearing in the impl query, because we can have an unbounded number of such levels with the types otherwise being the same.
Prior to this, both added testcases exhibit unbounded recursion.
Detect when evaluating an impl recursively tries to evaluate the same impl for the same or a more complex set of parameters.
In order to perform the check after we have tested that the type structure matches and before we check that constraints are recursively satisfied, argument deduction is extended to check structural matching properties earlier.
This requires us to separate match failures into two kinds: hard failures that produce errors that should never be swallowed, and soft failures such as a missing impl that lead us to merely discard an impl as a candidate. A flag has been added to `ImplScope` and `ArgumentDeduction` to specify whether soft failures should produce an error message or not.
The design of choice types expects the declaration of an alternative to match the usage: if an alternative is declared as `None`, then it should be used as `None` not `None()`, and if it is declared as `None()` then it should be used as `None()` not `None`. Update explorer to match.
Also clean up the handling of choice types and alternative values a little in general, by moving away from identifying choice types and alternatives as strings and towards identifying them symbolically.
Closes#2422
This adds tracking of call information plus basic type checking. It adds a builtin for the empty tuple, mainly so that I have the basis for a default function return type.
As an aside, it also unifies printing within SemanticsIR, fixing a missing comma after callables.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Reals were mostly handled, but this PR adds storage of them. It also switches a little towards the FloatingPointType semantic from TokenizedBuffer.
While real literals like `1.0` were handled, the type literals were not. This just adds `f64`, similar to how I also only support `i32`.
The String type literal wasn't used, so I've added support in lexer and parser. Per discussion with @zygoloid String might be renamed based on the newer type literal plan, but it's still String in explorer and the design, so this is just consistent.
The builtin_types.carbon tests the three basic types that are there right now. The test is added to both parser and semantics so that it's clear what the state is in both stages.
Return Integer overflow Runtime error for bit shift if The second operand is less than zero or bigger than or equal bit width of the first operand in this case (32 bit for int)
Issue #2595
There were some notes about default constructors being required in parse_tree.h, but they don't seem to be. Removing the default constructor forces the explicit `::Invalid` where there's no value immediately being assigned, which works fine for existing code. I think this actually reduces the chance of accidents more than the prior default-construct-as-invalid approach.
@chandlerc requested that leads questions not be marked inactive. These arise when there *isn't* a clear-cut answer, and so seem to make more sense to be effectively long-term.
The \n\n\n is trying to get a blank line between paragraphs. Right now it gets a newline, but the paragraphs sort of blend. I might need more \n's, don't know but I'll keep an eye on messages. The examples that I could find don't have multiple paragraphs.
It feels like every time I add a builtin, I forget to add it to the builtin IR. This approach enforces the addition of builtin nodes, and also takes advantage of the normalized structure to set types appropriately (I think the new approach is simpler on this aspect).
Support for global variables is still missing; they're a bit more tricky because they use a pattern to introduce their name.
Prior to this change, explorer heavily relied on name comparisons to determine whether two declarations declare the same entity. Some of those instances are fixed in this PR, but more remain to be fixed, and some TODOs are added for some harder-to-fix instances.
Using the same const/constexpr done in EnumBase, adds Invalid and Builtin* values to replace Make functions that produced the same. This should make it clearer at call sites what the cost actually is, and reduces the syntactic overhead for MakeBuiltinReference in particular.
Really, this is that MakeBuiltinReference has been feeling pretty verbose, so I did that, and then one MakeInvalid is right next to it, and then obviously I should replace the other MakeInvalid for consistency...
Name lookup into namespaces needs to be resolved early, as part of name resolution, so that we can properly diagnose references to entities before they are fully declared. Make name resolution set a target `value_node` on simple member accesses that name namespace members, and in type-checking rewrite those member accesses into `IdentifierExpression`s that directly reference the namespace member.
pre-commit autoupdate --freeze && pre-commit run -a
This update may be triggering incompatibilities with old installs. If you see an issue such as:
```
An unexpected error has occurred: CalledProcessError: command: ('python', '-mpip', 'install', '.')
```
Try updating pip packages, particularly:
```
pip3 install -U pre-commit
pip3 install -U virtualenv
```
First step towards permitting declarations within namespaces. Supports only functions within namespaces for now, with no way to call those functions except from within other such functions. Unqualified lookups within a function in a namespace look in that namespace first.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Split the `where A == B` constraint in two: `where .A = B` produces a new
constraint from an old one, where the value of `.A` in the new constraint is
known and eagerly rewritten to `B`, and `where A == B`, which does not cause
`A` and `B` to be considered as identical by language rules but does permit
implicit (no-op) conversion between them.
This aims to provide an efficiently-computable and human-understandable type
equality rule, with type canonicalization and therefore transitive type
equality, without sacrificing too much in the way of ergonomics and without
sacrificing determinism, while still providing the full power of a general type
constraint system in a less ergonomic form.
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
Jon Ross-PerkinsandAdrien Leravat
Calvin
Avi Aaron
josh11b
Richard Smith
Amr Hesham