This removes BindName, putting name information directly on VarStorage.
As a side-effect of updating semantics_ir_test for this change, I also
noted that function bodies were being generated as invalid YAML so am
fixing that (just `{}` to `[]` bracketing, otherwise the test wouldn't
work anymore).
Because names are now available, I've updated lowering to use them for
vars.
In the SemIR formatter, the name is now repeated because it's a
parameter to VarStorage. I believe this is just default behavior, and
we'd have to special-case VarStorage to remove it because it's automatic
argument printing in action. On the balance, it felt like letting it
print was reasonable.
I've noted in places that the name on VarStorage is expected to be
optional, but am not adding support because I'd have no way of testing
it at present.
There were two issues contributing to this crash:
- Primarily, the issue is that we queue up diagnostics and don't format
them into a string until we reach the end of compilation. In some code
paths in the driver, we destroyed the Semantics IR object before this
happened. But diagnostics can contain references to Semantics IR
objects, such as strings stored in the string table, which can lead to a
use after destruction bug.
This is fixed by ensuring the diagnotics consumer is flushed before
destroying any of the objects that it can refer to. The current approach
to this is not especially clean, unfortunately, but this requires
fighting C++ as this isn't the order in which it wants to destroy
things.
- This issue was obscured by the Semantics IR's string table holding a
reference to whatever underlying storage it was given rather than its
own string storage, so sometimes it would hold a reference to a string
from the source file, and sometimes a string from the tokenized buffer's
string table. The diagnostics were always flushed before the source file
was destroyed, but not before the tokenized buffer was destroyed. So to
see the issue, you'd need to have a string literal with certain contents
followed by an identifier with a name that matched those contents.
The crash is made more reliable by holding references to the Semantics
IR's string map in its string table, rather than references to someone
else's strings. This also fixes a latent bug where passing a string
temporary to SemanticsIR::AddString would store a dangling reference in
the string table. Incidentally, AddString is also changed to perform
only one hash table lookup rather than two for each added string.
This matches what we do for all other dump output.
No test: this is just changing the behavior of a dump mode, and is
really awkward to exercise without adding back in something like a lit
test to observe the behavior when stdout and stderr go to the same
place.
Add a language server for carbon as part of GSoC.
This currently does code outline using toolchain parser.
See development steps in utils/vscode/README.md for running and using
language server.
In #3064, code was changed to look at a future token. This is an issue
because the parser is set up to enforce that tokens aren't used without
being consumed. That's part of #3118; related validation fails. Also,
since it's not necessarily the open paren that was consumed, it could be
a different opening symbol, which the closing symbol handling doesn't
check.
Under this approach, it's tracked whether an open paren was consumed,
and the open paren is associated with the state. That's more aligned
with how the parser expects to be fed information.
In paren condition handling for if and while, I'm also adding some
special casing for `if {` in particular to not assume the `{` is a
struct. I just think that this will come up somewhat often and the
resulting output is better this way (an error either way). I'm not doing
similar with `for` because there's already some `var` handling there,
and I'd need a little more time to think about structure -- whereas
right now I'm just trying to fix the crashes (`if {}`, `if []`, etc).
Fixes#3118
Adds some notes about the required build flags for lldb to successfully
find the symbols on macOS debug builds. Also adds a recommended debugger
configuration for interactive debugging in VSCode on macOS.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
This PR implements the lowering for array element access. Besides, this
creates a function for pointerTY to avoid code duplication.
---------
Co-authored-by: Farzana Ahmed Siddique <fasiddique@google.com>
This also tries to restructure the command line interface to the
toolchain a bit to make it start operating more like a compiler that
could be integrated into a build system rather than primarily as
a testing tool.
1) This switches form a `dump` subcommand to a `compile` subcommand
which has "dump" actions that can be enabled within it.
2) A distinct set of compile _phases_ that match the toolchain
structure:
- `lex` to run the lexer
- `parse` to run the parser
- `check` to fully check that the code is valid
- `lower` to lower to LLVM's IR
- `codegen` to generate executable code
3) The codegen phase has two output formats: textual assembly and
a binary object. These outputs can be configured, with a default for
an object when writing to a file and more firm default for textual
assembly when writing to stdout.
4) Select and expose the use of the LLVM host detection to compute
a default code generation target in the driver so that the command
line interface can reflect this. For example, the `help` output will
include the default target.
5) The `//toolchain/codegen` library APIs have been restructured a bit
to make the code flow a bit more naturally when implementing the new
command line structure. No real changes to the logic though.
There are also some minor tweaks to the command line interface based on
trying to use the shortest names for things that still seem likely to be
learnable for users:
- Switched `target-triple` to just `target`: the "triple" component to
this name is historical and can be confusing. For example, almost all
"triple" strings have more than three components today.
- Switched to just `--output` as now the fact that it is a file can be
configured in the documentation -- it will render as `--output=FILE`.
This also adds support for two custom output filename modes. First, when
no output is specified, we now compute one in the conventional way for
compilers by removing the file extension of the input file and replacing
it with `.o` for an object file output or `.s` for an assembly file
output. This matches the behavior of Clang and GCC for example.
Second, output to stdout is enabled with the special output file name of
`-` since it is no longer the default. This also follows the convention
of most compilers and many other command line tools to use `-` as a file
name to signify using standard in/out pipes.
There are still some rough edges here that I suspect could be improved,
but this seems like a good start of switching over to a complete
argument parser.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Lucile Rose Nihlen <luci.the.rose@gmail.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The existing tuple index lowering was crashing while accessing the
element from return value. This PR fixes the bug.
Co-authored-by: Farzana Ahmed Siddique <fasiddique@google.com>
This library is designed around supporting the kinds of use cases we
expect in the toolchain and other Carbon tools. It supports subcommands,
options, and prints help.
There is still a decent chunk of work to be done to finish polishing
this, but it should give us a solid starting point.
For details about the library and a brief roadmap, see the main comment
in `command_line.h` which provides a comprehensive overview of the
library and a roadmap of the remaining work.
Porting the driver to this went fairly well, but did require some
changes. That port is separated into a follow-up PR #2979.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Lucile Rose Nihlen <luci.the.rose@gmail.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
This reflects changes from a number of approved proposals:
- #2138 : "generic" -> "checked generic", "template" -> "template
generic"
- #2360 : "type", "facet type", "facet". Note: I am not using the term
"generic type" from #2360 since that meaning conflicts with the
generally accepted meaning of "generic type" of a type with a
compile-time parameter.
- #2760 / #2770 : internal/external impl -> extending impl
- #2964 : "symbolic constant" and "template constant"
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Just some small refactorings stemming from #3073.
AddCheckLines -> BuildCheckLines because the two lists are now fully
separate. Adding is_blank to be more direct about behavior than the
Print call.
Where there was an object initialization immediately followed by
assignment I condensed it to one line, which seems to be the convention
looking at other files. I also deleted the repetition of 'of' in some
c++ comments because it was grammatically incorrect.
---------
Co-authored-by: Clayton Gearhart <claytongearhart240@gmail.com>
Makes the trace output for pattern match more consistent with the rest
of the trace by using `Match()` and `Indent()` prefix methods and
wrapping the code in the trace with backticks.
Unlike in the toolchain, we have not been preferring LLVM facilities
over standard ones. Update the documentation to describe this and
provide some rationale.
When producing textual semantics IR, map each branch instruction back to
the construct that produced it and use that to determine a name for the
corresponding block label.
Instead of manually putting symbols before and after heading, this
introduces two methods `Heading` and `SubHeading` inside `TraceStream`.
Both methods take `llvm::StringRef` as parameter, formats the given text
as follows and adds it into the output stream.
**Heading**
```
* * * * * * * * * * heading * * * * * * * * * *
-------------------------------------------------
```
**Sub heading**
```
- - - - - sub heading - - - - -
---------------------------------
```
Note: both methods assert that tracing should be enabled.
Also indicate what can appear within parentheses.
This is intended to be a clarification, not a design change. Note that
while we previously described the operand of `++` or `--` as being
simply an expression, the operand can never be anything other than the
kinds of expression the diagram now shows due to the expression category
rules added in #2006.
Fixes#3079.
Don't elide the StructType node if we've already created an equivalent
type. Its spelling and location may be interesting to diagnostics,
tooling, debug information, etc.
Add a textual IR format to the toolchain.
The exact details of the format are somewhat arbitrary right now, and I
expect them to change as we refine the semantics IR model, but at the
moment they're somewhat directly following the current structure of the
IR.
Semantics tests currently test both the "raw" format, which shows the
details of the representation, and the textual format, which is somewhat
higher level. We may want to revisit that decision once the textual
format is a bit more stable, and test only the textual format in most of
these tests, but for now it seems prudent to keep both sets of tests.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
- Only allow assignment at the top level in an expression statement.
- Allow both negation and complement as subexpressions of both bitwise
and numeric operators.
- Remove parsing support for postincrement and postdecrement.
- Add parsing support for `as` operator.
- Use the same ambient precedence for types and non-type expressions.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Defines methods into `TraceStream` for adding line prefixes. Instead of
directly using string literals.
Example usage,
```
trace_stream_->Start() << "declaring ... " << ... ;
```
will result in,
```
->> declaring ...
```
The Prelude path can be very long, which can make the `SourceLocation`'s
`Print(...)` method output unnecessarily verbose. This PR modifies the
`SourceLocation`'s `Print()` method to just print the **filename** and
**line number** for prelude.
Refactor type canonicalization so that we can reuse the same code for
building a `T*` expression and for forming the type of an `&x`
expression.
Add basic computation of expression category in order to check that we
only take the address of durable reference expressions. This is
currently computed on demand rather than being tracked as part of the
semantics node, but in most cases can be determined by looking at only a
single expression, so caching it in the node doesn't seem worthwhile
yet. This decision should be revisited if we start doing more complex
category calculations.
Also add trivial lowering support, but it doesn't work properly yet
because lowering doesn't yet take the expression category into account.
Allow interleaving of STDOUT and STDERR check lines. Put STDOUT lines
after the line they're attached to, and STDERR lines before. If no
STDOUT check line is attached to any line, then put them all at the end
of the file instead.
This is intended to better handle the case where stdout contains
unreplaced mentions of line numbers, and also reflects that stdout is
typically a consequence of the test rather than commentary on it, so
placing it after the test seems likely to read better.
Updates the trace output for type checking to be more consistent like
the rest of trace output.
This PR also includes few changes in the lit tests related to trace
output.
---------
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Currently, they are marked as left-associative which isn't completely
obvious for unary operators and wouldn't match the fact that we have
both prefix and postfix unary operators we intend to allow to repeat
without parentheses: `***x` and `T***`.
This fixes the graph by making the left-associative marker only for
binary operators, and using a separate marker for repeating unary
operators: a diamond.
It also adds a note that Carbon currently only has left-associativity
because the right-associative operators like assignment are statements
in Carbon.
This isn't intended ta be a change of anything in Carbon's design, just
an improvement to the documentation.
Introduce a concrete design for how Carbon values, objects, storage,
variables,
and pointers will work. This includes fleshing out the design for:
- The expression categories used in Carbon to represent values and
objects,
how they interact, and terminology that anchors on their expression
nature.
- An expression category model for readonly, abstract values that can
efficiently support function inputs.
- A customization system for value expression representations,
especially as
seen on function boundaries in the calling convention.
- An expression category model for references instead of a type system
model.
- How patterns match different expression categories.
- How initialization works in conjunction with function returns.
- Specific pointer syntax, semantics, and library customization
mechanisms.
- A `const` type qualifier for use when the value expression category
system
is too abstracted from the underlying objects in storage.
---------
Co-authored-by: Geoff Romer <gromer@google.com>
Co-authored-by: josh11b <josh11b@users.noreply.github.com>
Co-authored-by: Adrien Leravat <Pixep@users.noreply.github.com>
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
The constructor accepts a `bool` which we were setting to `true` by
converting from a heap allocated, and thus non-null, pointer. But that
heap allocation was dead and leaked, somewhat obviously. Leak checking
is disabled on our CI at the moment, but this was failing for me locally
with our default build on Linux where it uses ASan.
The `true` value also had no effect because the default argument to the
bool parameter is itself, `true`. :sigh:
It's really sad that this compiled. But it's the same thing as using a
non-null pointer in an `if`. There doesn't seem to be any `clang-tidy`
check for a `new` expression that is implicitly converted to a `bool`
type either. I've filled
https://github.com/llvm/llvm-project/issues/64461 requesting a check or
warning to catch this in the future.
This PR resolves the spelling error in the "Contributing" section of the
README. The sentence "Contribute the language design: feedback on
design, new design proposal" has been corrected to "Contribute to the
language design: feedback on design, new design proposal". This change
enhances the clarity and accuracy of the contribution guidelines.
Closes#3068
Co-authored-by: aroramrinaal <aroramrinaal@gmail.com>
Jon Ross-Perkins
Richard Smith
maan2003
Lucile Rose NihlenandChandler Carruth
Farzana Ahmed SiddiqueandFarzana Ahmed Siddique
Prabhat Sachdeva
josh11bandRichard Smith
Clayton GearhartandClayton Gearhart
Mrinaal Aroraandaroramrinaal