Start treating function calls as initializing expressions instead of as
value expressions.
This required adding support for expression categories. Value bindings
and temporary materialization conversions are created where necessary to
transition between expression categories. For a function call with a
return slot, we speculatively create a materialized temporary before the
call and either commit to it or replace it with something else later,
once we see how the function call expression is actually used.
This change follows the direction suggested in #3133 for initializing
expressions: depending on the return type of a function, the return
value will either be initialized in-place or returned directly. This is
visible in the semantics IR, which is a little unfortunate but is
probably necessary as this is part of the semantics of the program.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Splits IR files into SemIR, and logic files into Check. These will be
split into separate directories as part of a later move; the namespaces
are being done first in order to vet the switch, and hopefully make
conflicts a little easier to manage due to the substantial renames.
A lot of this is just automated removal of Semantics prefixes from
names, adding namespace references where needed. A few special-cases
are:
- SemanticsIR -> SemIR::File
- A few things were discussed, like Unit, CompileUnit, or CompiledUnit.
Unit was too vague for chandlerc, and I thought CompileUnit might lead
to incorrect inferences (CompilationUnit would be more precise, but
typically written as SemIR::CompilationUnit which is pretty long). File
seemed to be a short name that we could agree on.
- SemanticsIRFormatter -> SemIR::Formatter
- FormatSemanticsIR -> SemIR::FormatFile
- SemanticsFileTest -> CheckFileTest
- It remains in the Testing namespace, where just "FileTest" might be
too broad a name.
- SemanticsDeclarationNameStack::Context ->
Check::DeclarationNameStack::NameContext
- This avoids a Check::Context name shadowing.
Changes check_internal.h to include ostream.h to improve finding of
Print/operator<< (otherwise it didn't compile).
This is part of #3070
This removes BindName, putting name information directly on VarStorage.
As a side-effect of updating semantics_ir_test for this change, I also
noted that function bodies were being generated as invalid YAML so am
fixing that (just `{}` to `[]` bracketing, otherwise the test wouldn't
work anymore).
Because names are now available, I've updated lowering to use them for
vars.
In the SemIR formatter, the name is now repeated because it's a
parameter to VarStorage. I believe this is just default behavior, and
we'd have to special-case VarStorage to remove it because it's automatic
argument printing in action. On the balance, it felt like letting it
print was reasonable.
I've noted in places that the name on VarStorage is expected to be
optional, but am not adding support because I'd have no way of testing
it at present.
Add a textual IR format to the toolchain.
The exact details of the format are somewhat arbitrary right now, and I
expect them to change as we refine the semantics IR model, but at the
moment they're somewhat directly following the current structure of the
IR.
Semantics tests currently test both the "raw" format, which shows the
details of the representation, and the textual format, which is somewhat
higher level. We may want to revisit that decision once the textual
format is a bit more stable, and test only the textual format in most of
these tests, but for now it seems prudent to keep both sets of tests.
---------
Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Refactor type canonicalization so that we can reuse the same code for
building a `T*` expression and for forming the type of an `&x`
expression.
Add basic computation of expression category in order to check that we
only take the address of durable reference expressions. This is
currently computed on demand rather than being tracked as part of the
semantics node, but in most cases can be determined by looking at only a
single expression, so caching it in the node doesn't seem worthwhile
yet. This decision should be revisited if we start doing more complex
category calculations.
Also add trivial lowering support, but it doesn't work properly yet
because lowering doesn't yet take the expression category into account.
Semantic handling for use of `T*` and `const T` as types.
There's no way to form values of these types yet, and no conversions for
them are supported.
Factor out the common code to canonicalize types using a folding set,
and switch to using the same folding set for all kinds of type by adding
the kind as part of the folding set key.
Improve type printing to not include the `as type` portion when the type
is printed in a context within another type where a conversion to `type`
is implied, as in `{}*` and pre-existing cases like `({}, {}) as type`
(which we used to print as `({} as type, {} as type}) as type`.
For `Assign` and `ReturnExpression`, this field wasn't used for
anything. For `StructTypeField`, we stored the type of the field here,
and now store it as an argument of the node instead.
---------
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Handles tuples (including nested tuples) in the semantic phase of the
tool chain. Does not handle tuple element access yet.
---------
Co-authored-by: Farzana Ahmed Siddique <fasiddique@google.com>
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This handles namespacing of functions. Parsing and semantics are changed
significantly, while lowering works without changes. Variables can't be
namespaced yet because they're dealing with patterns, and I didn't dig
through that code.
Most of the logic is done through the new name declaration stack, which
is necessary because semantics isn't quite sure where the declaration
name ends. It'd be complex for parsing to send a signal about this,
probably involving node variants and rewrites of the tree, and this
solution seems to work well. Unfortunately this means a new stack, but
that may be inevitable due to the extra information needing to be
tracked.
Note this doesn't deal with scoped lookups of non-namespace things,
which we'll need for generics. That'll probably involve pushing resolved
scopes onto a stack (or maybe just setting a singleton value?) to affect
contextual name lookup. But, I think the basics are there to make it
work when we can test the behavior.
This renames "designator expression" to "qualified expression" and adds
"qualified declaration" in order to use terminology more consistent with
C++.
Namespaces will probably need to be considered for name mangling down
the line, but this still uses the basic name.
Add validation that every code block in a function is terminated by a sequence of terminating instructions, and that terminators don't appear anywhere else in code blocks.
This required tracking whether we're in a reachable code block. That's done on the fly when we create a new code block; the new `SemanticsNodeBlockId::Unreachable` is used to represent the case where we're not actually creating a code block because we're in unreachable code.
This follows the same structure as `if` expressions, except that no
result value is needed.
Semantics IR building for code blocks is also added. Rather than popping
all the node stack entries we push for statements within a code block,
change statements and declarations to not push themselves onto the
stack. We're not notionally performing work recursively within prior
statements, and we don't need their value for anything, so it seems
cleaner to not push them. This also allows statements and declarations
to determine what syntactic context they're in by peeking at the top of
the stack, though that's not used in this patch.
Add semantic analysis and semantics IR building for `if` expressions, and add the first parts of control flow handling to semantics IR. After discussion with @chandlerc, use [block arguments](https://en.wikipedia.org/wiki/Static_single-assignment_form#Block_arguments) to convey values from the two arms of the `if` to the result. For now, only a single block argument is supported, but we should revisit this as we explore more of the requirements of the Semantics IR form.
Functions can now contain multiple code blocks, so grab the entry block up-front instead of assuming the entry block will be at the top of the block stack when we reach the end of function emission.
Add trivial support for `bool` type literal, because without it we can't write testcases.
This shifts logic so that bindings are added to name lookup only after the scope is complete, removing logic around adding/removing/re-adding names in certain scopes.
This does mean that things like a function's forward declaration will need to go through an extra hoop for name conflict checks, because under this approach a function definition does conflict checking when it adds names for the body's use. But, that seems easy to address, and better than the current hoops.
This is a rename of a commonly used accessor. I tend to prefer shorter names, but it's semantics_ir in LoweringContext, and it feels odder to shorten to semantics there than to lengthen to semantics_ir here. I already have builtins_ir around here, too. I think it's helpful to use consistent names for semantics_ir since they're dealing with essentially the same thing.
This switches types to using SemanticsTypeId instead of SemanticsNodeId, and lowering pre-builds its list of types. The empty tuple type is special-cased because we don't want to emit it unless it's in-use, but as the implicit return for functions, it's frequently used. Callables use invalid to indicate the implicit return, and that seems undesirable to change due to the size increase.
We do use the empty tuple type for function returns, so I don't want to get rid of it, but the value is unused. With this change, builtins are all types. Long-term the empty tuple value should have a representation similar to structs; just a tuple value that's empty, not a built-in.
This removes special-casing of empty structs, handling them as just a regular value instead of a builtin. Note the `{} as Type` is still special-cased.
In lowering, removes the test of calling a function using `{}` because it's missing the proper load/store. This setup notices that error whereas the prior worked due to said special-casing. Fixing this will need to be done as part of generally adding loads for variable uses.
This is the first step to refactoring types into a SemanticsTypeId. This only tracks what's in-use, but as a consequence starts funneling type information through in ways similar to how I'd want it to do SemanticsTypeId.
Previously, IR for arguments in calls and struct values was separated out. This merges it back in. Additionally, parameters for functions and struct types had their own IR; the block is still there, but there's a TODO to decide what to do with it.
In the LLVM IR, this has the consequence of emitting expressions that are inputs to a call or struct value within the scope of the function, which is pretty much where it should be. Importantly it happens before the call is encountered.
This change also tinkers with the int and real literal lowering. I'm pretty sure both are still wrong, but was having trouble figuring out a "better" way to do it, and this seems like it'll work for now.
This echoes #2818 and the philosophy is mostly covered there.
Versus parsing, semantics uses fewer separate handler files (for now) because the logic has been shorter. However, the design is still intended to make it easy to split files along boundaries similar to the parser, as I've done for a couple more complex/inter-related sections.
This handles the basics of type and value for structs. Structurally, these look like parameters and arguments (respectively) because expressions/generics may result in multiple IR nodes being generated.
Because `{}` needs to be cast to a type for storage, I'm also adding some validation that's not specific to `{}`, e.g. that `1` shouldn't be valid as a type for storage (previously, nothing errored for that).
This adds more stringification of types, particularly literals, because they come up in value errors now.
ImplicitAs is the result of me mulling whether I'm taking the right approach on type conversions. I think it needs to return a value so that if the implicit cast rewrites the value, the result is accessible to the caller. I may reorient the current TryTypeConversion logic to be more based on the ImplicitAs logic.
I'm trying to reduce the amount of per-SemanticsNodeKind boilerplate, and make mistakes (e.g., SemanticsNodeKind not matching the Make name, misplacing the type, or Get/Make type mismatches) easier to see.
I could've done this with (more) macros, but felt that the template approach was reasonable enough and likely easier to understand/debug. I'm not sure whether there's more that I could be doing with variadics to reduce the amount of factory code, but this feels good right now.
These changes should make output more stable when builtins are added to semantics. By omitting them from nodes and printing nodes as "relative to the last builtin", I should be able to add and remove builtins without automatically affecting every test. Also by printing builtin nodes as `nodeNameOfBuiltin`, it's a little easier to understand what's going on (for me, at least).
This adds tracking of call information plus basic type checking. It adds a builtin for the empty tuple, mainly so that I have the basis for a default function return type.
As an aside, it also unifies printing within SemanticsIR, fixing a missing comma after callables.
Co-authored-by: Richard Smith <richard@metafoo.co.uk>
Reals were mostly handled, but this PR adds storage of them. It also switches a little towards the FloatingPointType semantic from TokenizedBuffer.
While real literals like `1.0` were handled, the type literals were not. This just adds `f64`, similar to how I also only support `i32`.
The String type literal wasn't used, so I've added support in lexer and parser. Per discussion with @zygoloid String might be renamed based on the newer type literal plan, but it's still String in explorer and the design, so this is just consistent.
The builtin_types.carbon tests the three basic types that are there right now. The test is added to both parser and semantics so that it's clear what the state is in both stages.
It feels like every time I add a builtin, I forget to add it to the builtin IR. This approach enforces the addition of builtin nodes, and also takes advantage of the normalized structure to set types appropriately (I think the new approach is simpler on this aspect).
For parameters (and in the future, arguments too; generally comma-separated lists) track two node blocks:
1. param_ir: The complete IR.
2. param_refs: Nodes within the IR that are the "root" parameter.
param_refs should allow quick counting of the # of parameters, and more efficient comparison of call args with function parameters. param_ir should be necessary to generate the actual signature.
In order to construct this, this refactors the node_block_stack into its own class, which is reused in params_stack. These carry references to the underlying SmallVector for lazy modification in order to avoid a dependency cycle with SemanticsIR (also see notes on empty node blocks below).
When finalized, the block pair is pushed onto finished_params_stack. That's because node_stack only has space for one thing, and this is two things -- so I'm essentially choosing a trade-off of adding another stack in order to avoid consuming more space in the expectation that most parse nodes have 0 or 1 things to return, and 2 will be very rare.
As factored, this currently consolidates most empty node blocks into a single canonical empty node block. This is because I think empty blocks, i.e. `()`, will be very common. In order to achieve this, SemanticsNodeBlockStack does lazy creation.
An alternative approach would have been to use 1 node block per parameter. We decided against this in order to reduce the number of vectors being created.
The parser and lexer already produce YAML, so this is fundamentally a consistency issue. I've been thinking about this, and was looking again because I'm working on adding callables, and figured I'd just fix it now.
The goal here is to (significantly) reduce the boilerplate needed when defining classes that wrap enums, especially those managed with the `.def`-file style X-macros that are common in the toolchain.
This should also provide both better and more consistent functionality to those classes once ported over to it.
Initially, only `ParserState`, `SemanticsNodeKind`, and `SemanticsBuiltinKind` are ported as these were also the three that JonMeow ported in his original pull/2453 "option 5". This is heavily based on that version of the code.
Goals I was considering that influenced the design:
- Keep the individual enum-wrapping classes as simple and easy to read as possible. Especially important is keeping the `.def` files that are often filled with really important documentation clean and easy to maintain over time.
- Don't rely on computed `#include`s as that is an especially dark corner of the preprocessor and breaks some build systems.
- Have a really good API of the enum-wrapping class, including nice constant names for the values, easy printing, and even easy debugger-callable methods to get the name (as opposed to the integer value).
- Keep the API that users interact with in the base class as clean and easy to read as possible.
- Reduce the boiler plate for each instance of these as much as possible.
- Avoid excessive inline generated code or constants that would result in steady growth in object file sizes and linker effort doing deduplication.
These goals aren't always compatible, so we end up needing to pick a compromise between them when in tension. I think this version is a pretty good compromise.
The original version I started with already pull most of the API into a CRTP-style base class. This version pulls *all* of the common API. This is the main tool for getting consistency and avoiding duplication. However, connecting this base class to the individual enum wrappers is still difficult. Some specific changes here that try to do as much as possible there:
- Use a slightly fancier macro pattern to reduce the boilerplate of defining the raw `enum class` prior to the wrapper class.
- Use a macro to simplify naming the base class.
- Move the name table to a `.cpp` file to avoid every inclusion generating a complete copy of the strings (that the linker has to deduplicate). This is done with some care to sharply reduce the boilerplate needed in that `.cpp` file.
- Sink the name _API_ fully into the CRTP base class. This requires some significant complexity in the implementation, but all of that is hidden behind a single implementation detail macro, and the API itself is simple and readable. This also makes it much more reasonable to test the entire system a single time next to the base class.
This version also moves from constant factory functions to normal constants. This requires two batches -- first a declaration, and then a definition -- but the API result is significantly better and similar to the original option, the macro structure reduces the cost of these. Unfortunately that makes the adoption a bit noisy, but I think its worth the churn.
Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
This switches to single list storage of SemanticsNode. The driving motivation behind this is to simplify cross-references within a given IR. Types of nodes will frequently refer to other blocks. This causes a significant increase in the number of cross-references, which can become difficult to manage (and reason about). By reducing to a single list of nodes, cross-references are only needed when crossing IR boundaries.
Because cross-references now only have 2 things to track (IR and index), they can be a regular SemanticsNode and don't need further indirection. This wasn't motivating, but feels like it reinforces the simplification.
Note this isn't being used to deduplicate nodes, at least right now. That could lead to difficult-to-update situations, but also most nodes are associated with the underlying ParseTree::Node in order to track sources for diagnostics; as a consequence, nodes representing equal text in different source locations wouldn't be the same node. There may be future opportunities here, discussed with @zygoloid, but no action is taken at present.
We may eventually want to switch the storage of NodeBlocks to have `[start, end)` ranges instead of individual numbers, but I'm leaving that alone for now.
As an aside, I noticed I was accidentally overloading the copy constructor on SemanticsIR. I've added some disambiguation on that, but am not deleting the copy constructor per style advice (even though the type should never be copied due to storage size).
codespell tries to change `CrossReference -> cross-reference` so disabling it there.
When binding a name, add it to name lookup. On NameReference nodes, use name lookup.
- Switches from "identifiers" to the more generic "strings". Not strictly necessary here, but it's the overall direction I think we've agreed upon and wanted to do it while building more support out.
- Starts doing deduplication of strings.
- On BindName, registers names with name lookup.
- Does name lookup based on the deduplicated string.
- Per discussion with zygoloid, design is intended to be constant-time lookup regardless of the number of parent scopes.
- Adds scopes so that we can track names which will be deregistered from lookup.
Richard SmithandChandler Carruth
Jon Ross-Perkins
Farzana Ahmed SiddiqueandFarzana Ahmed Siddique