Commit Graph
193 Commits
Author SHA1 Message Date
Jon Ross-Perkins dfe992f1bc Rename DotOrArrow to PeriodOrArrow (#4263)
We generally say Period (this is the only "Dot" in code), so this seems
more consistent.

Also renames a few struct tests that had "dot" in the name.
2024-08-27 23:46:07 +00:00
Chandler Carruth 5d0ec91c20 Collection of minor tweaks to get approx. 10-15% compile time (#4245)
Most of these are about enabling inlining, in a couple of cases moving
code to a header and throughout switching to `CARBON_DCHECK`. The code
size of `CARBON_CHECK` seems to make inliing quite unreliable. I'm going
to think about whether there are ways to improve this, but a reasonably
small number of these seem worth switching for now to get some compile
time savings.

Also moves VLOG out of the hot path which helps a bit as well.

All combined, this net a bit over 10%, although it varies a bit exactly
how much. We're now pretty consistently over 800k lines/second for check
in the compilation benchmark for files >=4k lines, which makes me happy.
That's remarkably close to our original target.

Not really planning to keep optimizing here, just was glancing at the
profile and many of these stood out to me and were easy to fix.
2024-08-23 14:43:54 +00:00
935715e704 Implement new precedence from #4075 (#4236)
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
Co-authored-by: Geoff Romer <gromer@google.com>
2024-08-22 22:32:19 +00:00
David Blaikie a17480133f Remove excess use of auto on initializers (auto x = Y(z) -> Y x(z)) (#4239) 2024-08-22 20:29:35 +00:00
Brymer Meneses 435ee44e4c Report invalid parse on var x: i32 = {.7 = 8} (#4237) 2024-08-22 16:21:46 +00:00
Brymer Meneses 34ae2b001b move tuple testdata from index/ to tuple/access (#4226) 2024-08-19 16:47:14 +00:00
Brymer Meneses c353f6bd78 change tuple index (#4218)
This changes the tuple index from tuple[0] to tuple.0 in accordance with
the accepted propsal
https://github.com/carbon-language/carbon-lang/pull/3646

I messed up syncing to trunk on my original PR
https://github.com/carbon-language/carbon-lang/pull/4186, that's why I'm
starting on a blank state. Please let me know if I missed incorporating
a change from my prior PR.
2024-08-15 16:06:02 +00:00
Jon Ross-Perkins f67791cfee Separate subtree size information from parse nodes. (#4174)
Move subtree sizes over to TreeAndSubtrees, using the different
structure to represent the additional parse work that occurs, as well as
making it clear which functions require the extra information. My intent
is to make it hard to use this by accident.

The subtree size is still tracked during Parse::Tree construction. I
think a lot of that can be cleaned up, although we use it during
placeholder assignment so it may take some work. I wanted to see what
people thought about this before taking action on such a change.

I'm using a 1m line source file generated by #4124 for testing. Command
is `time bazel-bin/toolchain/install/prefix_root/bin/carbon compile
--phase=check --dump-mem-usage ~/tmp/data.carbon`

At head, what I'm seeing is:

```
...
parse_tree_.node_impls_:
  used_bytes:      61516116
  reserved_bytes:  61516116
...
Total:
  used_bytes:      447814230
  reserved_bytes:  551663894
...
1.43s user 0.14s system 99% cpu 1.565 total
```

With `Tree::Verify` disabled completely, it looks like:
```
parse_tree_.node_impls_:
  used_bytes:      41010744
  reserved_bytes:  41010744
...
Total:
  used_bytes:      427308858
  reserved_bytes:  531158522
...
1.20s user 0.13s system 99% cpu 1.332 total
```

Re-enabling just the basic verification (what is now `Tree::Verify`),
I'm seeing maybe 0.05s slower, but that's within noise for my system. I
do see variability in my timing results, and overall I think this is a
0.2s +/- 0.1s improvement versus the earlier (always testing `Extract`
code) implementation. That's opt; debug builds will be unaffected,
because the same checking occurs as before.

Note, the subtree size is a third of the node representation, which is
why I'm showing the decrease in memory usage here.
2024-07-31 19:39:45 +00:00
Jon Ross-Perkins 43c0b0a1f2 Refactor some check-phase postorder iterator use. (#4175)
Allow directly constructing a PostorderIterator, to get rid of
`tree.postorder(node_id).end()` indirect construction. For ranges that
don't need tree data, make it clearer that they're not validated.

Note, this subtly gets rid of a subtree size use in the
`tree.postorder(node_id).end()` case (to get the discarded `begin()`
value).
2024-07-27 02:15:56 +00:00
Jon Ross-Perkins ae675e61bd Add initial parsing for 'extern library' (#4173) 2024-07-26 20:47:25 +00:00
Richard Smith 3cb769a053 Rename "generic instance" to "specific" throughout the toolchain. (#4165)
As discussed in toolchain meeting, we want to avoid overloading the
meaning of "instance", and "specific" was the best name we found. It's a
little unorthodox and inventive, but hopefully over time will become as
unsurprising as the term "generic" is.
2024-07-25 16:42:01 +00:00
Jon Ross-Perkins db022658c6 Implement syntactic merge checks for parameters. (#4149)
Note this isn't implementing checking through imports. The parse node
there is harder to access through the context, so would require
examining the entity in order to get the import declaration, to get at
the ImportIR. We also don't have a parse tree attached in that case, and
would need to add one to SemIR::File. But I believe we do want to add
that, so it's explicitly a TODO.

Note GetTokenText re-lexes literal values, so there's a bit of potential
overhead there. Not sure if we want a more efficient manner for
comparing in cases like this.
2024-07-23 20:32:24 +00:00
Jon Ross-PerkinsandGeoff Romer 07c286e3cb Use the package/library name in ImportIRId formatting. (#4154)
Also adds import_ir_scope to namespace formatting. I'd done this as an
aid for #4153, and am splitting it out.

---------

Co-authored-by: Geoff Romer <gromer@google.com>
2024-07-22 22:43:36 +00:00
Jon Ross-Perkinsandjosh11b f1190a4792 Add basic output of where memory is stored after a compile. (#4136)
The output is really basic, I'm just adding this to help track how
memory is allocated.

```
---
filename:        'check/testdata/expr_category/in_place_tuple_init.carbon'
source_:
  used_bytes:      8057
  reserved_bytes:  8057
tokens_.allocator_:
  used_bytes:      0
  reserved_bytes:  0
tokens_.token_infos_:
  used_bytes:      1040
  reserved_bytes:  2032

(eliding)

value_stores_.string_literals_.set_:
  used_bytes:      320
  reserved_bytes:  320
Total:
  used_bytes:      20609
  reserved_bytes:  29437
...
```

---------

Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com>
2024-07-16 23:11:01 +00:00
Richard Smith 4aa34978eb Import support for tuple, struct and integer values. (#4042)
These can be imported as the argument values for a generic. Also, the
name of a generic is modeled as a `StructValue`.
2024-06-07 22:08:22 +00:00
Jon Ross-Perkins 60db3df24b Remove glob_sh_run (#4041)
file_test's --dump_output flag has essentially supplanted this
functionality, and is necessary for execution on multi-file tests.
2024-06-07 18:45:41 +00:00
Chandler Carruth 8c64f0bfdd Add -Wmissing-prototypes and fix issues it finds. (#4019)
Most of these are places where we failed to include a header file and
simply never got an error about this. The fix is to include the header
file.

Most other cases are functions that should have been marked `static` but
were not. Finding all of these was a main motivation for me enabling the
warning despite how much work it is.

One complicating factor was that we weren't including the `handle.h` for
all the state-based handler functions. While this isn't a tiny amount of
code, it is just declarations and doesn't add any extra dependencies. It
also lets us have the checking for which functions need to be `static`
and which don't. For the `parse` library I had to add the `handle.h`
header as well, I tried to match the design of it in `check`.

I have also had to work around a bug in the warning, but given the value
it seems to be providing, that seems reasonable. I've filed the bug
upstream: https://github.com/llvm/llvm-project/issues/94138

I also had to use some hacks to work around limitations of Bazel rules
that wrap `cc_library` rules and don't expose `copts`. I filed a bug for
`cc_proto_library` specifically:
~https://github.com/bazelbuild/bazel/issues/22610~ 
https://github.com/bazelbuild/bazel/issues/4446
2024-06-04 20:04:45 +00:00
Jon Ross-Perkins d9c62b106d Rename enclosing scope to parent scope (#4020)
Following up on discussion from #3948, doing a general rename of
"enclosing scope" to "parent scope" (and "enclosing scopes" to "ancestor
scopes"). The intent is to improve understandability and collide less
with C++ terminology for "enclosing scope". Note this changes most uses
of "enclosing", but leaves behind a few like "enclosing function" and
"enclosing block".

Note this does create some "parent class" mentions for "adapt" and "var"
(the class they're within), which is maybe unfortunate, but we'd
probably say "base class" if we meant inheritance so perhaps that's
okay. Along the same lines, these are the only `parent_class` uses I see
now, and we do have a few `base_class`.
2024-06-04 19:57:14 +00:00
Jon Ross-Perkins 83413479d7 Move some of the test information to TIP lines (#4007)
This has is a nice-to-have for me. Frequently I want to run a specific
test, and end up digging through output to be able to copy-paste the run
line. This uses TIP lines to inject the command into the file when using
AUTOUPDATE.

Note, one of the reasons I want this is because "bazel test
//toolchain/testing:file_test --test_output=all" has been regularly
exceeding bazel's output limit for me (workaround is either opening the
output file or specifying an obscure output limit flag), making it a
little harder for me to get the commands. However, frequently I'm adding
a file and want to iterate on it, so that's really the use case I have
in mind here.
2024-06-04 17:57:23 +00:00
Richard Smith c2ab520522 Add typed forms of TokenIndex for tokens with a known kind. (#4016)
Instead of adding a `Token<Kind>` field to typed parse nodes that
contains a `TokenIndex`, make the parse node directly hold a
`TokenIndex` of a more specific kind.
2024-06-04 00:20:49 +00:00
Chandler Carruth dd0890619a Enable a couple of boring warnings. (#4018)
Just spotted these while looking at warnings that seem to fire on our
code are probably are things we'd fix if we saw them. None of these seem
important FWIW.

Also removes a redundant flag that is part of `-Wall`.

I have a follow-up for the high-value warning I spotted that motivated
me to look at all of this. But it's noisy so kept it as a separate PR.
2024-06-03 05:02:26 +00:00
Jon Ross-Perkins 517a416852 Clean up some misc toolchain braced inits. (#4013)
Following up on #4012 and #4009, clean scattered cases which could be
making better use of designated initializers.
2024-05-31 23:23:57 +00:00
Richard SmithandJon Ross-Perkins 3c01ee69ed Move information on the token associated with a parse node from the .def file into the typed node. (#4001)
Instead of tracking the token associated with a parse node in the `.def`
file macro, track it on the typed node instead. List the token as a
field inside the node structure to show the order of the token relative
to the other components of the grammar production, and to allow the
token index to be accessed when the node is extracted.

Remove the corresponding information from the `.def` file, leaving
behind just a list of parse node kinds in the majority of cases.

This also removes the checking of the token kind associated with a parse
node in the case where the parse node has errors. Previously we had a
flag on the node kind to indicate whether we should check this, but per
[discord
discussion](https://discord.com/channels/655572317891461132/655578254970716160/1246214418979881052),
we have decided to remove this.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-05-31 23:11:51 +00:00
Jon Ross-Perkins cda5f66d22 Refactor NodeCategory to provide a class API (#4004)
Mirroring #4003 for NodeCategory.

Note we template a lot more on NodeCategory's enum, so this is a
slightly more awkward delta.

Also, switch from Enum in KeywordModifierSet to RawEnumType for
consistency with EnumBase. The templating on NodeCategory had me
thinking about that more.
2024-05-29 22:47:50 +00:00
419d2e39d8 Move child count and bracketing information for parse nodes into the node kind definition. (#4000)
Instead of tracking the bracketing and child count information in the
kind macro in `node_kind.def`, provide it to `NodeKind::Define` in
`typed_nodes.h`. If a node is both bracketed and has a fixed child
count, track both facts and check them both in tree verification, since
it's easy to do so now.

The overall goal here is to reduce `node_kind.def` down to a simple list
of names. I have a slightly different approach in mind for the token
kinds.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com>
2024-05-29 20:57:51 +00:00
Richard SmithandJon Ross-Perkins 28170c7867 Parse parameters in name qualifiers. (#3988)
Parse the name of a declaration as a sequence of `NameQualifier`s --
which have a name, possibly parameters, and a trailing period --
followed by a name and possibly parameters. This prepares us for parsing
declarations of members of generic classes and similar cases, but
actually supporting such member redeclarations is left to a future
change.

We previously required functions to have parameters, but no longer do,
following the direction of #3848. Cases like namespaces that can't
actually have parameters are now diagnosed in check instead of in parse.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-05-29 00:47:29 +00:00
Richard Smith a1ff49af03 Add more tests for #3963. (#3974)
I forgot to `git add` these files.
2024-05-21 21:36:31 +00:00
Jon Ross-Perkins 8bb33f96c3 Update some additional package rules where structure changed in #3963. (#3966)
I think the template files are simply an oversight. The fuzz files
should probably be updated because failing on the `package` line isn't
an interesting test of logic. Plus one minor comment edit.
2024-05-21 20:33:20 +00:00
josh11bandJosh L 7486acd6ae Fix comment typo (#3970)
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
2024-05-21 18:23:10 +00:00
Chandler Carruth 2a39e83779 Fix a new C-style cast clang-tidy error in #3963. (#3965)
This isn't *technically* a C-style cast, but a function-style cast. And
newer versions of `clang-tidy` no longer fire on this code, so it is a
little sad to have to work around. But the version on our build bots
seems old enough to still hit this issue (and possibly some others).

Our merge queue also doesn't seem to catch these reliably, but that's
for a separate PR.
2024-05-21 15:22:47 +00:00
Richard Smith 23f9065949 Implement proposal #3927. (#3963)
Change syntax for package declaration to put the `impl` keyword at the
start and remove the `api` keyword.

To support this, rearrange processing of package, library, and import
declarations to use the general modifier handling support in declaration
parsing rather than special-case logic.

There is an ambiguity in `impl package.Foo as Bar`, which we resolve by
treating `package` as an introducer after a modifier only if it's not
followed by `.`.
2024-05-21 00:36:03 +00:00
Jon Ross-Perkins 03e3a72628 Rename Directive->Decl in general, BindExport->ExportDecl (#3957)
Mechanically, replacing Directive->Decl, directive->declaration,
_declaration->_decl (to avoid comments).

Context:
https://discord.com/channels/655572317891461132/963846118964350976/1241145948625703062
2024-05-20 22:53:36 +00:00
Jon Ross-Perkins ecb300ee5e Move 'export name' out of the import section. (#3950)
This implements recent updates to #3938
2024-05-17 18:26:14 +00:00
Jon Ross-Perkins a16842ab37 Add export keyword handling. (#3944)
This provides `export import` logic in lex, parse, and check; `export
name` logic is only in lex and parse, not check.

I think with `export name` I'm going to need to modify import_ref and
some consolidation logic, whereas `export import` seems feasible to keep
as primarily import logic. Given the implementations were looking like
they'd diverge more substantially, I thought it'd be helpful to cut the
PR here.
2024-05-10 22:54:06 +00:00
Jon Ross-Perkins 5fa341f16a Unify more invalid declaration recovery. (#3942)
When we expect a semicolon, we should be able to provide more standard
recovery.

Note this affects a test of `base`, but it looks like a partial
improvement (the prior line was moving too far). Still, it looks like
recovery isn't handling `{}` quite right. I'm not trying to address that
here though -- leaving a TODO.
2024-05-08 22:41:42 +00:00
Jon Ross-Perkins 096f1dc68a Rename functions which only print a diagnostic to Diagnose* (#3886)
This is something I noticed working on
https://github.com/carbon-language/carbon-lang/pull/3884; I think we
have more functions named Diagnose* at present than Emit* or Report*, so
just trying to consolidate. Note a couple Diagnose* functions do a
little more validation, but maybe those should actually be renamed away
(zygoloid had mentioned wanting to generally split out diagnostics to
their own function, and then we'd have it be a more common pattern).
2024-04-16 00:00:22 +00:00
Richard Smith 28ceb4dd4e Basic check support for adapters. (#3874)
No support is provided yet for converting between a type and an adapter
for that type.
2024-04-10 09:57:48 +00:00
Richard Smith c5b7ff212b Parsing support for adapt declarations. (#3871) 2024-04-09 13:27:33 +00:00
Jon Ross-Perkins 37aa309c81 Add recovery tokens using AddToken to get correct parse tree sizes. (#3851)
The expected parse tree size is validated for valid parse trees, which
is possible even if lex encounters an error. In that case, we were
missing recovery tokens, resulting in a crash. AddToken should be the
only place we call `token_infos_.push_back`.
2024-04-04 16:03:54 +00:00
Jon Ross-Perkins 80ca6b1298 Run clang-format over def files. (#3852)
Noticed an unexpected format change in #3850, it's probably because
pre-commit needs to explicitly include def files. The def change here is
a consequence of `pre-commit run -a` with this change.
2024-04-03 23:39:48 +00:00
Jon Ross-Perkins a034f86272 Change struct literal parsing to use placeholders. (#3850)
This is achieving a similar goal as #3849, using placeholders instead of
an ambiguous start node to clarify structure and incrementally simplify
checking. The benefit isn't quite as big here because both paths are
structs, and so checking is more consistent than paren exprs versus
tuples. But I think this removes the only other multi-purpose parse
node.

This uses StructLiteral/StructTypeLiteral naming, reflecting equivalent
SemIR naming. Note, I would lean towards renaming StructLiteral to
StructValueLiteral, but I think consistency in naming takes precedence.
Any renames of StructLiteral might be better in a separate PR.

StructFieldType/StructFieldValue -> StructTypeField/StructField is
trying to making the reading more consistent with
StructTypeLiteral/StructLiteral. SemIR has StructTypeField but not a
value equivalent.
2024-04-03 20:36:49 +00:00
Jon Ross-Perkins b42612bcec Change tuple/paren expr parsing to use placeholders. (#3849)
I've been thinking about this since we decided to add placeholders in
the parse tree. This allows a clearer division of work in check
handling, where we were doing work for ExprOpenParen that's only
necessary for tuples (splitting/renaming handle_paren.cpp accordingly).
2024-04-03 18:31:16 +00:00
Jon Ross-Perkins e66406ec93 Disable bugprone-macro-parentheses and let clang-format insert braces. (#3825)
-
[bugprone-macro-parentheses](https://clang.llvm.org/extra/clang-tidy/checks/bugprone/macro-parentheses.html)
-- this is just a false positive issue, I don't think it's helping us
catch bugs.
-
[InsertBraces](https://clang.llvm.org/docs/ClangFormatStyleOptions.html#insertbraces)
-- although there's a warning about this creating issues due to
incomplete semantic information, it seems to be happy with our code, and
allows clang-format to fix something that clang-tidy would otherwise
warn about.
2024-04-02 11:18:25 +00:00
Richard SmithandJon Ross-Perkins f9ce0b194d Defer parsing of method bodies until the end of a suitable enclosing scope. (#3832)
In parse, form a list of methods that are defined inline, tracking where
they start, where they end, and which other inline methods are nested
within them.

In check, when we reach an inline method body, skip it and add it to a
worklist to be processed later. We also track when we reach the start
and end of a context in which inline method bodies are deferred, so that
we know when to replay the bodies.

When suspending a function definition to be processed later, the
`DeclNameStack` entry is moved to separate storage, including popping
the corresponding scopes from the scope stack and removing the
corresponding lexical names from lexical lookup. Later, when we return
to the function and parse its definition, the `DeclNameStack` entry is
restored. The same is done when we reach the end of a nested context
that can have inline methods, so that we can reenter the nested scope
before processing its members.

---------

Co-authored-by: Jon Ross-Perkins <jperkins@google.com>
2024-04-01 18:25:27 +00:00
josh11bandJosh L 49252592f0 Prevent tuples after . and -> (#3830)
Co-authored-by: Josh L <josh11b@users.noreply.github.com>
2024-03-29 16:53:44 +00:00
Jon Ross-Perkins b5d28f2c4b location -> loc abbreviation (#3826) 2024-03-28 18:15:18 +00:00
Jon Ross-Perkins b079acd86f Replace NodeId with a hybrid LocationId in SemIR diagnostics. (#3810)
The purpose of this change is to allow something such as a FunctionDecl
instruction to note an imported instruction as the "loc_id". Note that
doesn't occur here: this change is already very sweeping in edits. There
is no testdata affected, intended to show equivalent behavior.

We might want to consolidate NodeId references towards LocationId, but
if that's preferred, I'd still like to split it out. A lot of this just
piping through LocationId where it's a build error otherwise, enough
that imports should be able to start using it for diagnostics.
ValueStores are added but still unused -- just flushing out structure
for review.

Restructuring SemIRLocation is necessary to use LocationId this way. For
TokenOnly, it's not getting used in Parse, so I migrated it to Check and
it's now specific to SemIRLocation.

I also considered making LocationId reference an InstId (which would
need to be an ImportRef) instead of an ImportIRInstId. However, that
would've required import.cpp to add instructions for decls which are
reached during resolution -- we typically don't have an inst ready for
use. An extra inst is essentially 16 bytes in InstId's ValueStore + 4
bytes in LocationId's ValueStore, whereas this is 8 bytes per.
2024-03-27 22:55:22 +00:00
Jon Ross-Perkins 6c458ffe7e Add import context for locations. (#3807)
As discussed around #3792, identify the import a diagnostic message came
from prior to the diagnostic message itself. This occurs during location
translation so that the logic can be central.

I'd considered associating the parse node with ImportRef instructions,
but I realized about halfway through that because I need to store the
ImportDirectiveId on the ImportIR for cross-package imports, it's there
for use in location translation without extra work. That saves a fair
amount of stringing it through declarations, as well as an oddity where
ImportRef instructions would have a node that didn't really represent
them.
2024-03-27 22:22:15 +00:00
Jon Ross-Perkins b8ceb8dd8b Print a blank line after a diagnostic. (#3806)
The purpose of the newline is to make it clearer where a given
diagnostic begins and ends, particularly as the first message of a
diagnostic may not be the error.

This is a trivial code change, but ripples edits through test files.
2024-03-22 18:10:49 +00:00
Jon Ross-Perkins 0bd45f0d6b Rename DiagnosticLocationTranslator -> DiagnosticConverter (#3804)
Since the addition of TranslateArg, I don't think this type is going to
go away (cutting a TODO). Refactoring names slightly to fit the current
role, and adding const to ConvertLocation.
2024-03-21 23:35:04 +00:00