mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-10-05 07:21:04 +01:00
69353ed271dd2bb29dd5f7f59e4ee8437fa11bc3
2053
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Richard Smith
|
69353ed271 |
Basic support for incomplete types. (#3302)
Incomplete types may be nested within other types; for example, a tuple type might have an incomplete type as an element. Handle such cases by walking through nested incomplete types when completing a type. This is done non-recursively in case a very complex type is formed. Types are generally no longer completed at the point where they're formed. Instead, we attempt to complete a type when it is used in a context that requires a complete type, and diagnose if the type cannot be completed at that point. This will be necessary for classes, which can become complete after their first use, and helps tease out bugs where a type completeness check is missing. |
||
 Chandler CarruthandRichard Smith
|
3015135a52 |
Skip blocks of comments with identical prefixes. (#3299)
Specifically, after lexing a comment line, look at the next line and see if it starts with an identical sequence of indent, comment '/'s and character after the '/'s. If so, skip it as part of a block of comments. This skips repeatedly diagnosing the same erroneous comment introducer after the first one in a block, but that seems like a feature rather than a bug. The big motivation is to make sure the lexer is minimally impacted by the length of comment blocks and skips them as efficiently as possible. While they aren't exactly common, large block comments do come up and it'd be unfortunate for those to actually slow down the toolchain. It also happens that this is particularly easy to do because we're just looking to see if we see the same prefix byte sequence. With SIMD we can typically handle the most common indents with just a few instructions. Because of the diagnostic differences, I've included a scalar fallback that replicates the functionality but has no limit on indent size or CPU features. I've also added testing to cover this behavior. The only non-noise benchmark changes are as expected the comment ones, with a nice improvement across the board: ``` BM_CommentLines/1/0/0 15.8ms ± 2% 15.6ms ± 2% -0.87% (p=0.004 n=19+19) BM_CommentLines/4/0/0 20.2ms ± 1% 18.8ms ± 1% -6.75% (p=0.000 n=18+18) BM_CommentLines/128/0/0 221ms ± 1% 167ms ± 1% -24.44% (p=0.000 n=20+19) BM_CommentLines/1/30/0 16.6ms ± 3% 16.5ms ± 3% ~ (p=0.175 n=19+20) BM_CommentLines/4/30/0 26.1ms ± 1% 24.8ms ± 2% -5.05% (p=0.000 n=18+19) BM_CommentLines/128/30/0 233ms ± 1% 185ms ± 1% -20.38% (p=0.000 n=19+20) BM_CommentLines/1/70/0 19.2ms ± 1% 19.0ms ± 2% -0.66% (p=0.016 n=19+20) BM_CommentLines/4/70/0 27.9ms ± 1% 26.6ms ± 1% -4.63% (p=0.000 n=19+19) BM_CommentLines/128/70/0 251ms ± 1% 213ms ± 1% -15.18% (p=0.000 n=20+18) BM_CommentLines/1/0/2 15.9ms ± 1% 15.8ms ± 2% ~ (p=0.061 n=19+19) BM_CommentLines/4/0/2 20.5ms ± 2% 19.0ms ± 2% -7.53% (p=0.000 n=20+20) BM_CommentLines/128/0/2 213ms ± 1% 153ms ± 1% -28.18% (p=0.000 n=19+20) BM_CommentLines/1/30/2 16.8ms ± 2% 16.7ms ± 3% ~ (p=0.134 n=20+20) BM_CommentLines/4/30/2 26.6ms ± 1% 25.2ms ± 3% -5.50% (p=0.000 n=20+20) BM_CommentLines/128/30/2 238ms ± 1% 187ms ± 2% -21.49% (p=0.000 n=17+19) BM_CommentLines/1/70/2 19.3ms ± 1% 19.4ms ± 3% ~ (p=0.407 n=17+20) BM_CommentLines/4/70/2 28.2ms ± 1% 26.9ms ± 2% -4.70% (p=0.000 n=19+19) BM_CommentLines/128/70/2 257ms ± 2% 214ms ± 1% -16.52% (p=0.000 n=20+18) BM_CommentLines/1/0/8 16.3ms ± 2% 16.1ms ± 2% -1.22% (p=0.001 n=20+20) BM_CommentLines/4/0/8 22.7ms ± 2% 20.4ms ± 2% -10.20% (p=0.000 n=20+20) BM_CommentLines/128/0/8 244ms ± 1% 153ms ± 1% -37.26% (p=0.000 n=20+18) BM_CommentLines/1/30/8 17.3ms ± 2% 17.2ms ± 3% ~ (p=0.192 n=20+20) BM_CommentLines/4/30/8 28.0ms ± 2% 25.6ms ± 3% -8.46% (p=0.000 n=19+18) BM_CommentLines/128/30/8 272ms ± 1% 196ms ± 2% -27.90% (p=0.000 n=18+20) BM_CommentLines/1/70/8 19.9ms ± 2% 19.9ms ± 2% ~ (p=0.531 n=20+19) BM_CommentLines/4/70/8 29.3ms ± 1% 27.3ms ± 1% -6.87% (p=0.000 n=19+19) BM_CommentLines/128/70/8 292ms ± 1% 228ms ± 1% -21.97% (p=0.000 n=20+19) ``` --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Chandler Carruth
|
7371354dc7 |
Consolidate indent handling to following newlines. (#3298)
The biggest advantage of this is reducing the repeated code in every non-whitespace code path of the lexer to set indent appropriately. Now we handle it cleanly at the start and after vertical whitespace. While this makes the generated code for all the other paths through the lexer quite a bit nicer, it doesn't actually move performance in interesting ways outside of making large blocks of blank lines slightly slower. While there are lots of fluctuations in the benchmark data, they mostly seem to be either noise or artifacts of loop alignment and not really due to an important change here. Raw benchmark data: ``` BM_ValidKeywords 3.10ms ± 1% 3.13ms ± 1% +1.02% (p=0.000 n=20+19) BM_ValidIdentifiers<1, 64, false> 10.8ms ± 3% 10.7ms ± 3% ~ (p=0.383 n=20+20) BM_ValidIdentifiers<1, 1, true> 3.71ms ± 1% 3.67ms ± 2% -1.05% (p=0.000 n=19+19) BM_ValidIdentifiers<3, 5, true> 13.1ms ± 2% 13.0ms ± 2% ~ (p=0.120 n=20+19) BM_ValidIdentifiers<3, 16, true> 13.1ms ± 2% 13.2ms ± 2% ~ (p=0.091 n=20+20) BM_ValidIdentifiers<12, 64, true> 15.1ms ± 1% 15.1ms ± 1% ~ (p=0.138 n=19+19) BM_HorizontalWhitespace/1 13.2ms ± 3% 13.2ms ± 0% ~ (p=0.458 n=20+15) BM_HorizontalWhitespace/4 13.4ms ± 3% 13.3ms ± 1% -1.12% (p=0.000 n=20+19) BM_HorizontalWhitespace/16 14.1ms ± 2% 14.0ms ± 2% -0.89% (p=0.010 n=20+20) BM_HorizontalWhitespace/64 17.9ms ± 2% 17.8ms ± 1% -0.73% (p=0.002 n=20+16) BM_HorizontalWhitespace/128 24.2ms ± 2% 24.1ms ± 1% ~ (p=0.346 n=19+17) BM_RandomSource 7.88ms ± 2% 7.83ms ± 3% ~ (p=0.052 n=20+20) BM_GroupingSymbols/1/0/0 6.25ms ± 2% 6.38ms ± 1% +1.99% (p=0.000 n=20+19) BM_GroupingSymbols/2/0/0 5.24ms ± 2% 5.27ms ± 2% ~ (p=0.065 n=20+19) BM_GroupingSymbols/3/0/0 4.09ms ± 1% 4.07ms ± 1% ~ (p=0.063 n=20+20) BM_GroupingSymbols/4/0/0 3.80ms ± 1% 3.79ms ± 2% ~ (p=0.820 n=20+20) BM_GroupingSymbols/8/0/0 3.05ms ± 1% 3.09ms ± 1% +1.41% (p=0.000 n=20+20) BM_GroupingSymbols/16/0/0 2.95ms ± 1% 3.01ms ± 1% +1.79% (p=0.000 n=20+20) BM_GroupingSymbols/32/0/0 3.67ms ± 1% 3.77ms ± 1% +2.75% (p=0.000 n=20+20) BM_GroupingSymbols/0/1/0 5.66ms ± 1% 5.61ms ± 1% -0.88% (p=0.000 n=20+18) BM_GroupingSymbols/0/2/0 4.43ms ± 2% 4.40ms ± 1% -0.74% (p=0.005 n=20+17) BM_GroupingSymbols/0/3/0 3.15ms ± 2% 3.12ms ± 2% -0.96% (p=0.002 n=20+19) BM_GroupingSymbols/0/4/0 2.79ms ± 2% 2.77ms ± 2% -0.88% (p=0.005 n=20+20) BM_GroupingSymbols/0/8/0 1.81ms ± 2% 1.79ms ± 2% ~ (p=0.056 n=20+20) BM_GroupingSymbols/0/16/0 1.26ms ± 2% 1.26ms ± 2% ~ (p=0.547 n=20+20) BM_GroupingSymbols/0/32/0 1.05ms ± 1% 0.96ms ± 1% -8.52% (p=0.000 n=19+19) BM_GroupingSymbols/0/0/1 5.68ms ± 2% 5.65ms ± 2% ~ (p=0.126 n=20+18) BM_GroupingSymbols/0/0/2 4.44ms ± 2% 4.40ms ± 2% -0.99% (p=0.001 n=20+18) BM_GroupingSymbols/0/0/3 3.15ms ± 1% 3.13ms ± 1% -0.74% (p=0.005 n=20+18) BM_GroupingSymbols/0/0/4 2.80ms ± 1% 2.77ms ± 2% -1.01% (p=0.000 n=20+19) BM_GroupingSymbols/0/0/8 1.81ms ± 2% 1.79ms ± 2% -0.97% (p=0.006 n=20+20) BM_GroupingSymbols/0/0/16 1.26ms ± 1% 1.26ms ± 2% ~ (p=0.678 n=20+20) BM_GroupingSymbols/0/0/32 1.05ms ± 1% 0.96ms ± 1% -8.59% (p=0.000 n=20+19) BM_GroupingSymbols/32/1/0 3.57ms ± 1% 3.69ms ± 1% +3.39% (p=0.000 n=19+20) BM_GroupingSymbols/32/2/0 3.49ms ± 1% 3.60ms ± 1% +3.16% (p=0.000 n=20+20) BM_GroupingSymbols/32/3/0 3.41ms ± 1% 3.52ms ± 1% +3.49% (p=0.000 n=19+20) BM_GroupingSymbols/32/4/0 3.35ms ± 1% 3.45ms ± 1% +3.13% (p=0.000 n=20+20) BM_GroupingSymbols/32/8/0 3.12ms ± 1% 3.22ms ± 1% +3.09% (p=0.000 n=19+20) BM_GroupingSymbols/32/16/0 2.75ms ± 1% 2.82ms ± 1% +2.71% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/0 2.24ms ± 2% 2.24ms ± 1% ~ (p=0.311 n=20+17) BM_GroupingSymbols/32/32/1 2.20ms ± 1% 2.23ms ± 1% +1.23% (p=0.000 n=20+20) BM_GroupingSymbols/32/32/2 2.19ms ± 1% 2.21ms ± 1% +1.08% (p=0.000 n=20+20) BM_GroupingSymbols/32/32/3 2.16ms ± 0% 2.19ms ± 1% +1.09% (p=0.000 n=18+20) BM_GroupingSymbols/32/32/4 2.14ms ± 1% 2.19ms ± 1% +2.25% (p=0.000 n=20+20) BM_GroupingSymbols/32/32/8 2.07ms ± 1% 2.11ms ± 1% +2.04% (p=0.000 n=17+20) BM_GroupingSymbols/32/32/16 1.94ms ± 1% 1.98ms ± 1% +1.95% (p=0.000 n=20+20) BM_GroupingSymbols/32/32/32 1.74ms ± 1% 1.77ms ± 1% +1.88% (p=0.000 n=18+20) BM_BlankLines/1 14.2ms ± 1% 14.4ms ± 1% +1.47% (p=0.000 n=18+16) BM_BlankLines/4 17.3ms ± 1% 17.5ms ± 2% +1.32% (p=0.000 n=17+20) BM_BlankLines/16 31.3ms ± 1% 33.0ms ± 2% +5.35% (p=0.000 n=19+20) BM_BlankLines/64 89.6ms ± 1% 101.3ms ± 1% +12.98% (p=0.000 n=20+20) BM_BlankLines/128 167ms ± 3% 185ms ± 1% +11.11% (p=0.000 n=19+20) BM_CommentLines/1/0/0 15.7ms ± 1% 15.8ms ± 1% ~ (p=0.109 n=19+16) BM_CommentLines/4/0/0 19.8ms ± 1% 20.2ms ± 1% +2.12% (p=0.000 n=20+19) BM_CommentLines/128/0/0 208ms ± 1% 221ms ± 1% +6.29% (p=0.000 n=20+19) BM_CommentLines/1/30/0 16.6ms ± 1% 16.6ms ± 1% ~ (p=0.354 n=19+19) BM_CommentLines/4/30/0 25.8ms ± 2% 26.1ms ± 1% +1.39% (p=0.000 n=20+19) BM_CommentLines/128/30/0 222ms ± 2% 232ms ± 1% +4.46% (p=0.000 n=20+18) BM_CommentLines/1/70/0 19.2ms ± 2% 19.1ms ± 1% ~ (p=0.478 n=20+19) BM_CommentLines/4/70/0 27.6ms ± 2% 27.9ms ± 1% +0.95% (p=0.001 n=20+18) BM_CommentLines/128/70/0 245ms ± 2% 250ms ± 1% +1.96% (p=0.000 n=20+18) BM_CommentLines/1/0/2 16.0ms ± 1% 15.9ms ± 2% -0.68% (p=0.015 n=15+18) BM_CommentLines/4/0/2 20.6ms ± 1% 20.5ms ± 1% -0.53% (p=0.024 n=18+19) BM_CommentLines/128/0/2 216ms ± 2% 213ms ± 1% -1.70% (p=0.000 n=17+19) BM_CommentLines/1/30/2 16.9ms ± 2% 16.8ms ± 3% ~ (p=0.072 n=20+20) BM_CommentLines/4/30/2 26.8ms ± 2% 26.5ms ± 1% -0.91% (p=0.000 n=20+19) BM_CommentLines/128/30/2 244ms ± 2% 238ms ± 1% -2.30% (p=0.000 n=19+17) BM_CommentLines/1/70/2 19.4ms ± 2% 19.4ms ± 2% ~ (p=0.059 n=18+20) BM_CommentLines/4/70/2 28.4ms ± 2% 28.2ms ± 1% -0.81% (p=0.003 n=20+19) BM_CommentLines/128/70/2 261ms ± 2% 255ms ± 1% -2.33% (p=0.000 n=20+18) BM_CommentLines/1/0/8 16.2ms ± 2% 16.2ms ± 2% ~ (p=0.904 n=20+20) BM_CommentLines/4/0/8 25.0ms ± 1% 22.6ms ± 1% -9.40% (p=0.000 n=20+19) BM_CommentLines/128/0/8 246ms ± 2% 244ms ± 1% -1.14% (p=0.000 n=20+19) BM_CommentLines/1/30/8 17.3ms ± 2% 17.3ms ± 1% ~ (p=1.000 n=19+18) BM_CommentLines/4/30/8 30.3ms ± 2% 27.9ms ± 1% -7.83% (p=0.000 n=20+20) BM_CommentLines/128/30/8 277ms ± 3% 271ms ± 1% -2.26% (p=0.000 n=20+17) BM_CommentLines/1/70/8 19.9ms ± 2% 19.9ms ± 2% ~ (p=0.687 n=19+20) BM_CommentLines/4/70/8 31.7ms ± 2% 29.3ms ± 1% -7.55% (p=0.000 n=20+18) BM_CommentLines/128/70/8 296ms ± 2% 291ms ± 1% -1.50% (p=0.000 n=20+18) ``` |
||
|
Richard SmithandChandler Carruth
|
e4caf7d604 |
Compute and cache the value representation of a type when it becomes complete. (#3271)
Using the computed value representation, fix lowering of struct and tuple values to use the value representation rather than the object representation. Fixes an issue found in the review of #3257. This currently causes us to compute value representations of all types as they are created, which generates substantially more SemIR to represent types. We can get some of that back by deferring computation of the value representation until the type is required to be complete, but some of the additional cost here will persist with this approach. I also considered making the computation of the value representation type be something that lives entirely within the lowering phase, but I think that's not the right approach in the longer term, because the value representation will be semantically visible and relevant once we start allowing it to be customized. We should consider moving the nodes that exist to compute canonical non-local types, including value representations, out into a separate global block. That will clean up the SemIR representation substantially, and make the SemIR produced for a function not depend on which types we happen to have encountered beforehand. But that's not being done in this PR. --------- Co-authored-by: Chandler Carruth <chandlerc@gmail.com> |
||
|
Richard Smith
|
1ae5fe0cd5 |
Provide the callee expression to the Call node. (#3291)
Track the callee expression in full, instead of only tracking the callee's FunctionId. This results in the `name_reference` denoting the function actually being used. Lowering now propagates a `llvm::Function*` as the value associated with expressions of type `<function>`. We were not creating `NameReference` node for names produced by member access into a namespace, such as the second name in `Namespace.Function`, which caused lowering of calls to such names to fail. This is now fixed, but the resulting `NameReference` node only refers to the name and the lookup result, not to the `Namespace.` qualifier. We'll need to decide how to fit a third operand into that node (perhaps we can stop storing the `name_id`, since it can be derived from the lookup result) but for now the qualifier is not tracked. |
||
|
Richard Smith
|
c61b6a0e1d |
Rename //toolchain/sem_ir:file_test -> :yaml_test. (#3293)
While this test is testing //toolchain/sem_ir:file, the interesting thing about it is that it's testing the YAML output from File printing. Calling this test `:file_test` is also deeply confusing because that's also the name we give to all of our tests that are testing the files in `testdata/`, which is not what this test does. |
||
 josh11bandRichard Smith
|
a112f2e802 |
Validate parse nodes correspond to expected tokens (#3295)
Can specify which tokens are allowed generally, and any additional tokens that only occur when the parse node has an error. --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Chandler Carruth
|
95a1cc8cba |
Tweak lexer to improve generated code. (#3296)
This is a collection of tweaks and they are ones I'm less confident in FWIW. I set out to make the generated code cleaner and reduce loading pointers through pointers in a bunch of cases. It also works to reduce the working-set-size, and reduce the set of mutated values on each iteration. But I didn't benchmark at each step and it's not clear that incremental benchmarks will even be meaningful, so its hard to say which tweaks were load bearing and which weren't. For example, with this, the core lexer dispatch loop doesn't mutate anything in memory -- the position is passed in register and incremented in register throughout. The pointer to the source text, the size, and the pointer to the lexer are also passed in registers. By making the tokenized buffer a direct member of the lexer, all of its members can be accessed at a constant offset from the lexer pointer itself. The cost of this is a move at the end of lexing, but especially as the buffer size gets large this seems like a trivial cost compared to the previous double-indirection. This uses a (significantly) cheaper representation for the line index -- the `Line` type is optimized for dense *storage*, and is not a great type for using in a tight loop like the lexer. Once using a good index and once the line table is readily available from the above, we can simply index the line table rather than store (and update) a separate pointer. Last but not least, this removes the column as suggested in a previous review. Now it is computed from the position and the line information. My macro benchmark looks like it improves in the 2% - 5% range, but its getting into the noise sadly (or maybe this is good?). On a quiet AMD server with 20 runs before/after I get reasonably compelling across-the-board improvements on all the microbenchmarks, including 5% on `RandomSource` which is key: ``` BM_ValidKeywords 3.44ms ± 1% 3.11ms ± 1% -9.52% (p=0.000 n=19+20) BM_ValidIdentifiers<1, 64, false> 11.2ms ± 1% 10.8ms ± 2% -3.23% (p=0.000 n=19+20) BM_ValidIdentifiers<1, 1, true> 3.96ms ± 1% 3.72ms ± 1% -6.10% (p=0.000 n=18+20) BM_ValidIdentifiers<3, 5, true> 13.5ms ± 1% 13.2ms ± 3% -2.45% (p=0.000 n=19+20) BM_ValidIdentifiers<3, 16, true> 13.5ms ± 2% 13.1ms ± 2% -2.81% (p=0.000 n=19+20) BM_ValidIdentifiers<12, 64, true> 15.6ms ± 1% 15.2ms ± 3% -2.41% (p=0.000 n=19+20) BM_HorizontalWhitespace/1 13.6ms ± 2% 13.3ms ± 3% -2.34% (p=0.000 n=19+20) BM_HorizontalWhitespace/4 13.9ms ± 2% 13.6ms ± 3% -2.06% (p=0.000 n=19+20) BM_HorizontalWhitespace/16 14.5ms ± 2% 14.3ms ± 2% -1.17% (p=0.002 n=19+20) BM_HorizontalWhitespace/64 18.5ms ± 3% 18.1ms ± 2% -2.17% (p=0.000 n=19+19) BM_HorizontalWhitespace/128 24.7ms ± 1% 24.4ms ± 2% -1.29% (p=0.000 n=18+20) BM_RandomSource 8.38ms ± 2% 7.92ms ± 2% -5.50% (p=0.000 n=19+20) BM_GroupingSymbols/1/0/0 6.63ms ± 2% 6.30ms ± 2% -4.97% (p=0.000 n=19+20) BM_GroupingSymbols/2/0/0 5.65ms ± 2% 5.27ms ± 2% -6.75% (p=0.000 n=19+20) BM_GroupingSymbols/3/0/0 4.51ms ± 1% 4.12ms ± 1% -8.62% (p=0.000 n=19+18) BM_GroupingSymbols/4/0/0 4.23ms ± 1% 3.82ms ± 1% -9.65% (p=0.000 n=19+18) BM_GroupingSymbols/8/0/0 3.50ms ± 1% 3.08ms ± 1% -12.21% (p=0.000 n=19+19) BM_GroupingSymbols/16/0/0 3.38ms ± 1% 2.97ms ± 1% -11.98% (p=0.000 n=19+19) BM_GroupingSymbols/32/0/0 4.10ms ± 1% 3.69ms ± 1% -10.02% (p=0.000 n=17+19) BM_GroupingSymbols/0/1/0 5.85ms ± 2% 5.70ms ± 1% -2.54% (p=0.000 n=19+20) BM_GroupingSymbols/0/2/0 4.60ms ± 2% 4.46ms ± 1% -3.12% (p=0.000 n=19+19) BM_GroupingSymbols/0/3/0 3.32ms ± 2% 3.18ms ± 1% -4.21% (p=0.000 n=18+20) BM_GroupingSymbols/0/4/0 2.94ms ± 2% 2.82ms ± 1% -4.22% (p=0.000 n=19+20) BM_GroupingSymbols/0/8/0 1.95ms ± 1% 1.83ms ± 1% -6.47% (p=0.000 n=17+19) BM_GroupingSymbols/0/16/0 1.40ms ± 1% 1.27ms ± 1% -9.12% (p=0.000 n=19+20) BM_GroupingSymbols/0/32/0 1.19ms ± 1% 1.05ms ± 1% -11.50% (p=0.000 n=19+20) BM_GroupingSymbols/0/0/1 5.88ms ± 2% 5.71ms ± 1% -2.87% (p=0.000 n=19+20) BM_GroupingSymbols/0/0/2 4.62ms ± 1% 4.45ms ± 2% -3.58% (p=0.000 n=18+20) BM_GroupingSymbols/0/0/3 3.32ms ± 1% 3.17ms ± 1% -4.72% (p=0.000 n=16+17) BM_GroupingSymbols/0/0/4 2.95ms ± 1% 2.81ms ± 2% -4.50% (p=0.000 n=16+19) BM_GroupingSymbols/0/0/8 1.96ms ± 1% 1.82ms ± 2% -6.80% (p=0.000 n=19+20) BM_GroupingSymbols/0/0/16 1.40ms ± 1% 1.27ms ± 1% -8.96% (p=0.000 n=19+20) BM_GroupingSymbols/0/0/32 1.19ms ± 1% 1.05ms ± 1% -11.64% (p=0.000 n=19+20) BM_GroupingSymbols/32/1/0 4.04ms ± 1% 3.60ms ± 1% -10.81% (p=0.000 n=17+20) BM_GroupingSymbols/32/2/0 3.93ms ± 1% 3.51ms ± 1% -10.55% (p=0.000 n=18+20) BM_GroupingSymbols/32/3/0 3.84ms ± 1% 3.44ms ± 1% -10.38% (p=0.000 n=18+19) BM_GroupingSymbols/32/4/0 3.76ms ± 2% 3.38ms ± 1% -10.21% (p=0.000 n=19+20) BM_GroupingSymbols/32/8/0 3.51ms ± 1% 3.14ms ± 1% -10.61% (p=0.000 n=19+19) BM_GroupingSymbols/32/16/0 3.11ms ± 1% 2.77ms ± 1% -10.95% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/0 2.54ms ± 2% 2.25ms ± 1% -11.77% (p=0.000 n=19+16) BM_GroupingSymbols/32/32/1 2.50ms ± 1% 2.22ms ± 2% -11.19% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/2 2.48ms ± 1% 2.21ms ± 1% -11.15% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/3 2.46ms ± 1% 2.18ms ± 1% -11.11% (p=0.000 n=18+20) BM_GroupingSymbols/32/32/4 2.43ms ± 1% 2.16ms ± 1% -11.04% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/8 2.35ms ± 1% 2.09ms ± 1% -11.11% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/16 2.20ms ± 1% 1.96ms ± 1% -11.30% (p=0.000 n=19+18) BM_GroupingSymbols/32/32/32 1.98ms ± 1% 1.75ms ± 1% -11.54% (p=0.000 n=19+19) BM_BlankLines/1 14.6ms ± 1% 14.2ms ± 3% -2.53% (p=0.000 n=19+19) BM_BlankLines/4 18.3ms ± 1% 17.4ms ± 2% -4.68% (p=0.000 n=19+20) BM_BlankLines/16 34.9ms ± 2% 31.6ms ± 3% -9.42% (p=0.000 n=19+20) BM_BlankLines/64 102ms ± 2% 91ms ± 3% -10.75% (p=0.000 n=19+20) BM_BlankLines/128 190ms ± 2% 169ms ± 3% -10.82% (p=0.000 n=19+20) BM_CommentLines/1/0/0 16.2ms ± 1% 15.8ms ± 2% -2.29% (p=0.000 n=19+19) BM_CommentLines/4/0/0 21.0ms ± 1% 19.9ms ± 2% -5.17% (p=0.000 n=19+19) BM_CommentLines/128/0/0 237ms ± 1% 211ms ± 2% -10.93% (p=0.000 n=19+20) BM_CommentLines/1/30/0 17.0ms ± 1% 16.8ms ± 3% -1.36% (p=0.001 n=17+20) BM_CommentLines/4/30/0 27.2ms ± 2% 26.0ms ± 2% -4.24% (p=0.000 n=19+20) BM_CommentLines/128/30/0 255ms ± 2% 223ms ± 2% -12.58% (p=0.000 n=19+20) BM_CommentLines/1/70/0 19.7ms ± 1% 19.2ms ± 1% -2.35% (p=0.000 n=18+18) BM_CommentLines/4/70/0 29.0ms ± 2% 27.6ms ± 1% -4.81% (p=0.000 n=18+17) BM_CommentLines/128/70/0 273ms ± 7% 249ms ± 3% -9.05% (p=0.000 n=20+20) BM_CommentLines/1/0/2 16.4ms ± 2% 16.0ms ± 1% -2.49% (p=0.000 n=19+15) BM_CommentLines/4/0/2 22.0ms ± 2% 20.7ms ± 1% -5.84% (p=0.000 n=19+17) BM_CommentLines/128/0/2 256ms ± 2% 218ms ± 1% -14.88% (p=0.000 n=19+17) BM_CommentLines/1/30/2 17.4ms ± 1% 17.0ms ± 2% -2.26% (p=0.000 n=19+19) BM_CommentLines/4/30/2 28.5ms ± 1% 26.9ms ± 2% -5.67% (p=0.000 n=19+18) BM_CommentLines/128/30/2 288ms ± 5% 246ms ± 3% -14.70% (p=0.000 n=19+20) BM_CommentLines/1/70/2 19.9ms ± 1% 19.4ms ± 1% -2.56% (p=0.000 n=19+18) BM_CommentLines/4/70/2 30.0ms ± 2% 28.5ms ± 2% -5.20% (p=0.000 n=19+19) BM_CommentLines/128/70/2 300ms ± 3% 265ms ± 3% -11.69% (p=0.000 n=18+20) BM_CommentLines/1/0/8 16.7ms ± 2% 16.3ms ± 2% -2.22% (p=0.000 n=19+19) BM_CommentLines/4/0/8 26.4ms ± 1% 25.2ms ± 1% -4.42% (p=0.000 n=19+18) BM_CommentLines/128/0/8 285ms ± 2% 248ms ± 1% -13.16% (p=0.000 n=19+18) BM_CommentLines/1/30/8 17.9ms ± 2% 17.4ms ± 1% -2.99% (p=0.000 n=19+17) BM_CommentLines/4/30/8 32.0ms ± 1% 30.5ms ± 2% -4.73% (p=0.000 n=19+20) BM_CommentLines/128/30/8 320ms ± 2% 280ms ± 3% -12.52% (p=0.000 n=18+20) BM_CommentLines/1/70/8 20.7ms ± 2% 20.0ms ± 2% -3.22% (p=0.000 n=19+19) BM_CommentLines/4/70/8 33.5ms ± 2% 31.8ms ± 1% -5.23% (p=0.000 n=19+19) BM_CommentLines/128/70/8 338ms ± 3% 298ms ± 2% -11.64% (p=0.000 n=19+20) ``` |
||
|
Richard Smith
|
4e64b1948d |
Bare-bones support for forward-declared classes. (#3294)
This is mostly scaffolding, but is just about enough for pointers to classes to work properly as types. |
||
|
Chandler Carruth
|
0e2b6c7f1a |
Optimize runs of horizontal whitespace. (#3288)
So, this is a somewhat fun, simple improvement. =] Just use a loop and count runs of whitespace. I didn't even work all that hard to make the loop fast, but it seems great. Makes long runs of indentation more than 2x faster at basically no code complexity. I thought about doing this for vertical whitespace as well but it's not easy to do, and didn't seem worth adding complexity. Huge runs of blank lines aren't nearly as common as lots of indentation. I do have a plan for an analogous optimization for comment blocks, but want to simplify some other code first. We could also make this (hilariously) faster with some judicious use of SIMD or clever use of a string function, but it doesn't seem worth it given how fast the simple loop is already. I didn't work to get a high N count and so there's plenty of noise here, but the benchmark data speaks for itself: ``` BM_ValidKeywords 2.71ms ± 1% 2.75ms ± 1% +1.32% (p=0.016 n=5+5) BM_ValidIdentifiers<1, 64, false> 9.71ms ± 2% 9.74ms ± 1% ~ (p=1.000 n=5+5) BM_ValidIdentifiers<1, 1, true> 3.17ms ± 1% 3.22ms ± 2% ~ (p=0.151 n=5+5) BM_ValidIdentifiers<3, 5, true> 11.5ms ± 2% 11.6ms ± 3% ~ (p=0.548 n=5+5) BM_ValidIdentifiers<3, 16, true> 11.3ms ± 3% 11.6ms ± 3% ~ (p=0.095 n=5+5) BM_ValidIdentifiers<12, 64, true> 12.8ms ± 1% 12.8ms ± 1% ~ (p=1.000 n=5+5) BM_HorizontalWhitespace/1 11.6ms ± 1% 11.7ms ± 2% ~ (p=0.310 n=5+5) BM_HorizontalWhitespace/4 12.9ms ± 1% 11.8ms ± 0% -8.50% (p=0.008 n=5+5) BM_HorizontalWhitespace/16 17.3ms ± 4% 12.5ms ± 1% -27.69% (p=0.008 n=5+5) BM_HorizontalWhitespace/64 28.9ms ± 3% 16.0ms ± 2% -44.88% (p=0.008 n=5+5) BM_HorizontalWhitespace/128 47.6ms ± 3% 21.0ms ± 0% -55.88% (p=0.016 n=5+4) BM_RandomSource 7.92ms ± 1% 7.70ms ± 3% -2.74% (p=0.016 n=5+5) BM_GroupingSymbols/1/0/0 5.92ms ± 2% 5.86ms ± 1% ~ (p=0.310 n=5+5) BM_GroupingSymbols/2/0/0 5.30ms ± 1% 5.01ms ± 1% -5.52% (p=0.008 n=5+5) BM_GroupingSymbols/3/0/0 4.48ms ± 0% 3.95ms ± 1% -11.69% (p=0.008 n=5+5) BM_GroupingSymbols/4/0/0 4.61ms ± 1% 3.73ms ± 1% -19.12% (p=0.008 n=5+5) BM_GroupingSymbols/8/0/0 5.34ms ± 6% 3.05ms ± 1% -42.82% (p=0.008 n=5+5) BM_GroupingSymbols/16/0/0 6.44ms ± 1% 3.20ms ± 2% -50.24% (p=0.008 n=5+5) BM_GroupingSymbols/32/0/0 10.3ms ± 5% 4.2ms ± 1% -59.81% (p=0.008 n=5+5) BM_GroupingSymbols/0/1/0 5.17ms ± 2% 5.17ms ± 1% ~ (p=0.690 n=5+5) BM_GroupingSymbols/0/2/0 4.04ms ± 1% 4.04ms ± 2% ~ (p=1.000 n=5+5) BM_GroupingSymbols/0/3/0 2.89ms ± 1% 2.89ms ± 1% ~ (p=1.000 n=5+5) BM_GroupingSymbols/0/4/0 2.54ms ± 1% 2.55ms ± 1% ~ (p=0.421 n=5+5) BM_GroupingSymbols/0/8/0 1.65ms ± 2% 1.67ms ± 1% ~ (p=0.310 n=5+5) BM_GroupingSymbols/0/16/0 1.18ms ± 1% 1.19ms ± 2% ~ (p=0.222 n=5+5) BM_GroupingSymbols/0/32/0 948µs ± 1% 957µs ± 2% ~ (p=0.310 n=5+5) BM_GroupingSymbols/0/0/1 5.17ms ± 1% 5.16ms ± 2% ~ (p=0.841 n=5+5) BM_GroupingSymbols/0/0/2 4.03ms ± 2% 4.04ms ± 2% ~ (p=0.548 n=5+5) BM_GroupingSymbols/0/0/3 2.89ms ± 1% 2.88ms ± 1% ~ (p=0.841 n=5+5) BM_GroupingSymbols/0/0/4 2.54ms ± 1% 2.54ms ± 2% ~ (p=0.548 n=5+5) BM_GroupingSymbols/0/0/8 1.66ms ± 3% 1.67ms ± 2% ~ (p=0.690 n=5+5) BM_GroupingSymbols/0/0/16 1.19ms ± 1% 1.18ms ± 2% ~ (p=0.548 n=5+5) BM_GroupingSymbols/0/0/32 949µs ± 2% 955µs ± 1% ~ (p=0.310 n=5+5) BM_GroupingSymbols/32/1/0 10.0ms ± 1% 4.0ms ± 1% -59.76% (p=0.008 n=5+5) BM_GroupingSymbols/32/2/0 9.73ms ± 2% 3.92ms ± 2% -59.73% (p=0.008 n=5+5) BM_GroupingSymbols/32/3/0 9.40ms ± 2% 3.84ms ± 3% -59.10% (p=0.008 n=5+5) BM_GroupingSymbols/32/4/0 9.20ms ± 2% 3.74ms ± 1% -59.35% (p=0.008 n=5+5) BM_GroupingSymbols/32/8/0 8.38ms ± 2% 3.50ms ± 1% -58.22% (p=0.008 n=5+5) BM_GroupingSymbols/32/16/0 7.18ms ± 2% 3.06ms ± 3% -57.41% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/0 5.53ms ± 1% 2.44ms ± 2% -55.99% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/1 5.47ms ± 2% 2.40ms ± 2% -56.08% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/2 5.41ms ± 3% 2.40ms ± 2% -55.66% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/3 5.28ms ± 2% 2.37ms ± 2% -55.18% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/4 5.25ms ± 2% 2.34ms ± 2% -55.52% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/8 5.03ms ± 3% 2.25ms ± 1% -55.29% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/16 4.62ms ± 1% 2.15ms ± 3% -53.41% (p=0.008 n=5+5) BM_GroupingSymbols/32/32/32 3.99ms ± 1% 1.86ms ± 1% -53.24% (p=0.008 n=5+5) BM_BlankLines/1 12.8ms ± 1% 12.6ms ± 2% ~ (p=0.310 n=5+5) BM_BlankLines/4 16.0ms ± 3% 15.8ms ± 1% ~ (p=0.310 n=5+5) BM_BlankLines/16 33.1ms ± 2% 32.1ms ± 1% -3.29% (p=0.008 n=5+5) BM_BlankLines/64 84.6ms ± 3% 84.2ms ± 3% ~ (p=0.690 n=5+5) BM_BlankLines/128 159ms ± 4% 155ms ± 3% ~ (p=0.310 n=5+5) BM_CommentLines/1/0/0 14.5ms ± 2% 14.5ms ± 2% ~ (p=0.690 n=5+5) BM_CommentLines/4/0/0 19.0ms ± 1% 19.0ms ± 1% ~ (p=0.548 n=5+5) BM_CommentLines/128/0/0 188ms ± 2% 186ms ± 1% ~ (p=0.151 n=5+5) BM_CommentLines/1/30/0 14.8ms ± 3% 14.7ms ± 2% ~ (p=0.421 n=5+5) BM_CommentLines/4/30/0 21.8ms ± 1% 21.3ms ± 4% ~ (p=0.095 n=5+5) BM_CommentLines/128/30/0 201ms ± 1% 200ms ± 2% ~ (p=0.421 n=5+5) BM_CommentLines/1/70/0 15.5ms ± 3% 15.2ms ± 4% ~ (p=0.095 n=5+5) BM_CommentLines/4/70/0 23.2ms ± 1% 22.5ms ± 2% -2.76% (p=0.008 n=5+5) BM_CommentLines/128/70/0 213ms ± 1% 209ms ± 1% -1.54% (p=0.016 n=5+5) BM_CommentLines/1/0/2 15.3ms ± 1% 14.5ms ± 1% -4.99% (p=0.008 n=5+5) BM_CommentLines/4/0/2 21.4ms ± 1% 20.2ms ± 2% -5.48% (p=0.008 n=5+5) BM_CommentLines/128/0/2 242ms ± 2% 218ms ± 5% -10.13% (p=0.008 n=5+5) BM_CommentLines/1/30/2 15.7ms ± 4% 15.1ms ± 2% -3.37% (p=0.008 n=5+5) BM_CommentLines/4/30/2 24.3ms ± 1% 22.5ms ± 3% -7.42% (p=0.008 n=5+5) BM_CommentLines/128/30/2 268ms ± 2% 240ms ± 3% -10.22% (p=0.008 n=5+5) BM_CommentLines/1/70/2 16.1ms ± 2% 15.3ms ± 3% -5.24% (p=0.008 n=5+5) BM_CommentLines/4/70/2 25.7ms ± 3% 24.0ms ± 3% -6.69% (p=0.008 n=5+5) BM_CommentLines/128/70/2 272ms ± 1% 247ms ± 1% -9.21% (p=0.008 n=5+5) BM_CommentLines/1/0/8 17.2ms ± 5% 14.8ms ± 2% -14.32% (p=0.008 n=5+5) BM_CommentLines/4/0/8 30.4ms ± 2% 20.8ms ± 2% -31.47% (p=0.008 n=5+5) BM_CommentLines/128/0/8 463ms ± 1% 260ms ± 1% -43.89% (p=0.008 n=5+5) BM_CommentLines/1/30/8 17.2ms ± 2% 15.2ms ± 2% -12.01% (p=0.008 n=5+5) BM_CommentLines/4/30/8 32.4ms ± 1% 23.2ms ± 3% -28.61% (p=0.008 n=5+5) BM_CommentLines/128/30/8 498ms ± 3% 287ms ± 2% -42.32% (p=0.008 n=5+5) BM_CommentLines/1/70/8 17.6ms ± 2% 15.5ms ± 3% -11.61% (p=0.008 n=5+5) BM_CommentLines/4/70/8 34.0ms ± 4% 24.7ms ± 3% -27.31% (p=0.008 n=5+5) BM_CommentLines/128/70/8 497ms ± 2% 297ms ± 4% -40.29% (p=0.008 n=5+5) ``` Stacked on top of #3287 -- only the last commit should be reviewed here. |
||
|
Richard Smith
|
870ccf3eac |
Suppress follow-on errors for various forms of expression. (#3292)
Don't produce an error diagnostic if any of the information leading to identifying that error is itself known to be affected by an already-diagnosed error. |
||
|
Chandler Carruth
|
a79ea4b28d |
Do more precise table dispatch for symbols. (#3287)
When originally switching to the table dispatch approach we discussed that it'd be nice to disentangle the monolithic symbol lexing routine with this as we'll typically have fairly precise dispatch. This is especially true for grouping symbols, which in Carbon are all constructively one-character (at this point). I think this provides a substantial improvement to the clarity of the code by disentangling the different paths. It also allowed a bunch of simplifications / clarifications to exactly what the behavior with closing invalid groups actually involves currently. This was initially motivated by code organization improvements, and any performance wins were speculative. However, when benchmarking it surfaced a problem that hadn't been clear -- we're generating too many distinct functions here, and the table-based dispatch slows down in the face of that. So this PR also includes a fix for that, removing the template-generated fan-out of dispatch functions for distinct symbols. Instead, we have a dedicated table to translate one character into the token kinds. This seems to work quite well, avoiding the huge branch-y structure and just do fairly cheap table translation & dispatch for all one-character symbols. Building the table requires the token kinds to be default constructable, so this also enables that and arranges for the zero-value kind to be the error kind. Combined, this is a modest speedup for *non* grouping symbols (3-4%, a bit noisy). And in some cases it is a huge speedup for grouping symbols (>10%). Raw benchmark data with 20 runs before/after -- despite the # of runs, the grouping symbols benchmarks were frustratingly noisy in non-uniform ways that couldn't fully be accounted for here. Still, this seems like an overall improvement. ``` BM_RandomSource 7.98ms ± 2% 7.73ms ± 3% -3.12% (p=0.000 n=18+19) BM_GroupingSymbols/1/0/0 5.90ms ± 2% 5.82ms ± 4% -1.38% (p=0.001 n=20+20) BM_GroupingSymbols/2/0/0 5.21ms ± 2% 5.15ms ± 2% -1.14% (p=0.002 n=20+18) BM_GroupingSymbols/3/0/0 4.42ms ± 2% 4.34ms ± 2% -1.87% (p=0.000 n=19+18) BM_GroupingSymbols/4/0/0 4.29ms ± 2% 4.38ms ± 5% ~ (p=0.297 n=17+20) BM_GroupingSymbols/8/0/0 5.09ms ±10% 5.10ms ± 7% ~ (p=0.919 n=18+20) BM_GroupingSymbols/16/0/0 6.35ms ± 8% 6.29ms ± 6% ~ (p=0.201 n=20+20) BM_GroupingSymbols/32/0/0 9.88ms ± 2% 9.83ms ± 1% ~ (p=0.167 n=18+20) BM_GroupingSymbols/0/1/0 5.12ms ± 2% 5.01ms ± 2% -2.14% (p=0.000 n=20+19) BM_GroupingSymbols/0/2/0 4.01ms ± 2% 3.93ms ± 4% -2.03% (p=0.000 n=20+19) BM_GroupingSymbols/0/3/0 2.92ms ± 3% 2.81ms ± 2% -3.87% (p=0.000 n=20+19) BM_GroupingSymbols/0/4/0 2.61ms ± 3% 2.47ms ± 2% -5.30% (p=0.000 n=20+18) BM_GroupingSymbols/0/8/0 1.77ms ± 3% 1.61ms ± 2% -8.91% (p=0.000 n=18+19) BM_GroupingSymbols/0/16/0 1.41ms ± 3% 1.16ms ± 4% -17.66% (p=0.000 n=20+20) BM_GroupingSymbols/0/32/0 1.10ms ± 2% 0.92ms ± 3% -16.36% (p=0.000 n=20+17) BM_GroupingSymbols/0/0/1 5.09ms ± 2% 5.03ms ± 3% -1.11% (p=0.001 n=20+18) BM_GroupingSymbols/0/0/2 4.01ms ± 2% 3.91ms ± 2% -2.67% (p=0.000 n=20+18) BM_GroupingSymbols/0/0/3 2.93ms ± 3% 2.81ms ± 2% -4.23% (p=0.000 n=20+19) BM_GroupingSymbols/0/0/4 2.59ms ± 2% 2.48ms ± 3% -4.48% (p=0.000 n=20+19) BM_GroupingSymbols/0/0/8 1.75ms ± 1% 1.62ms ± 3% -7.65% (p=0.000 n=17+19) BM_GroupingSymbols/0/0/16 1.40ms ± 2% 1.15ms ± 3% -17.67% (p=0.000 n=19+20) BM_GroupingSymbols/0/0/32 1.10ms ± 2% 0.92ms ± 3% -15.91% (p=0.000 n=20+19) BM_GroupingSymbols/32/1/0 9.62ms ± 2% 9.65ms ± 2% ~ (p=0.654 n=18+20) BM_GroupingSymbols/32/2/0 9.41ms ± 2% 9.37ms ± 2% ~ (p=0.095 n=20+19) BM_GroupingSymbols/32/3/0 9.13ms ± 2% 9.13ms ± 3% ~ (p=0.687 n=19+20) BM_GroupingSymbols/32/4/0 8.93ms ± 1% 8.87ms ± 2% -0.69% (p=0.010 n=20+18) BM_GroupingSymbols/32/8/0 8.15ms ± 2% 8.14ms ± 3% ~ (p=0.729 n=19+19) BM_GroupingSymbols/32/16/0 7.04ms ± 3% 6.92ms ± 1% -1.71% (p=0.000 n=20+18) BM_GroupingSymbols/32/32/0 5.48ms ± 2% 5.38ms ± 3% -1.81% (p=0.000 n=20+20) BM_GroupingSymbols/32/32/1 5.39ms ± 2% 5.29ms ± 2% -1.87% (p=0.000 n=19+19) BM_GroupingSymbols/32/32/2 5.34ms ± 2% 5.21ms ± 1% -2.45% (p=0.000 n=20+18) BM_GroupingSymbols/32/32/3 5.27ms ± 3% 5.16ms ± 2% -2.18% (p=0.000 n=20+19) BM_GroupingSymbols/32/32/4 5.21ms ± 2% 5.10ms ± 3% -2.11% (p=0.000 n=19+20) BM_GroupingSymbols/32/32/8 4.98ms ± 2% 4.83ms ± 2% -2.85% (p=0.000 n=19+19) BM_GroupingSymbols/32/32/16 4.55ms ± 2% 4.45ms ± 2% -2.25% (p=0.000 n=18+20) BM_GroupingSymbols/32/32/32 3.95ms ± 2% 3.84ms ± 2% -2.98% (p=0.000 n=19+20) ``` |
||
|
Richard Smith
|
17e053798c |
Fix incorrect argc calculation. (#3290)
Fix off-by-one error: the terminating `nullptr` in `argv` is not included in `argc`. This is currently causing crashing tests to crash again in their crash handler, meaning we don't get a backtrace or CHECK-failure message. |
||
|
Chandler Carruth
|
9ce9572522 |
Add a benchmark to stress test grouping symbols. (#3286)
This isn't really representative of anything, but it should help make it obvious when the handling of grouping symbols improves or regresses. Notably, the random source microbenchmark has *no* grouping symbols (in order to let it be random but always lexically valid), and so it's especially useful to have something that checks grouping symbols given their prevalence in realistic source code. |
||
|
josh11bandRichard Smith
|
8d0831f431 |
Made function and namespace nodes typed to remove a crash (#3285)
Bug found by fuzzing. Problem was untyped SemIR nodes had an invalid type id, which was retrieved by `HandlePrefixOperator` and then passed to `context.GetUnqualifiedType`, ultimately performing an invalid access in `semantics_ir_->GetNode`. We prefer to make a placeholder type for functions and namespaces to remove the need for checking for the untyped case everywhere. Eventually functions will have their own types, but this approach will be needed for namespaces (and perhaps other non-first-class entities like unbound methods and interface members) long term. Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
|
6ba8712fbd |
Predetermine all the line splits in the lexer. (#3278)
## Summary ## Restructures the lexer to first scan the entire source text for newlines and create all the line structures needed. Doing this up-front makes it easy to produce an optimized version with minimal complexity. Currently, it leverages the system `memchr`, but even when expanded to handle more complex cases like CR+LF line endings, being isolated in this way will result in a significantly simpler implementation. This change improves the lexing of comment lines significantly by skipping their contents immediately. The overhead of the pre-scan is unmeasurable in all realistic benchmarks, and 10-30% in benchmarks consisting almost entirely of blank lines or comments. The improvement of comment lexing with average length comment lines mixed with code starts at 20% and goes up. Regressing blank line handling for non-empty comments seems like the right tradeoff (by far). ## Background and details ## One weak point in the lexer implementation were large runs of comments. While those aren't terribly common, they shouldn't present a hazard to the lexer performance. A bit more common is a pattern of comments like the following: ```carbon // Some method comment here. fn SomeMethodName(...) -> ...; // Some other method comment here. fn SomeOtherMethodName(...) -> ...; ``` Here, the lexer spends an inordinate amount of time getting from the `\n` after the first semicolon to the `fn` token. It has to skip a blank line, scan a line, find the `//` comment start, then scan to find the next `\n`, and then scan horizontal whitespace, etc. It is tempting to build a scanner *exactly* for this. In fact, I built one, and I can publish it in a PR if folks are interested in what it looks like. For x86-64, the PSHUFB trick used for scanning identifiers technically works. But it is *complicated*. Amazingly so. 150 lines of very subtle code with subtle performance pitfalls at ever turn. I felt very uncomfortable submitting it, but we can always go back to it. Nothing I've come up with quite matches it for sheer speed. However, most of the complexity and time is spent walking from a `//` to the end of the line. And *that* is something we can do very simply. In fact, there is a tuned function for that in libc: `memchr`. Using this we can build a very fast and much simpler scanner to split lines up-front. This PR uses that and a carefully crafted fast loop to first build up all the line info we need. Getting this to be as fast as possible required some other subtle changes, for example always creating a line structure that goes from the last `\n` and the end of the file. We then back up the EOF token to avoid surfacing this to users. The nice thing is the EOF token isn't part of any hot loop, and so this removes branches everywhere else at modest complexity. Once we have that, the rest of the lexer just needs to keep track of its current line in order to record column offsets. I've taken some care to try and optimize the lexer's usage of the line structures but there are more opportunities here I suspect. Combined, this gets much but not all of the performance of a huge SIMD scanner for newline-through-to-next-token. For extreme cases (100s of blank lines or empty comment lines between tokens) the holistic scanner is of course still much faster, but those don't seem nearly worth the cost. I was initially worried about the overhead of taking two passes over the source text, but in practice I've not been able to measure any appreciable cost to this with realistic source files. In some cases benchmarks with no newlines get *faster* because we use a much more efficient approach to fetch the source text into cache as a happenstance. And that in turn makes the byte-wise dispatched loop run faster as it stalls less. I'm particularly happy with this approach because it seems very clear how to extend this to support CR+LF, bare CR, and even complex mixtures without any significant speed cost. That wasn't at all true for the other approaches explored. I may try some further PRs to smooth out the last bits of slowness here, but already this is working excellent for me in practice. My 10mloc test case is down to 2.3s to lex. ## Raw benchmark data Using a tool that runs benchmarks before and after and analyzes the results, the following summarizes the CPU-time impact, each of these for lexing 100k tokens: ``` BM_ValidKeywords 2.57ms ± 1% 2.58ms ± 0% ~ (p=0.190 n=5+4) BM_ValidIdentifiers<1, 64, false> 9.24ms ± 4% 9.31ms ± 4% ~ (p=0.421 n=5+5) BM_ValidIdentifiers<1, 1, true> 3.05ms ± 4% 3.11ms ± 4% ~ (p=0.222 n=5+5) BM_ValidIdentifiers<3, 5, true> 10.9ms ± 0% 11.1ms ± 1% +1.76% (p=0.016 n=4+5) BM_ValidIdentifiers<3, 16, true> 11.1ms ± 7% 11.0ms ± 1% ~ (p=0.310 n=5+5) BM_ValidIdentifiers<12, 64, true> 12.2ms ± 1% 12.3ms ± 2% ~ (p=0.111 n=4+5) BM_HorizontalWhitespace/1 11.2ms ± 6% 11.1ms ± 2% ~ (p=0.841 n=5+5) BM_HorizontalWhitespace/4 12.0ms ± 3% 12.0ms ± 2% ~ (p=0.548 n=5+5) BM_HorizontalWhitespace/16 16.2ms ± 6% 15.9ms ± 8% ~ (p=0.690 n=5+5) BM_HorizontalWhitespace/64 27.7ms ± 3% 28.4ms ± 3% ~ (p=0.151 n=5+5) BM_HorizontalWhitespace/128 44.3ms ± 1% 45.6ms ± 6% +3.15% (p=0.032 n=5+5) BM_RandomSource 7.75ms ± 2% 7.72ms ± 1% ~ (p=1.000 n=5+5) BM_BlankLines/1 11.7ms ± 1% 12.1ms ± 1% +3.46% (p=0.008 n=5+5) BM_BlankLines/4 14.0ms ± 2% 15.2ms ± 3% +8.12% (p=0.008 n=5+5) BM_BlankLines/16 23.5ms ± 2% 31.1ms ± 4% +32.26% (p=0.008 n=5+5) BM_BlankLines/64 75.3ms ± 1% 81.2ms ± 3% +7.83% (p=0.008 n=5+5) BM_BlankLines/128 133ms ± 3% 150ms ± 2% +12.74% (p=0.008 n=5+5) BM_CommentLines/1/0/0 13.1ms ± 0% 13.7ms ± 1% +5.11% (p=0.008 n=5+5) BM_CommentLines/4/0/0 16.6ms ± 1% 18.2ms ± 4% +9.56% (p=0.008 n=5+5) BM_CommentLines/128/0/0 169ms ± 4% 182ms ± 1% +7.24% (p=0.008 n=5+5) BM_CommentLines/1/30/0 18.7ms ± 5% 14.1ms ± 0% -24.84% (p=0.008 n=5+5) BM_CommentLines/4/30/0 36.5ms ± 6% 20.6ms ± 3% -43.59% (p=0.008 n=5+5) BM_CommentLines/128/30/0 525ms ± 4% 198ms ± 1% -62.38% (p=0.008 n=5+5) BM_CommentLines/1/70/0 23.4ms ± 6% 14.7ms ± 2% -37.15% (p=0.008 n=5+5) BM_CommentLines/4/70/0 53.3ms ± 7% 22.4ms ± 4% -57.99% (p=0.008 n=5+5) BM_CommentLines/128/70/0 1.05s ± 4% 0.21s ± 2% -80.31% (p=0.008 n=5+5) BM_CommentLines/1/0/2 14.1ms ± 6% 14.3ms ± 1% ~ (p=0.151 n=5+5) BM_CommentLines/4/0/2 19.4ms ± 5% 20.1ms ± 1% ~ (p=0.151 n=5+5) BM_CommentLines/128/0/2 238ms ± 8% 229ms ± 0% ~ (p=0.151 n=5+5) BM_CommentLines/1/30/2 19.2ms ± 7% 14.6ms ± 1% -23.87% (p=0.008 n=5+5) BM_CommentLines/4/30/2 40.3ms ±13% 22.3ms ± 4% -44.63% (p=0.008 n=5+5) BM_CommentLines/128/30/2 568ms ± 7% 254ms ± 3% -55.28% (p=0.008 n=5+5) BM_CommentLines/1/70/2 23.3ms ± 1% 15.0ms ± 3% -35.61% (p=0.016 n=4+5) BM_CommentLines/4/70/2 57.2ms ± 9% 24.1ms ± 2% -57.81% (p=0.008 n=5+5) BM_CommentLines/128/70/2 1.07s ± 0% 0.26s ± 2% -75.51% (p=0.016 n=4+5) BM_CommentLines/1/0/8 15.9ms ± 7% 16.0ms ± 1% ~ (p=0.151 n=5+5) BM_CommentLines/4/0/8 24.2ms ± 6% 27.9ms ± 2% +15.36% (p=0.008 n=5+5) BM_CommentLines/128/0/8 386ms ± 4% 445ms ± 1% +15.28% (p=0.008 n=5+5) BM_CommentLines/1/30/8 20.6ms ± 5% 16.3ms ± 1% -20.95% (p=0.008 n=5+5) BM_CommentLines/4/30/8 45.3ms ± 6% 30.3ms ± 3% -32.98% (p=0.008 n=5+5) BM_CommentLines/128/30/8 699ms ± 3% 477ms ± 3% -31.83% (p=0.008 n=5+5) BM_CommentLines/1/70/8 25.7ms ± 5% 16.8ms ± 2% -34.67% (p=0.008 n=5+5) BM_CommentLines/4/70/8 62.0ms ± 4% 31.6ms ± 2% -49.10% (p=0.008 n=5+5) BM_CommentLines/128/70/8 1.20s ± 2% 0.48s ± 4% -59.60% (p=0.008 n=5+5) ``` The horizontal whitespace benchmark (and all of the non-line-oriented ones) are noisier than they appear here but do show some improvements (surprisingly). My guess is that it has a lot to do with system load, as the advantage is that we're using a vectorized loop to scan the text first and then doing the byte-dispatched loop. So when the cache is a bit slower to populate, the vectorized version starts to be faster. --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> Co-authored-by: josh11b <josh11b@users.noreply.github.com> |
||
|
|
5041a14f59 |
Add whitespace- and comment-specific benchmarking. (#3276)
These benchmarks zero-in and stress test horizontal and vertical whitespace as well as comment lexing performance. They set up essentially a worst-case scenario of ramping up whitespace between very sparse tokens to show how the lexer copes with this. The horizontal whitespace benchmark is perhaps less important as frequent runs of 50-characters of horizontal whitespace are relatively rare already, and likely to be exceedingly rare without trailing comments. But its good to include for completeness and it shows reasonably strong performance with the current table-dispatch approach. The blank line and comment line benchmarks are much more important. Lots of code is relatively line-sparse, especially API files that are perhaps the most useful to parse quickly. And many of these are a mixture of sparse with blank lines and sparse with large comment blocks. The benchmarks show that there are some serious limits here, even falling below 100k tokens per second throughput on some of the stress tests here. --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> Co-authored-by: josh11b <josh11b@users.noreply.github.com> |
||
|
Chandler CarruthandRichard Smith
|
2d735bbc51 |
Enhance the main lexer benchmark. (#3275)
This generalizes the main lexer benchmark's source generation to be a bit more comprehensive, specifically including whitespace and comments. Without these, we're missing a key part of the lexer's performance. This also tidies up a bit of the code and adds a more specific distribution of the different factors in lexing based on analysis of LLVM's source code. The enhancements to the source statistics script that helped collect the data here will be in a separate PR. With this, the output on my AMD cloud instance is: ``` ------------------------------------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations bytes_per_second tokens_per_second ------------------------------------------------------------------------------------------------------------------------- BM_ValidKeywords 2809011 ns 2808912 ns 243 212.616M/s 35.601M/s BM_ValidIdentifiers<1, 64, false> 11461783 ns 11461652 ns 61 128.499M/s 8.72475M/s BM_ValidIdentifiers<1, 1, true> 3397293 ns 3397240 ns 208 84.2158M/s 29.4357M/s BM_ValidIdentifiers<3, 5, true> 14165143 ns 14164962 ns 51 40.3956M/s 7.05967M/s BM_ValidIdentifiers<3, 16, true> 15283128 ns 15282583 ns 47 71.7623M/s 6.5434M/s BM_ValidIdentifiers<12, 64, true> 17417323 ns 17417109 ns 41 219.007M/s 5.74148M/s ------------------------------------------------------------------------------------------------------------------------------------------ Benchmark Time CPU Iterations bytes_per_second lines_per_second tokens_per_second ------------------------------------------------------------------------------------------------------------------------------------------ BM_RandomSource 8132211 ns 8132227 ns 84 137.223M/s 3.9036M/s 12.2968M/s BM_SpeedOfLightStrCpy 29610 ns 29608 ns 24631 36.8065G/s 1072.17M/s 3.37745G/s BM_SpeedOfLightDispatch<1> 2144418 ns 2144421 ns 327 520.387M/s 14.8035M/s 46.6326M/s BM_SpeedOfLightDispatch<2> 1945954 ns 1945827 ns 351 573.498M/s 16.3144M/s 51.392M/s BM_SpeedOfLightDispatch<4> 2519565 ns 2519467 ns 292 442.923M/s 12.5999M/s 39.6909M/s BM_SpeedOfLightDispatch<8> 3011965 ns 3011968 ns 238 370.498M/s 10.5396M/s 33.2009M/s BM_SpeedOfLightDispatch<16> 4379575 ns 4379579 ns 160 254.803M/s 7.24841M/s 22.8332M/s BM_SpeedOfLightDispatch<32> 6678423 ns 6678353 ns 102 167.096M/s 4.75342M/s 14.9738M/s BM_SpeedOfLightDispatch<MaxDispatchTargets> 9373075 ns 9372688 ns 75 119.062M/s 3.38697M/s 10.6693M/s ``` I've compared the profile of the `BM_RandomSource` benchmark with this change and it largely corresponds to what I expect based on profiling hand-crafted Carbon inputs. And as you can see, we're closing in on lexing at least hitting the 10-million-lines-per-second mark. =] --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Richard Smithandjosh11b
|
9c3e25decf |
Add SemIR::Node::Is<TypedNode>(). (#3284)
This allows node kind checks to be written a little more concisely. --------- Co-authored-by: josh11b <josh11b@users.noreply.github.com> |
||
|
Richard Smith
|
8779fcd77c |
Switch from SemIR::NodeKind::Foo to SemIR::Foo::Kind wherever possible. (#3283)
This is shorter, more closely connected to code using the typed node types, and avoids using the ambiguous word `Node` in places referring to typed `SemIR` nodes. |
||
|
Richard Smith
|
6e9d83e746 |
Support for while statements. (#3258)
Basic support for type-checking and lowering of `while` statements, as well as `break` and `continue`. |
||
|
Richard Smith
|
c7a9e29a89 |
Add typed nodes to SemIR. (#3280)
Replace `SemIR::Node::GetAsFoo` and `SemIR::Node::Foo::Make` with `SemIR::Foo` class that represents a particular kind of node, with named fields. Rename `SemIR::IntegerLiteral` and `SemIR::RealLiteral` to `IntegerValue` / `RealValue` to better reflect their purpose and avoid a name collision with the corresponding `SemIR` node kinds. Remove `NodeKind::Invalid` and the `SemIR::Node` default constructor entirely, as they were not used for anything. |
||
|
Chandler Carruth
|
0e8e94d8bf |
Expand source stats to roughly cover more tokens. (#3277)
This is pretty rough both in code-quality and what it tries to cover. The coverage approximation is probably fine as it mostly just gives some vague idea of the distributions we should aim at when evaluating performance. We still want performance with unusual distributions to be good. Happy to improve the code quality if folks want, and especially with suggestions on what would help. For this kind of quick scripting, it seemed "fine" to me. |
||
|
Chandler Carruth
|
6f5934a505 |
Unblock more lexer inlining. (#3274)
The big change is to make the lexer helpers have internal linkage, making all of them easy to inline into single call sites. Looking at the profile showed several other cases of unfortunate out-of-line functions. Two were due to the code size produced for checks -- those are switched to `DCHECK`s to remove that code from optimized builds. The loss of coverage seems minor. A last one was closing open groups. This was a surprising routine to be hot, but it the paths to discover "nothing to do here" were intertwined into the code. This PR extracts this common trace into a separate function that delegates to the looping recovery path. This lets the hot path inline easily. At this point, for a large lexing benchmark I'm using, 50% of the time is in the identifier hash table at this point. The remaining improvements are to actually make some of the hot routines like symbol lexing and comment lexing faster. |
||
|
Chandler Carruth
|
d1e3749aa8 |
Workaround issues with sanitizers on macOS. (#3279)
The latest versions of Clang, at least on an ARM mac, enforce that we not use static sanitizer runtimes. The flag is also a bit frustrating: it has to be split out of other flags in order to remove it, you can't just disable it or ignore a warning about it being unused. This gets things to be build and run again. However, the runtimes (I'm guessing ones that ship with the OS?) its using dynamically don't actually work -- at least one check was hitting pretty obvious false positives. So I've added a set of sanitizer flag workarounds we can expand as needed to continue to work around the limitations of sanitizers on this platform. Together, this restores fastbuild on my ARM macOS with the latest Clang installed. |
||
|
Chandler Carruth
|
03c3b86758 |
Switch lexer to fully table-driven design. (#3273)
This uses the musttail dispatched table approach to drive the entire lexing. The result is that there is no main lexer loop at all in a traditional sense, now everything is driven through tail recursive dispatch on the next byte of the source text. This should be easy to extend still -- the design pattern is to add lexer methods for handling specific cases, and then add a dispatch function to dispatch to them from the table. For example, we can add a method that handles decoding UTF-8 outside of the ASCII subset and set the table entries used by non-ASCII initial bytes to dispatch to it. The performance is already surprisingly good, benchmarks show a modest improvement across the board. That's despite there still being some *serious* performance issues that I'll fix in a separate patch. There are also opportunities to leverage this structure more heavily as needed by putting more specialized dispatch targets in for specific bytes. A follow-up PR will re-organize the functions here, as almost all of the methods on the `Lexer` should become private, but I wanted to keep that a separate change since it will probably render the diff even more hard to read than it already is. |
||
 Hana Dusíková
|
48d40aa0f0 |
Remove unnecessory constexpr for lambdas in tests. (#3272)
After reading Chandler's code I asked him why he has these. Lambdas will implicitly constexpr anyway. |
||
|
Richard Smith
|
74d52738ff |
Support for let declarations. (#3257)
A `let` declaration is represented by a `bind_name` node in SemIR: ```carbon-semir %b: i32 = bind_name "b", %a ``` Because `Check` encounters the pattern before it sees the value, we first create the `bind_name` node with an unset value and don't add it to the block. Then, once we've seen and converted the initializer, we update the `bind_name` to have the value and add it to the current block, after the initializer code. |
||
|
Chandler Carruth
|
c7e6238fa8 |
Introduce two speed-of-light benchmarks. (#3270)
The goal of these kinds of benchmarks is to help calibrate other
benchmarks and expectations. They benchmark the underlying hardware
capabilities that we can't avoid, and help illustrate bounds for what is
possible. The term "speed-of-light benchmark" references the aspect of
measuring how fast thing could possible run.
The first is a simple memory bandwidth measurement in the best case
scenario -- using `strcpy` over the buffer. This still does a minimal
number of writes to memory and examines each byte of input to see if it
is null, but can cheat in every way possible to run at the maximum speed
of hardware. To a certain extent, we never expect to get close to this
speed, but it's a good illustration of how much headroom the hardware
has available.
The second is potentially more interesting. This illustrates how fast a
byte-by-byte dispatch loop can potentially be. It uses the technique
that I'm hoping to use in the lexer itself of guaranteed tail recursion
to achieve this with a very small code footprint. The performance of
this technique, even when running in this extremely minimal setting to
establish bounds, is hugely dependent on the number of distinct dispatch
targets, and so the benchmark includes a healthy range to show the range
of performance that we might expect when running in a byte-by-byte mode.
Note that we should expect the lexer to be *faster* than this
"speed-of-light" whenever it is able to lex in larger granules than
byte-wise. But for complex, dense token sequences that force looking at
every byte, this shows the "worst case" "speed-of-light" in a sense.
On my recent AMD cloud VM instance, I get the following results running
the main lexer benchmark with these changes included:
```
-------------------------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations bytes_per_second tokens_per_second
-------------------------------------------------------------------------------------------------------------------------
BM_ValidKeywords 3169403 ns 3169283 ns 221 188.44M/s 31.5529M/s
BM_ValidIdentifiers<1, 64, false> 12486725 ns 12486445 ns 51 117.953M/s 8.00868M/s
BM_ValidIdentifiers<1, 1, true> 3950455 ns 3950298 ns 178 72.4252M/s 25.3145M/s
BM_ValidIdentifiers<3, 5, true> 15562294 ns 15561178 ns 45 36.7712M/s 6.42625M/s
BM_ValidIdentifiers<3, 16, true> 16118656 ns 16118374 ns 44 68.0412M/s 6.2041M/s
BM_ValidIdentifiers<12, 64, true> 19116271 ns 19116258 ns 35 199.541M/s 5.23115M/s
BM_ValidMix/10/40 7074336 ns 7073795 ns 93 140.744M/s 14.1367M/s
BM_ValidMix/25/30 6790722 ns 6790006 ns 102 131.793M/s 14.7275M/s
BM_ValidMix/50/20 5960514 ns 5960443 ns 118 112.594M/s 16.7773M/s
BM_ValidMix/75/10 4325546 ns 4325556 ns 159 102.559M/s 23.1184M/s
BM_SpeedOfLightStrCpy 24339 ns 24339 ns 29650 35.9049G/s 4.10858G/s
BM_SpeedOfLightDispatch<1>
|
||
|
Richard Smith
|
d306a41453 |
Allow our test runners to be more easily run outside bazel. (#3269)
Don't assume that the bazel-supplied environment variable TEST_TARGET is present. We don't actually need it for anything other than providing feedback to the developer if the test fails. This makes it a bit easier to reproduce test failures under gdb, particularly for multi-file tests. |
||
|
Richard Smith
|
bfa5463e5b |
Add signature enforcement for Main.Run and give it the symbol name main. (#3266)
For #2550. |
||
|
Chandler Carruth
|
b8802035ed |
Switch benchmarks to match unit test namespacing. (#3268)
Specifically, rather than nesting them in `Carbon::Testing`, nest them in `Carbon::Foo` for whatever component they're benchmarking. All our current benchmarks are lexer benchmarks so its `Carbon::Lex`. This makes even more sense for benchmarks than unittests I think. |
||
|
Chandler Carruth
|
7dfd7ca3b4 |
Fix minor UB in the treesitter scanner. (#3267)
In C, the signature `f()` is distinct from `f(void)` -- dating from K&R style prototype-less functions. We need to use the `f(void)` form for the scanner creation function so that it can be called by function pointer `void *(*)(void)` without UB. This was causing a local test that includes treesitter to fail with our fastbuild that enables most sanitizers. With this, we may be able to enable treesitter on our CI as well, but it at least fixes my local test runs. |
||
|
Chandler Carruth
|
4596cd230d |
Avoid building the non-test file group in :all. (#3191)
This file group exists to allow a `genquery` rule and a Python test to verify our non-test dependency graph. We don't actually need to build the binaries in the file group as part of that. The `genquery` rule seems to do the right thing -- building it directly doesn't cause the binaries in the group to be built. But without a manual tag, the group itself is part of `:all` and thus part of `//...` and part of the rules that will be built even with PR #3106. A consequence is that any change to the toolchain causes several other binaries to be built as well because this file group is in the impacted set. Making it manual should avoid all of this, and without breaking the actual use from `genquery`. For example, before this change, in a fully cached build after a `bazel clean`: ``` > bazel test //bazel/check_deps:all INFO: Invocation ID: 2d83ebee-4c00-425d-be33-23f42b079614 INFO: Analyzed 3 targets (103 packages loaded, 7137 targets configured). INFO: Found 2 targets and 1 test target... INFO: Elapsed time: 4.081s, Critical Path: 2.61s INFO: 3111 processes: 2796 disk cache hit, 315 internal. INFO: Build completed successfully, 3111 total actions ``` After this change: ``` > bazel test //bazel/check_deps:al INFO: Invocation ID: c94089e8-a420-4d3c-9902-134e6b55b297 INFO: Analyzed 2 targets (92 packages loaded, 568 targets configured). INFO: Found 1 target and 1 test target... INFO: Elapsed time: 0.700s, Critical Path: 0.01s INFO: 7 processes: 2 disk cache hit, 5 internal. INFO: Build completed successfully, 7 total actions ``` While here, re-generate the file group, and fix several issues it uncovers: mark test utilities as `testonly` and update our LLVM package allowlist to include `clangd`'s package. |
||
|
Richard Smith
|
5ccee62918 |
Add name_reference SemIR node for references to names. (#3260)
Also add `name_reference_untyped` for references to non-first-class names without types, which currently covers namespaces and functions. This improves the fidelity of the SemIR representation, and fixes some issues where we would use the wrong location for nodes and diagnostics downstream of a name reference. We're still missing a representation for dotted name expressions, such as `Namespace.Function`, and we don't use the `untyped` node as an operand of any other node yet. |
||
|
Chandler CarruthandRichard Smith
|
d552545c6d |
Move dispatch routines to be static member functions. (#3265)
These routines have a regular signature and are used to build a table of function pointers for fast dispatch. However, the previous approach relied on lambdas to build these functions which resulted in very hard to read functions in the profile and backtrace. In preparation for expanding dispatch to handle (many) more cases in the lexer and also enabling more aggressive inlining into the dispatch routines, I wanted to tidy up how they appear. This PR alone shouldn't have any interesting functional effect, it's just re-organizing the code. Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Richard Smith
|
e15b24e77d |
Use the name of a function as the name of its file-scope value. (#3262)
As requested in #3260. |
||
|
Chandler Carruth
|
a46ca6bf7a |
Add a start-of-file token and parse node. (#3263)
This removes a (very) hot branch in the lexer where we need to special case when a token is the first token and can't look at its previous token. It also seems like a generally nice change to the structure of both the token buffer and parse tree as there are now bracketing elements for both ends and we should be able to avoid similar branching in the future. Mostly mechanical updates to the lexer and parser code to handle this, but also needed to special case the location information in the autoupdate code. And then the usual large body of auto-updated tests. No benchmark data for this change alone as in isolation and in the current lexer structure it doesn't make a big difference. But this branch was particularly difficult to handle when trying to update the whitespace skipping code to be faster, and so I think it is worth systematically avoiding the special case here. |
||
|
josh11b
|
412e4fb461 | Update comments in toolchain to point to new names of files (#3264) | ||
|
Richard Smith
|
1ac7002bf6 |
Don't try to infer where to put loads in lowering. (#3146)
Trust semantics to have put them in the right places. Many parts of lowering still need to be updated to use the value representation chosen at the semantics layer, but this is an incremental step towards that. |
||
|
josh11b
|
6833652ffa |
Misc fixes to toolchain (#3261)
Was reading through the code and saw some small improvements. |
||
|
Richard Smith
|
e469545eec |
Make autoupdate leave blaze-bin alone and produce less spam. (#3259)
Minor changes to the blaze command line executed by our autoupdate scripts: - Don't change the convenience symlinks. Running autoupdate shouldn't cause `./bazel-bin/...` to switch to running a different binary. - Don't produce so much spam. Bazel will still log its build progress if necessary, and still report compile and runtime errors, but won't produce half a dozen lines of INFO at the start of the command. |
||
|
josh11bandRichard Smith
|
88e87b86b9 |
Updates to generics design details, part 2 (#3253)
Continued from part 1: #3231. Second step updating `docs/design/generics/details.md`. There remains some work to incorporate proposal #2200. - The biggest changes are incorporating much of the text of proposals: - #2173 - #2687 - It incorporates changes from proposals: - #989 - #1178 - #2138 - #2200 - #2360 - #2964 - #3162 - It also updates the text to reflect the latest thinking from leads issues: - #996 - #2153 -- most notably deleting the section on `TypeId`. - Update to rule for prioritization blocks with mixed type structures from [discussion on 2023-07-18](https://docs.google.com/document/d/1gnJBTfY81fZYvI_QXjwKk1uQHYBNHGqRLI2BS_cYYNQ/edit?resourcekey=0-ql1Q1WvTcDvhycf8LbA9DQ#heading=h.7jxges9ojgy3) - Adds reference links to proposals, issues, and discussions relevant to the text. - Also tries to use more precise language when talking about implementations, to avoid confusing `impl` declaration and definitions with the `impls` operator used in `where` clauses, an issue brought up in - #2495 - #2483 --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Richard Smith
|
fea6cf88fc |
Unify initialization and conversion logic (#3255)
Combine the initialization, implicit conversion, and value category conversion functions into a single function. This substantially reduces the duplication between these steps, and ensures that we support the same set of conversions in all these contexts. This also fixes some issues where we would not use the proper value representation for tuples and structs after performing implicit conversions. |
||
 Geoff Romer
|
7899154a21 |
Add "ERROR" to all error diagnostics (#3251)
This makes the difference between errors and lower-level diagnostics visible to users, and aligns the toolchain's behavior with the expectations in `driver_fuzzer.cpp`. |
||
|
Richard Smith
|
491fa1bdd5 |
Place the computation of the destination of an initialization into the return slot. (#3252)
Fix a bug where we would perform the computation of the return location in SemIR after we have already used it in some cases, leading to assertion failures during lowering. Instead, accumulate a sequence of instructions to compute the return location in a temporary block, and overwrite the return slot with those instructions when we perform initialization. StubReference is replaced by a more general SpliceBlock node, that takes a code block and a result value, executes the instructions in the block, and produces the result. This is used in the uncommon case where more than one instruction is required to compute the return slot, which can happen if we need to first emit a temporary and then index into it, or if we need to perform multiple levels of indexing before we reach an entity to initialize. |
||
|
josh11bandChandler Carruth
|
58c106010c |
Updates to design docs to reflect accepted proposals (#3254)
Includes proposals: - #990 - #2188 - #2138 - #2200 - #2360 - #2760 - #2964 - #3162 Also tries to use more precise language when talking about: - implementations, to avoid confusing `impl` declaration and definitions with the `impls` operator used in `where` clauses, an issue brought up in #2495 and #2483; - "binding patterns", like `x: i32`, and "bindings" like `x`. --------- Co-authored-by: Chandler Carruth <chandlerc@gmail.com> |
||
|
Richard Smith
|
caecdc7fdc |
Refactor finalization of initializers. (#3250)
Factor out the common code to find the return slot for an initializing expression, and use it to simplify the two different ways we finalize initializers, as either initializing a temporary or initializing some specific object. This slightly changes the SemIR we create for function calls: instead of rewriting the Call node to have a different destination and replacing its temporary with a `no_op`, we now replace its temporary with a `stub_reference` to the new destination. This results in the same amount of SemIR being produced, but allows calls and other kinds of initializers to be handled uniformly. |
||
|
Richard Smith
|
31d55da403 |
Stop creating fake integer literals as indexes for tuple initialization. (#3248)
This makes struct and tuple initialization basically identical; unify them. |
||
|
Richard Smith
|
f389844893 |
Remove nearly all uses of StubReferences. (#3249)
The speculative insertion of StubReferences after elements in an argument list turned out to not be necessary, because we decided we want to insert per-argument initialization steps after all arguments are evaluated, rather than interleaving them. The StubReferences we insert are causing some minor code complexity, so remove them. We still create StubReferences when performing patch-ups of already-emitted code, but we no longer ever need to look through them when determining whether an initializer was a literal or when evaluating a type expression. |