Just reorganizing logic a little, trying to mirror the direction we've gone with check, lower, etc. That is, lex.h contains a function `Lex` that is used directly. Note, I'm avoiding making meaningful changes here. It could in theory still affect inlining in benchmarks, but I'm not seeing an impact. Before: ``` ------------------------------------------------------------------------------------------------------ Benchmark Time CPU Iterations UserCounters... ------------------------------------------------------------------------------------------------------ BM_ValidKeywords 2784949 ns 2784867 ns 249 bytes_per_second=214.452M/s tokens_per_second=35.9084M/s BM_ValidKeywordsAsRawIdentifiers 3222597 ns 3222551 ns 210 bytes_per_second=244.513M/s tokens_per_second=31.0313M/s BM_RawIdentifierFocus 5907836 ns 5907518 ns 103 bytes_per_second=264.873M/s tokens_per_second=16.9276M/s BM_ValidIdentifiers<1, 64, false> 6255128 ns 6254297 ns 105 bytes_per_second=235.488M/s tokens_per_second=15.989M/s BM_ValidIdentifiers<1, 1, true>3677630ns 3677398 ns 192 bytes_per_second=77.7999M/s tokens_per_second=27.1931M/s BM_ValidIdentifiers<3, 5, true> 5427693 ns 5427116 ns 110 bytes_per_second=105.434M/s tokens_per_second=18.426M/s BM_ValidIdentifiers<3, 16, true> 5063246 ns 5062761 ns 115 bytes_per_second=216.623M/s tokens_per_second=19.7521M/s BM_ValidIdentifiers<12, 64, true>5518589ns 5518118 ns 100 bytes_per_second=691.264M/s tokens_per_second=18.1221M/s BM_ValidIdentifiers<16, 16, true> 4890776 ns 4890782 ns 112 bytes_per_second=350.989M/s tokens_per_second=20.4466M/s BM_ValidIdentifiers<24, 24, true> 4974729 ns 4974582 ns 112 bytes_per_second=498.444M/s tokens_per_second=20.1022M/s BM_ValidIdentifiers<32, 32, true> 5517583 ns 5517085 ns 99 bytes_per_second=587.718M/s tokens_per_second=18.1255M/s BM_ValidIdentifiers<48, 48, true> 5914759 ns 5914222 ns 94 bytes_per_second=806.255M/s tokens_per_second=16.9084M/s BM_ValidIdentifiers<64, 64, true> 7556040 ns 7556036 ns 77 bytes_per_second=833.009M/s tokens_per_second=13.2345M/s BM_ValidIdentifiers<80, 80, true> 7739113 ns 7737696 ns 76 bytes_per_second=1010.65M/s tokens_per_second=12.9237M/s BM_HorizontalWhitespace/1 5015062 ns 5014443 ns 108 bytes_per_second=114.111M/s tokens_per_second=19.9424M/s BM_HorizontalWhitespace/4 5165496 ns 5165425 ns 111 bytes_per_second=166.163M/s tokens_per_second=19.3595M/s BM_HorizontalWhitespace/16 5616796 ns 5616447 ns 102 bytes_per_second=356.578M/s tokens_per_second=17.8049M/s BM_HorizontalWhitespace/64 7912904 ns 7912346 ns 78 bytes_per_second=831.648M/s tokens_per_second=12.6385M/s BM_HorizontalWhitespace/128 11086218 ns 11083155 ns 57 bytes_per_second=1.11759G/s tokens_per_second=9.0227M/s BM_RandomSource 4796549 ns 4795733 ns 145 bytes_per_second=216.783M/s lines_per_second=6.61943M/s tokens_per_second=20.8519M/s BM_GroupingSymbols/1/0/0 3937151 ns 3936581 ns 176 bytes_per_second=216.914M/s lines_per_second=19.0521M/s tokens_per_second=25.4028M/s BM_GroupingSymbols/2/0/0 3000029 ns 2999506 ns 239 bytes_per_second=243.125M/s lines_per_second=27.7812M/s tokens_per_second=33.3388M/s BM_GroupingSymbols/3/0/0 2729059 ns 2728834 ns 251 bytes_per_second=261.26M/s lines_per_second=32.065M/s tokens_per_second=36.6457M/s BM_GroupingSymbols/4/0/0 2432363 ns 2432209 ns 291 bytes_per_second=304.738M/s lines_per_second=37.0034M/s tokens_per_second=41.1149M/s BM_GroupingSymbols/8/0/0 2144080 ns 2143967 ns 326 bytes_per_second=468.547M/s lines_per_second=44.0468M/s tokens_per_second=46.6425M/s BM_GroupingSymbols/16/0/0 2290055 ns 2289711 ns 308 bytes_per_second=741.709M/s lines_per_second=42.3866M/s tokens_per_second=43.6736M/s BM_GroupingSymbols/32/0/0 3102790 ns 3102186 ns 220 bytes_per_second=1027.1M/s lines_per_second=31.7437M/s tokens_per_second=32.2353M/s BM_GroupingSymbols/0/1/0 3406964 ns 3406421 ns 207 bytes_per_second=222.677M/s lines_per_second=7.33908M/s tokens_per_second=29.3563M/s BM_GroupingSymbols/0/2/0 2244101 ns 2244006 ns 312 bytes_per_second=239.985M/s lines_per_second=7.42689M/s tokens_per_second=44.5632M/s BM_GroupingSymbols/0/3/0 1976449 ns 1976080 ns 347 bytes_per_second=216M/s lines_per_second=6.32566M/s tokens_per_second=50.6052M/s BM_GroupingSymbols/0/4/0 1600539 ns 1600325 ns 429 bytes_per_second=224.777M/s lines_per_second=6.24873M/s tokens_per_second=62.4873M/s BM_GroupingSymbols/0/8/0 1085044 ns 1084941 ns 646 bytes_per_second=222.765M/s lines_per_second=5.12009M/s tokens_per_second=92.1709M/s BM_GroupingSymbols/0/16/0 824804 ns 824756 ns 817 bytes_per_second=209.166M/s lines_per_second=3.5659M/s tokens_per_second=121.248M/s BM_GroupingSymbols/0/32/0 685741 ns 685741 ns 1004 bytes_per_second=196.576M/s lines_per_second=2.20929M/s tokens_per_second=145.828M/s BM_GroupingSymbols/0/0/1 3467507 ns 3467077 ns 206 bytes_per_second=218.781M/s lines_per_second=7.21069M/s tokens_per_second=28.8427M/s BM_GroupingSymbols/0/0/2 2284154 ns 2283937 ns 309 bytes_per_second=235.79M/s lines_per_second=7.29705M/s tokens_per_second=43.784M/s BM_GroupingSymbols/0/0/3 1965548 ns 1965225 ns 356 bytes_per_second=217.193M/s lines_per_second=6.36059M/s tokens_per_second=50.8848M/s BM_GroupingSymbols/0/0/4 1623965 ns 1623725 ns 440 bytes_per_second=221.538M/s lines_per_second=6.15868M/s tokens_per_second=61.5868M/s BM_GroupingSymbols/0/0/8 1080601 ns 1080452 ns 650 bytes_per_second=223.691M/s lines_per_second=5.14137M/s tokens_per_second=92.5539M/s BM_GroupingSymbols/0/0/16 840820 ns 840677 ns 828 bytes_per_second=205.205M/s lines_per_second=3.49837M/s tokens_per_second=118.952M/s BM_GroupingSymbols/0/0/32 707793 ns 707734 ns 991 bytes_per_second=190.467M/s lines_per_second=2.14063M/s tokens_per_second=141.296M/s BM_GroupingSymbols/32/1/0 2804159 ns 2803869 ns 245 bytes_per_second=1103.7M/s lines_per_second=34.0779M/s tokens_per_second=35.665M/s BM_GroupingSymbols/32/2/0 2676229 ns 2675855 ns 261 bytes_per_second=1123.78M/s lines_per_second=34.688M/s tokens_per_second=37.3712M/s BM_GroupingSymbols/32/3/0 2652102 ns 2651836 ns 262 bytes_per_second=1103.24M/s lines_per_second=34.0217M/s tokens_per_second=37.7097M/s BM_GroupingSymbols/32/4/0 2600677 ns 2600552 ns 268 bytes_per_second=1096.17M/s lines_per_second=33.7678M/s tokens_per_second=38.4534M/s BM_GroupingSymbols/32/8/0 2382017 ns 2381869 ns 294 bytes_per_second=1083.42M/s lines_per_second=33.2659M/s tokens_per_second=41.9838M/s BM_GroupingSymbols/32/16/0 2102340 ns 2102243 ns 325 bytes_per_second=1034.45M/s lines_per_second=31.5377M/s tokens_per_second=47.5682M/s BM_GroupingSymbols/32/32/0 1745079 ns 1744914 ns 403 bytes_per_second=952.64M/s lines_per_second=28.6461M/s tokens_per_second=57.3094M/s BM_GroupingSymbols/32/32/1 1701414 ns 1701255 ns 415 bytes_per_second=962.73M/s lines_per_second=28.9228M/s tokens_per_second=58.7802M/s BM_GroupingSymbols/32/32/2 1688503 ns 1688121 ns 415 bytes_per_second=957.019M/s lines_per_second=28.7242M/s tokens_per_second=59.2375M/s BM_GroupingSymbols/32/32/3 1679701 ns 1679499 ns 419 bytes_per_second=948.651M/s lines_per_second=28.446M/s tokens_per_second=59.5416M/s BM_GroupingSymbols/32/32/4 1647815 ns 1647816 ns 427 bytes_per_second=953.342M/s lines_per_second=28.559M/s tokens_per_second=60.6864M/s BM_GroupingSymbols/32/32/8 1581283 ns 1581075 ns 450 bytes_per_second=942.39M/s lines_per_second=28.1201M/s tokens_per_second=63.2481M/s BM_GroupingSymbols/32/32/16 1445591 ns 1445526 ns 477 bytes_per_second=936.105M/s lines_per_second=27.7442M/s tokens_per_second=69.179M/s BM_GroupingSymbols/32/32/32 1296335 ns 1296094 ns 544 bytes_per_second=882.943M/s lines_per_second=25.8276M/s tokens_per_second=77.1549M/s BM_BlankLines/1 5774442 ns 5774454 ns 107 bytes_per_second=99.0921M/s lines_per_second=17.3175M/s tokens_per_second=17.3177M/s BM_BlankLines/4 8712135 ns 8711977 ns 75 bytes_per_second=98.5198M/s lines_per_second=45.9133M/s tokens_per_second=11.4785M/s BM_BlankLines/16 32313782 ns 32307655 ns 22 bytes_per_second=61.9884M/s lines_per_second=49.5234M/s tokens_per_second=3.09524M/s BM_BlankLines/64 87562163 ns 87543476 ns 7 bytes_per_second=75.166M/s lines_per_second=73.1058M/s tokens_per_second=1.14229M/s BM_BlankLines/128 160336428 ns 160298644 ns 4 bytes_per_second=79.1257M/s lines_per_second=79.8502M/s tokens_per_second=623.836k/s BM_CommentLines/1/0/0 7298959 ns 7298427 ns 89 bytes_per_second=117.601M/s lines_per_second=27.4029M/s tokens_per_second=13.7016M/s BM_CommentLines/4/0/0 10002223 ns 10002239 ns 67 bytes_per_second=171.622M/s lines_per_second=49.9883M/s tokens_per_second=9.99776M/s BM_CommentLines/128/0/0 143356660 ns 143356412 ns 5 bytes_per_second=259.444M/s lines_per_second=89.9846M/s tokens_per_second=697.562k/s BM_CommentLines/1/30/0 7421994 ns 7421289 ns 87 bytes_per_second=501.166M/s lines_per_second=26.9492M/s tokens_per_second=13.4747M/s BM_CommentLines/4/30/0 11304903 ns 11303972 ns 56 bytes_per_second=1.13696G/s lines_per_second=44.2318M/s tokens_per_second=8.84645M/s BM_CommentLines/128/30/0 156244144 ns 156217089 ns 4 bytes_per_second=2.52178G/s lines_per_second=82.5766M/s tokens_per_second=640.135k/s BM_CommentLines/1/70/0 7486688 ns 7485340 ns 80 bytes_per_second=1006.49M/s lines_per_second=26.7186M/s tokens_per_second=13.3594M/s BM_CommentLines/4/70/0 14762881 ns 14761417 ns 45 bytes_per_second=1.88011G/s lines_per_second=33.8717M/s tokens_per_second=6.77442M/s BM_CommentLines/128/70/0 179933089 ns 179903592 ns 4 bytes_per_second=4.84025G/s lines_per_second=71.7044M/s tokens_per_second=555.853k/s BM_CommentLines/1/0/2 7473119 ns 7472370 ns 87 bytes_per_second=140.389M/s lines_per_second=26.765M/s tokens_per_second=13.3826M/s BM_CommentLines/4/0/2 9702727 ns 9700268 ns 66 bytes_per_second=255.615M/s lines_per_second=51.5445M/s tokens_per_second=10.309M/s BM_CommentLines/128/0/2 140504132 ns 140480254 ns 5 bytes_per_second=438.544M/s lines_per_second=91.8269M/s tokens_per_second=711.844k/s BM_CommentLines/1/30/2 7530847 ns 7529227 ns 82 bytes_per_second=519.314M/s lines_per_second=26.5629M/s tokens_per_second=13.2816M/s BM_CommentLines/4/30/2 11646677 ns 11644972 ns 56 bytes_per_second=1.16764G/s lines_per_second=42.9366M/s tokens_per_second=8.5874M/s BM_CommentLines/128/30/2 161292392 ns 161273904 ns 4 bytes_per_second=2.59054G/s lines_per_second=79.9873M/s tokens_per_second=620.063k/s BM_CommentLines/1/70/2 7700751 ns 7700365 ns 83 bytes_per_second=1003.16M/s lines_per_second=25.9725M/s tokens_per_second=12.9864M/s BM_CommentLines/4/70/2 14133018 ns 14131523 ns 45 bytes_per_second=2.01664G/s lines_per_second=35.3815M/s tokens_per_second=7.07638M/s BM_CommentLines/128/70/2 179085026 ns 179057072 ns 4 bytes_per_second=4.99628G/s lines_per_second=72.0433M/s tokens_per_second=558.481k/s BM_CommentLines/1/0/8 7792060 ns 7791951 ns 83 bytes_per_second=208.065M/s lines_per_second=25.6673M/s tokens_per_second=12.8338M/s BM_CommentLines/4/0/8 10329194 ns 10329006 ns 61 bytes_per_second=461.644M/s lines_per_second=48.4069M/s tokens_per_second=9.68147M/s BM_CommentLines/128/0/8 140917902 ns 140917975 ns 5 bytes_per_second=956.927M/s lines_per_second=91.5417M/s tokens_per_second=709.633k/s BM_CommentLines/1/30/8 7813308 ns 7811702 ns 82 bytes_per_second=573.784M/s lines_per_second=25.6024M/s tokens_per_second=12.8013M/s BM_CommentLines/4/30/8 12052779 ns 12051440 ns 54 bytes_per_second=1.31373G/s lines_per_second=41.4884M/s tokens_per_second=8.29776M/s BM_CommentLines/128/30/8 163767409 ns 163753783 ns 4 bytes_per_second=2.9881G/s lines_per_second=78.776M/s tokens_per_second=610.673k/s BM_CommentLines/1/70/8 8188137 ns 8187614 ns 80 bytes_per_second=1013.35M/s lines_per_second=24.4269M/s tokens_per_second=12.2136M/s BM_CommentLines/4/70/8 15723650 ns 15721559 ns 43 bytes_per_second=1.95485G/s lines_per_second=31.8031M/s tokens_per_second=6.36069M/s BM_CommentLines/128/70/8 182579515 ns 182554119 ns 4 bytes_per_second=5.29237G/s lines_per_second=70.6633M/s tokens_per_second=547.783k/s BM_SpeedOfLightStrCpy 27009 ns 27006 ns 25887 bytes_per_second=37.594G/s lines_per_second=1.17547G/s tokens_per_second=3.70286G/s BM_SpeedOfLightDispatch<1> 1850860 ns 1850660 ns 381 bytes_per_second=561.764M/s lines_per_second=17.1533M/s tokens_per_second=54.0348M/s BM_SpeedOfLightDispatch<2> 1876473 ns 1876369 ns 347 bytes_per_second=554.067M/s lines_per_second=16.9183M/s tokens_per_second=53.2944M/s BM_SpeedOfLightDispatch<4> 2208441 ns 2208443 ns 314 bytes_per_second=470.755M/s lines_per_second=14.3744M/s tokens_per_second=45.2808M/s BM_SpeedOfLightDispatch<8> 2824174 ns 2824137 ns 250 bytes_per_second=368.125M/s lines_per_second=11.2406M/s tokens_per_second=35.409M/s BM_SpeedOfLightDispatch<16> 4306633 ns 4306325 ns 165 bytes_per_second=241.42M/s lines_per_second=7.37172M/s tokens_per_second=23.2217M/s BM_SpeedOfLightDispatch<32> 6300653 ns 6300102 ns 113 bytes_per_second=165.019M/s lines_per_second=5.03881M/s tokens_per_second=15.8728M/s BM_SpeedOfLightDispatch<MaxDispatchTargets> 8879663 ns 8878510 ns 79 bytes_per_second=117.096M/s lines_per_second=3.57549M/s tokens_per_second=11.2632M/s ``` After: ``` ------------------------------------------------------------------------------------------------------ Benchmark Time CPU Iterations UserCounters... ------------------------------------------------------------------------------------------------------ BM_ValidKeywords 2821833 ns 2821832 ns 247 bytes_per_second=211.642M/s tokens_per_second=35.438M/s BM_ValidKeywordsAsRawIdentifiers 3204326 ns 3203964 ns 216 bytes_per_second=245.931M/s tokens_per_second=31.2113M/s BM_RawIdentifierFocus 6076723 ns 6076107 ns 98 bytes_per_second=257.524M/s tokens_per_second=16.4579M/s BM_ValidIdentifiers<1, 64, false> 6303093 ns 6302632 ns 101 bytes_per_second=233.682M/s tokens_per_second=15.8664M/s BM_ValidIdentifiers<1, 1, true> 3687668 ns 3687338 ns 194 bytes_per_second=77.5902M/s tokens_per_second=27.1198M/s BM_ValidIdentifiers<3, 5, true> 5298920 ns 5298465 ns 106 bytes_per_second=107.994M/s tokens_per_second=18.8734M/s BM_ValidIdentifiers<3, 16, true> 4880695 ns 4879704 ns 118 bytes_per_second=224.75M/s tokens_per_second=20.493M/s BM_ValidIdentifiers<12, 64, true> 5413070 ns 5411832 ns 103 bytes_per_second=704.84M/s tokens_per_second=18.478M/s BM_ValidIdentifiers<16, 16, true> 5052780 ns 5051309 ns 110 bytes_per_second=339.835M/s tokens_per_second=19.7968M/s BM_ValidIdentifiers<24, 24, true> 5221580 ns 5220851 ns 104 bytes_per_second=474.933M/s tokens_per_second=19.154M/s BM_ValidIdentifiers<32, 32, true> 5786811 ns 5786806 ns 99 bytes_per_second=560.325M/s tokens_per_second=17.2807M/s BM_ValidIdentifiers<48, 48, true> 5959506 ns 5959502 ns 96 bytes_per_second=800.129M/s tokens_per_second=16.7799M/s BM_ValidIdentifiers<64, 64, true> 7831469 ns 7830629 ns 75 bytes_per_second=803.799M/s tokens_per_second=12.7704M/s BM_ValidIdentifiers<80, 80, true> 7905843 ns 7904727 ns 75 bytes_per_second=989.298M/s tokens_per_second=12.6507M/s BM_HorizontalWhitespace/1 5091171 ns 5090660 ns 109 bytes_per_second=112.402M/s tokens_per_second=19.6438M/s BM_HorizontalWhitespace/4 5020344 ns 5020344 ns 112 bytes_per_second=170.965M/s tokens_per_second=19.919M/s BM_HorizontalWhitespace/16 5846801 ns 5846459 ns 99 bytes_per_second=342.549M/s tokens_per_second=17.1044M/s BM_HorizontalWhitespace/64 7803183 ns 7802357 ns 78 bytes_per_second=843.372M/s tokens_per_second=12.8166M/s BM_HorizontalWhitespace/128 10600500 ns 10598602 ns 59 bytes_per_second=1.16869G/s tokens_per_second=9.43521M/s BM_RandomSource 4824062 ns 4823496 ns 139 bytes_per_second=215.536M/s lines_per_second=6.58133M/s tokens_per_second=20.7319M/s BM_GroupingSymbols/1/0/0 4116846 ns 4116549 ns 170 bytes_per_second=207.431M/s lines_per_second=18.2191M/s tokens_per_second=24.2922M/s BM_GroupingSymbols/2/0/0 3024336 ns 3024156 ns 236 bytes_per_second=241.144M/s lines_per_second=27.5548M/s tokens_per_second=33.0671M/s BM_GroupingSymbols/3/0/0 2789794 ns 2789256 ns 252 bytes_per_second=255.601M/s lines_per_second=31.3704M/s tokens_per_second=35.8519M/s BM_GroupingSymbols/4/0/0 2496498 ns 2496237 ns 283 bytes_per_second=296.921M/s lines_per_second=36.0543M/s tokens_per_second=40.0603M/s BM_GroupingSymbols/8/0/0 2200846 ns 2200611 ns 313 bytes_per_second=456.487M/s lines_per_second=42.9131M/s tokens_per_second=45.4419M/s BM_GroupingSymbols/16/0/0 2415237 ns 2415015 ns 288 bytes_per_second=703.225M/s lines_per_second=40.1873M/s tokens_per_second=41.4076M/s BM_GroupingSymbols/32/0/0 3171195 ns 3170504 ns 215 bytes_per_second=1004.97M/s lines_per_second=31.0597M/s tokens_per_second=31.5407M/s BM_GroupingSymbols/0/1/0 3627393 ns 3626737 ns 193 bytes_per_second=209.15M/s lines_per_second=6.89325M/s tokens_per_second=27.573M/s BM_GroupingSymbols/0/2/0 2501189 ns 2500946 ns 275 bytes_per_second=215.33M/s lines_per_second=6.66388M/s tokens_per_second=39.9849M/s BM_GroupingSymbols/0/3/0 2149513 ns 2149282 ns 318 bytes_per_second=198.593M/s lines_per_second=5.8159M/s tokens_per_second=46.5272M/s BM_GroupingSymbols/0/4/0 1793658 ns 1793292 ns 384 bytes_per_second=200.59M/s lines_per_second=5.57634M/s tokens_per_second=55.7634M/s BM_GroupingSymbols/0/8/0 1302555 ns 1302272 ns 542 bytes_per_second=185.589M/s lines_per_second=4.26562M/s tokens_per_second=76.7889M/s BM_GroupingSymbols/0/16/0 1042993 ns 1042818 ns 671 bytes_per_second=165.428M/s lines_per_second=2.82024M/s tokens_per_second=95.8941M/s BM_GroupingSymbols/0/32/0 955561 ns 955471 ns 749 bytes_per_second=141.082M/s lines_per_second=1.58561M/s tokens_per_second=104.66M/s BM_GroupingSymbols/0/0/1 3659797 ns 3659369 ns 194 bytes_per_second=207.285M/s lines_per_second=6.83178M/s tokens_per_second=27.3271M/s BM_GroupingSymbols/0/0/2 2467556 ns 2467190 ns 281 bytes_per_second=218.276M/s lines_per_second=6.75505M/s tokens_per_second=40.5319M/s BM_GroupingSymbols/0/0/3 2152274 ns 2151938 ns 326 bytes_per_second=198.348M/s lines_per_second=5.80872M/s tokens_per_second=46.4697M/s BM_GroupingSymbols/0/0/4 1805982 ns 1805877 ns 368 bytes_per_second=199.192M/s lines_per_second=5.53747M/s tokens_per_second=55.3747M/s BM_GroupingSymbols/0/0/8 1313041 ns 1312833 ns 539 bytes_per_second=184.096M/s lines_per_second=4.23131M/s tokens_per_second=76.1711M/s BM_GroupingSymbols/0/0/16 1065565 ns 1065301 ns 659 bytes_per_second=161.937M/s lines_per_second=2.76072M/s tokens_per_second=93.8702M/s BM_GroupingSymbols/0/0/32 946630 ns 946514 ns 726 bytes_per_second=142.417M/s lines_per_second=1.60061M/s tokens_per_second=105.651M/s BM_GroupingSymbols/32/1/0 2991120 ns 2991121 ns 233 bytes_per_second=1034.6M/s lines_per_second=31.9445M/s tokens_per_second=33.4323M/s BM_GroupingSymbols/32/2/0 2893280 ns 2892944 ns 245 bytes_per_second=1039.45M/s lines_per_second=32.085M/s tokens_per_second=34.5669M/s BM_GroupingSymbols/32/3/0 2819177 ns 2819024 ns 239 bytes_per_second=1037.81M/s lines_per_second=32.004M/s tokens_per_second=35.4733M/s BM_GroupingSymbols/32/4/0 2778376 ns 2778018 ns 249 bytes_per_second=1026.15M/s lines_per_second=31.6107M/s tokens_per_second=35.9969M/s BM_GroupingSymbols/32/8/0 2538279 ns 2538279 ns 275 bytes_per_second=1016.65M/s lines_per_second=31.216M/s tokens_per_second=39.3968M/s BM_GroupingSymbols/32/16/0 2291819 ns 2291693 ns 305 bytes_per_second=948.937M/s lines_per_second=28.9306M/s tokens_per_second=43.6359M/s BM_GroupingSymbols/32/32/0 1943560 ns 1943560 ns 366 bytes_per_second=855.273M/s lines_per_second=25.7183M/s tokens_per_second=51.452M/s BM_GroupingSymbols/32/32/1 1902069 ns 1901915 ns 375 bytes_per_second=861.158M/s lines_per_second=25.8713M/s tokens_per_second=52.5786M/s BM_GroupingSymbols/32/32/2 1877847 ns 1877752 ns 379 bytes_per_second=860.371M/s lines_per_second=25.8234M/s tokens_per_second=53.2552M/s BM_GroupingSymbols/32/32/3 1837280 ns 1837016 ns 381 bytes_per_second=867.308M/s lines_per_second=26.0069M/s tokens_per_second=54.4361M/s BM_GroupingSymbols/32/32/4 1841010 ns 1840902 ns 380 bytes_per_second=853.349M/s lines_per_second=25.5636M/s tokens_per_second=54.3212M/s BM_GroupingSymbols/32/32/8 1734676 ns 1734437 ns 405 bytes_per_second=859.062M/s lines_per_second=25.6337M/s tokens_per_second=57.6556M/s BM_GroupingSymbols/32/32/16 1641169 ns 1640934 ns 422 bytes_per_second=824.63M/s lines_per_second=24.4403M/s tokens_per_second=60.9409M/s BM_GroupingSymbols/32/32/32 1506988 ns 1506914 ns 472 bytes_per_second=759.418M/s lines_per_second=22.2143M/s tokens_per_second=66.3608M/s BM_BlankLines/1 5658057 ns 5657150 ns 110 bytes_per_second=101.147M/s lines_per_second=17.6766M/s tokens_per_second=17.6767M/s BM_BlankLines/4 8346196 ns 8346052 ns 77 bytes_per_second=102.839M/s lines_per_second=47.9264M/s tokens_per_second=11.9817M/s BM_BlankLines/16 31147085 ns 31144610 ns 22 bytes_per_second=64.3033M/s lines_per_second=51.3727M/s tokens_per_second=3.21083M/s BM_BlankLines/64 83743719 ns 83743762 ns 8 bytes_per_second=78.5765M/s lines_per_second=76.4228M/s tokens_per_second=1.19412M/s BM_BlankLines/128 152299627 ns 152274606 ns 4 bytes_per_second=83.2952M/s lines_per_second=84.0578M/s tokens_per_second=656.708k/s BM_CommentLines/1/0/0 7535704 ns 7535149 ns 83 bytes_per_second=113.906M/s lines_per_second=26.542M/s tokens_per_second=13.2711M/s BM_CommentLines/4/0/0 10107724 ns 10106088 ns 66 bytes_per_second=169.858M/s lines_per_second=49.4746M/s tokens_per_second=9.89503M/s BM_CommentLines/128/0/0 130061022 ns 130029826 ns 5 bytes_per_second=286.034M/s lines_per_second=99.207M/s tokens_per_second=769.054k/s BM_CommentLines/1/30/0 7773816 ns 7772726 ns 83 bytes_per_second=478.506M/s lines_per_second=25.7307M/s tokens_per_second=12.8655M/s BM_CommentLines/4/30/0 11436116 ns 11434452 ns 56 bytes_per_second=1.12398G/s lines_per_second=43.7271M/s tokens_per_second=8.7455M/s BM_CommentLines/128/30/0 155047059 ns 155033899 ns 4 bytes_per_second=2.54103G/s lines_per_second=83.2068M/s tokens_per_second=645.02k/s BM_CommentLines/1/70/0 8016209 ns 8014861 ns 75 bytes_per_second=939.998M/s lines_per_second=24.9534M/s tokens_per_second=12.4768M/s BM_CommentLines/4/70/0 14894752 ns 14891800 ns 44 bytes_per_second=1.86365G/s lines_per_second=33.5752M/s tokens_per_second=6.7151M/s BM_CommentLines/128/70/0 176667108 ns 176631061 ns 4 bytes_per_second=4.92993G/s lines_per_second=73.0329M/s tokens_per_second=566.152k/s BM_CommentLines/1/0/2 7764475 ns 7763675 ns 84 bytes_per_second=135.121M/s lines_per_second=25.7607M/s tokens_per_second=12.8805M/s BM_CommentLines/4/0/2 10238104 ns 10236809 ns 65 bytes_per_second=242.217M/s lines_per_second=48.8429M/s tokens_per_second=9.76867M/s BM_CommentLines/128/0/2 130208823 ns 130190640 ns 5 bytes_per_second=473.204M/s lines_per_second=99.0845M/s tokens_per_second=768.104k/s BM_CommentLines/1/30/2 7941224 ns 7940584 ns 78 bytes_per_second=492.411M/s lines_per_second=25.1868M/s tokens_per_second=12.5935M/s BM_CommentLines/4/30/2 11936879 ns 11934453 ns 56 bytes_per_second=1.13932G/s lines_per_second=41.8951M/s tokens_per_second=8.3791M/s BM_CommentLines/128/30/2 156978531 ns 156967142 ns 4 bytes_per_second=2.66162G/s lines_per_second=82.182M/s tokens_per_second=637.076k/s BM_CommentLines/1/70/2 8223614 ns 8222923 ns 79 bytes_per_second=939.409M/s lines_per_second=24.322M/s tokens_per_second=12.1611M/s BM_CommentLines/4/70/2 15216239 ns 15215047 ns 45 bytes_per_second=1.87303G/s lines_per_second=32.8619M/s tokens_per_second=6.57244M/s BM_CommentLines/128/70/2 176810589 ns 176755958 ns 4 bytes_per_second=5.06133G/s lines_per_second=72.9813M/s tokens_per_second=565.752k/s BM_CommentLines/1/0/8 7901146 ns 7899003 ns 82 bytes_per_second=205.245M/s lines_per_second=25.3194M/s tokens_per_second=12.6598M/s BM_CommentLines/4/0/8 10135919 ns 10134576 ns 66 bytes_per_second=470.501M/s lines_per_second=49.3356M/s tokens_per_second=9.86721M/s BM_CommentLines/128/0/8 132206448 ns 132206347 ns 5 bytes_per_second=1019.98M/s lines_per_second=97.5738M/s tokens_per_second=756.393k/s BM_CommentLines/1/30/8 7981189 ns 7981202 ns 79 bytes_per_second=561.598M/s lines_per_second=25.0586M/s tokens_per_second=12.5294M/s BM_CommentLines/4/30/8 12311389 ns 12309078 ns 54 bytes_per_second=1.28623G/s lines_per_second=40.62M/s tokens_per_second=8.12409M/s BM_CommentLines/128/30/8 160432032 ns 160393999 ns 4 bytes_per_second=3.05069G/s lines_per_second=80.4261M/s tokens_per_second=623.465k/s BM_CommentLines/1/70/8 8199080 ns 8199078 ns 79 bytes_per_second=1011.93M/s lines_per_second=24.3927M/s tokens_per_second=12.1965M/s BM_CommentLines/4/70/8 16087954 ns 16086159 ns 44 bytes_per_second=1.91055G/s lines_per_second=31.0823M/s tokens_per_second=6.21652M/s BM_CommentLines/128/70/8 175714908 ns 175675241 ns 4 bytes_per_second=5.4996G/s lines_per_second=73.4302M/s tokens_per_second=569.232k/s BM_SpeedOfLightStrCpy 28860 ns 28858 ns 26092 bytes_per_second=35.181G/s lines_per_second=1.10002G/s tokens_per_second=3.46519G/s BM_SpeedOfLightDispatch<1> 1823558 ns 1823473 ns 385 bytes_per_second=570.14M/s lines_per_second=17.4091M/s tokens_per_second=54.8404M/s BM_SpeedOfLightDispatch<2> 2013453 ns 2013250 ns 343 bytes_per_second=516.396M/s lines_per_second=15.768M/s tokens_per_second=49.6709M/s BM_SpeedOfLightDispatch<4> 2225145 ns 2224920 ns 312 bytes_per_second=467.268M/s lines_per_second=14.2679M/s tokens_per_second=44.9454M/s BM_SpeedOfLightDispatch<8> 2851730 ns 2851590 ns 251 bytes_per_second=364.581M/s lines_per_second=11.1324M/s tokens_per_second=35.0682M/s BM_SpeedOfLightDispatch<16> 4431584 ns 4431156 ns 161 bytes_per_second=234.619M/s lines_per_second=7.16404M/s tokens_per_second=22.5675M/s BM_SpeedOfLightDispatch<32> 6229285 ns 6228092 ns 111 bytes_per_second=166.927M/s lines_per_second=5.09707M/s tokens_per_second=16.0563M/s BM_SpeedOfLightDispatch<MaxDispatchTargets> 8967228 ns 8965583 ns 79 bytes_per_second=115.958M/s lines_per_second=3.54076M/s tokens_per_second=11.1538M/s ``` --------- Co-authored-by: Chandler Carruth <chandlerc@gmail.com>
Carbon Language:
An experimental successor to C++
Why? | Goals | Status | Getting started | Join us
See our announcement video from CppNorth. Note that Carbon is not ready for use.
Fast and works with C++
- Performance matching C++ using LLVM, with low-level access to bits and addresses
- Interoperate with your existing C++ code, from inheritance to templates
- Fast and scalable builds that work with your existing C++ build systems
Modern and evolving
- Solid language foundations that are easy to learn, especially if you have used C++
- Easy, tool-based upgrades between Carbon versions
- Safer fundamentals, and an incremental path towards a memory-safe subset
Welcoming open-source community
- Clear goals and priorities with robust governance
- Community that works to be welcoming, inclusive, and friendly
- Batteries-included approach: compiler, libraries, docs, tools, package manager, and more
Why build Carbon?
C++ remains the dominant programming language for performance-critical software, with massive and growing codebases and investments. However, it is struggling to improve and meet developers' needs, as outlined above, in no small part due to accumulating decades of technical debt. Incrementally improving C++ is extremely difficult, both due to the technical debt itself and challenges with its evolution process. The best way to address these problems is to avoid inheriting the legacy of C or C++ directly, and instead start with solid language foundations like modern generics system, modular code organization, and consistent, simple syntax.
Existing modern languages already provide an excellent developer experience: Go, Swift, Kotlin, Rust, and many more. Developers that can use one of these existing languages should. Unfortunately, the designs of these languages present significant barriers to adoption and migration from C++. These barriers range from changes in the idiomatic design of software to performance overhead.
Carbon is fundamentally a successor language approach, rather than an attempt to incrementally evolve C++. It is designed around interoperability with C++ as well as large-scale adoption and migration for existing C++ codebases and developers. A successor language for C++ requires:
- Performance matching C++, an essential property for our developers.
- Seamless, bidirectional interoperability with C++, such that a library anywhere in an existing C++ stack can adopt Carbon without porting the rest.
- A gentle learning curve with reasonable familiarity for C++ developers.
- Comparable expressivity and support for existing software's design and architecture.
- Scalable migration, with some level of source-to-source translation for idiomatic C++ code.
With this approach, we can build on top of C++'s existing ecosystem, and bring along existing investments, codebases, and developer populations. There are a few languages that have followed this model for other ecosystems, and Carbon aims to fill an analogous role for C++:
- JavaScript → TypeScript
- Java → Kotlin
- C++ → Carbon
Language Goals
We are designing Carbon to support:
- Performance-critical software
- Software and language evolution
- Code that is easy to read, understand, and write
- Practical safety and testing mechanisms
- Fast and scalable development
- Modern OS platforms, hardware architectures, and environments
- Interoperability with and migration from existing C++ code
While many languages share subsets of these goals, what distinguishes Carbon is their combination.
We also have explicit non-goals for Carbon, notably including:
- A stable application binary interface (ABI) for the entire language and library
- Perfect backwards or forwards compatibility
Our detailed goals document fleshes out these ideas and provides a deeper view into our goals for the Carbon project and language.
Project status
Carbon Language is currently an experimental project. There is no working compiler or toolchain. You can see the demo interpreter for Carbon on compiler-explorer.com.
We want to better understand whether we can build a language that meets our successor language criteria, and whether the resulting language can gather a critical mass of interest within the larger C++ industry and community.
Currently, we have fleshed out several core aspects of both Carbon the project and the language:
- The strategy of the Carbon Language and project.
- An open-source project structure, governance model, and evolution process.
- Critical and foundational aspects of the language design informed by our
experience with C++ and the most difficult challenges we anticipate. This
includes designs for:
- Generics
- Class types
- Inheritance
- Operator overloading
- Lexical and syntactic structure
- Code organization and modular structure
- A prototype interpreter demo that can both run isolated examples and gives a detailed analysis of the specific semantic model and abstract machine of Carbon. We call this the Carbon Explorer.
If you're interested in contributing, we would love help completing the 0.1 language designs, and completing the Carbon Explorer implementation of this design. We are also currently working to get more broad feedback and participation from the C++ community. Beyond that, we plan to prioritize C++ interoperability and a realistic toolchain that implements the 0.1 language and can be used to evaluate Carbon in more detail.
You can see our full roadmap for more details.
Carbon and C++
If you're already a C++ developer, Carbon should have a gentle learning curve. It is built out of a consistent set of language constructs that should feel familiar and be easy to read and understand.
C++ code like this:
corresponds to this Carbon code:
You can call Carbon from C++ without overhead and the other way around. This means you migrate a single C++ library to Carbon within an application, or write new Carbon on top of your existing C++ investment. For example:
Read more about C++ interop in Carbon.
Beyond interoperability between Carbon and C++, we're also planning to support migration tools that will mechanically translate idiomatic C++ code into Carbon code to help you switch an existing C++ codebase to Carbon.
Generics
Carbon provides a modern generics system with checked definitions, while still supporting opt-in templates for seamless C++ interop. Checked generics provide several advantages compared to C++ templates:
- Generic definitions are fully type-checked, removing the need to
instantiate to check for errors and giving greater confidence in code.
- Avoids the compile-time cost of re-checking the definition for every instantiation.
- When using a definition-checked generic, usage error messages are clearer, directly showing which requirements are not met.
- Enables automatic, opt-in type erasure and dynamic dispatch without a separate implementation. This can reduce the binary size and enables constructs like heterogeneous containers.
- Strong, checked interfaces mean fewer accidental dependencies on implementation details and a clearer contract for consumers.
Without sacrificing these advantages, Carbon generics support specialization, ensuring it can fully address performance-critical use cases of C++ templates. For more details about Carbon's generics, see their design.
In addition to easy and powerful interop with C++, Carbon templates can be constrained and incrementally migrated to checked generics at a fine granularity and with a smooth evolutionary path.
Memory safety
Safety, and especially memory safety, remains a key challenge for C++ and something a successor language needs to address. Our initial priority and focus is on immediately addressing important, low-hanging fruit in the safety space:
- Tracking uninitialized states better, increased enforcement of initialization, and systematically providing hardening against initialization bugs when desired.
- Designing fundamental APIs and idioms to support dynamic bounds checks in debug and hardened builds.
- Having a default debug build mode that is both cheaper and more comprehensive than existing C++ build modes even when combined with Address Sanitizer.
Once we can migrate code into Carbon, we will have a simplified language with room in the design space to add any necessary annotations or features, and infrastructure like generics to support safer design patterns. Longer term, we will build on this to introduce a safe Carbon subset. This will be a large and complex undertaking, and won't be in the 0.1 design. Meanwhile, we are closely watching and learning from efforts to add memory safe semantics onto C++ such as Rust-inspired lifetime annotations.
Getting started
As there is no compiler yet, to try out Carbon, you can use the Carbon explorer to interpret Carbon code and print its output. You can try it out immediately at compiler-explorer.com.
To build the Carbon explorer yourself, you'll need to install dependencies (Bazel, Clang, libc++), and then you can run:
# Download Carbon's code.
$ git clone https://github.com/carbon-language/carbon-lang
$ cd carbon-lang
# Build and run the explorer.
$ bazel run //explorer -- ./explorer/testdata/print/format_only.carbon
For complete instructions, including installing dependencies, see our contribution tools documentation.
Learn more about the Carbon project:
Conference talks
Past Carbon focused talks from the community:
2022
- Carbon Language: An experimental successor to C++, CppNorth
- Carbon Language: Syntax and trade-offs, Core C++
2023
- Carbon’s Successor Strategy: From C++ interop to memory safety, C++Now
- Definition-Checked Generics (Part 1, Part 2), C++Now
- Modernizing Compiler Design for Carbon’s Toolchain, C++Now
Join us
We'd love to have folks join us and contribute to the project. Carbon is committed to a welcoming and inclusive environment where everyone can contribute.
- Most of Carbon's design discussions occur on Discord.
- Carbon is a Google Summer of Code 2023 organization.
- To watch for major release announcements, subscribe to our Carbon release post on GitHub and star carbon-lang.
- See our code of conduct and contributing guidelines for information about the Carbon development community.
Contributing
You can also directly:
- Contribute to the language design: feedback on design, new design proposal
- Contribute to the language implementation
- Carbon Explorer: bug report, bug fix, language feature implementation
- Carbon Toolchain, and project infrastructure
You can check out some
"good first issues",
or join the #contributing-help channel on
Discord. See our full
CONTRIBUTING documentation for more details.
