mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-09-24 13:50:10 +01:00
d037848a96271873bd819b5e17af1ef5850dde08
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chandler Carruth
|
d037848a96 |
Replace hashtable ForEach callback with range-based iteration (#7806)
Replaces the callback-based `ForEach` methods on `RawHashtable`, `Map`,
and
`Set` with a range object supporting range-for loops, structured
bindings, and
the standard range concepts.
- Adds `.entries()` on `Map`, `Set`, and `RawHashtable`, returning a
range that
models `std::ranges::forward_range` and `std::ranges::common_range`.
Obtaining one is an explicit call rather than `begin()`/`end()` on the
container, as scanning a whole table is costly and shouldn't be hidden.
- Iterating a `Map` yields a `std::pair` of key and value references,
which
fits in two registers and is returned without being materialized in
memory.
- `Map::Range` and `Set::Range` are aliases of the raw hashtable's range
rather
than wrappers around it. The raw iterator produces the user-facing
reference
itself -- a `KeyT&` for a set, a pair of references for a map -- picked
by
`StorageEntry`, which is already specialized on whether there is a value
type. That leaves one iterator to reason about instead of three.
- Deletes the rvalue `.entries()` overloads on the owning containers, as
a
range built from a temporary table would dangle. Views don't own their
storage, so the operation remains available on them.
- In release builds, the walk over the groups is a single induction
variable: a
negative byte offset counting up to zero, anchored at the ends of the
metadata and entry arrays. Both arrays are then reached by indexed
addressing
off a base that stays put, and the entry pointer is formed only once a
group
with a present entry has been found.
- In debug builds, the range hashes the table's metadata when it is
built and
re-checks that hash when it is destroyed, catching mutation of the table
while a range is live. It also picks a random starting group and a
random odd
group stride, which varies the traversal order between ranges while
still
visiting every group exactly once. That entropy is drawn when the range
is
built rather than in `begin()`, so `begin()` stays a pure function of
the
range and the multi-pass guarantee holds.
- Removes `ForEachEntry` and all of its callers.
Measured against the iteration benchmark added in its own commit, a
traversal is at or ahead of what the callback compiled to across nearly
the
whole size range. The largest tables spend 3-5% fewer cycles, small
`Set`s as
much as 24% fewer, and instruction counts stay within about 1%. What
remains
behind is a handful of mid-sized `Map`s by up to 1%, and `Set` at 65536,
which
sits at exactly half its load factor, by 2%.
Both revisions were built with `-c opt --copt=-march=x86-64-v3` and
compared
with:
```
./scripts/bench_runner.py --exp_benchmark=... --base_benchmark=... \
--benchmark_args=--benchmark_perf_counters=INSTRUCTIONS,CYCLES \
--benchmark_args='--benchmark_filter=(Set|Map)Iterate<(Set|Map)<' \
--extra_metrics_filter='(INSTRUCTIONS|CYCLES)'
```
Trimmed below to the primary integer configurations and to the two
counters;
the pointer- and string-keyed configurations follow the same pattern.
```
Benchmark ┃ CYCLES ┃ INSTRUCTIONS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
BM_MapIterate<Map<int, int>>/1....... │ 👍 -6.032% p=1.14e-05 │ ?? p=0.752
baseline: │ 12.06 ± 1.520% │ 64 ± 3.125%
experiment: │ 11.33 ± 2.765% │ 65.5 ± 3.817%
│ │
BM_MapIterate<Map<int, int>>/2....... │ ?? p=0.155 │ ?? p=0.343
baseline: │ 7.587 ± 1.285% │ 41 ± 0.000%
experiment: │ 7.652 ± 0.865% │ 41 ± 2.439%
│ │
BM_MapIterate<Map<int, int>>/3....... │ ?? p=0.343 │ 👍 -1.020% p=0.0039
baseline: │ 6.663 ± 4.260% │ 32.67 ± 2.041%
experiment: │ 6.368 ± 12.224% │ 32.33 ± 2.062%
│ │
BM_MapIterate<Map<int, int>>/4....... │ ?? p=0.343 │ 👍 -1.786% p=0.0297
baseline: │ 6.091 ± 15.470% │ 28 ± 3.571%
experiment: │ 5.957 ± 8.932% │ 27.5 ± 3.636%
│ │
BM_MapIterate<Map<int, int>>/8....... │ ?? p=0.323 │ 👍 -1.220% p=0.000148
baseline: │ 4.845 ± 0.800% │ 20.5 ± 0.000%
experiment: │ 4.814 ± 3.585% │ 20.25 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/16...... │ 👍 -2.195% p=0.00908 │ 👍 0.769% p=6.58e-06
baseline: │ 4.312 ± 0.187% │ 16.25 ± 0.000%
experiment: │ 4.218 ± 2.368% │ 16.13 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/32...... │ ?? p=0.236 │ 👍 0.442% p=9.53e-06
baseline: │ 4.051 ± 1.084% │ 14.13 ± 0.000%
experiment: │ 4.063 ± 0.737% │ 14.06 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/64...... │ ?? p=0.693 │ 👎 0.227% p=4.52e-06
baseline: │ 4.021 ± 0.239% │ 13.75 ± 0.000%
experiment: │ 4.019 ± 0.417% │ 13.78 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/256..... │ 👍 0.360% p=0.00119 │ 👎 0.754% p=1.37e-05
baseline: │ 3.996 ± 0.173% │ 13.47 ± 0.000%
experiment: │ 3.982 ± 0.272% │ 13.57 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/4096.... │ 👍 0.581% p=1.96e-05 │ 👎 0.923% p=1.96e-05
baseline: │ 4.005 ± 0.816% │ 13.38 ± 0.000%
experiment: │ 3.981 ± 0.192% │ 13.5 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/65536... │ 👍 -4.957% p=1.14e-05 │ 👎 0.934% p=1.14e-05
baseline: │ 5.307 ± 0.501% │ 13.38 ± 0.000%
experiment: │ 5.044 ± 1.746% │ 13.5 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/1048576. │ 👍 -3.947% p=9.09e-05 │ 👎 0.935% p=3.3e-05
baseline: │ 6.074 ± 0.807% │ 13.38 ± 0.000%
experiment: │ 5.834 ± 2.159% │ 13.5 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/16777216 │ ?? p=0.155 │ 👎 0.935% p=2.11e-05
baseline: │ 5.082 ± 3.650% │ 13.38 ± 0.000%
experiment: │ 5.012 ± 1.316% │ 13.5 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/56...... │ 👎 0.825% p=0.0268 │ 👍 0.270% p=1.14e-05
baseline: │ 3.918 ± 0.501% │ 13.21 ± 0.000%
experiment: │ 3.951 ± 0.342% │ 13.18 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/224..... │ 👎 0.788% p=0.000504 │ 👎 0.346% p=1.64e-05
baseline: │ 3.895 ± 0.111% │ 12.89 ± 0.000%
experiment: │ 3.926 ± 0.285% │ 12.94 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/3584.... │ 👎 1.028% p=0.000148 │ 👎 0.545% p=1.14e-05
baseline: │ 3.913 ± 0.427% │ 12.79 ± 0.000%
experiment: │ 3.954 ± 0.325% │ 12.86 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/57344... │ ?? p=0.236 │ 👎 0.558% p=2.55e-06
baseline: │ 4.574 ± 0.721% │ 12.79 ± 0.000%
experiment: │ 4.51 ± 3.709% │ 12.86 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/917504.. │ 👍 -3.826% p=6.58e-06 │ 👎 0.559% p=2.33e-05
baseline: │ 5.221 ± 0.507% │ 12.79 ± 0.000%
experiment: │ 5.021 ± 0.556% │ 12.86 ± 0.000%
│ │
BM_MapIterate<Map<int, int>>/14680064 │ 👍 -3.839% p=1.37e-05 │ 👎 0.559% p=3.31e-05
baseline: │ 5.129 ± 1.194% │ 12.79 ± 0.000%
experiment: │ 4.932 ± 1.475% │ 12.86 ± 0.000%
│ │
Benchmark ┃ CYCLES ┃ INSTRUCTIONS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
BM_SetIterate<Set<int>>/1....... │ 👍 -3.104% p=0.000583 │ ?? p=0.206
baseline: │ 11.2 ± 6.323% │ 60 ± 3.333%
experiment: │ 10.85 ± 5.820% │ 61 ± 3.279%
│ │
BM_SetIterate<Set<int>>/2....... │ 👍 -7.037% p=0.0362 │ ?? p=0.155
baseline: │ 7.086 ± 16.857% │ 37 ± 0.000%
experiment: │ 6.587 ± 0.479% │ 36 ± 4.167%
│ │
BM_SetIterate<Set<int>>/3....... │ 👎 1.400% p=2.34e-05 │ 👍 -1.163% p=0.00136
baseline: │ 5.363 ± 0.463% │ 28.67 ± 2.326%
experiment: │ 5.438 ± 32.763% │ 28.33 ± 1.176%
│ │
BM_SetIterate<Set<int>>/4....... │ ?? p=0.968 │ ?? p=0.286
baseline: │ 4.642 ± 32.751% │ 23.5 ± 2.128%
experiment: │ 4.658 ± 38.416% │ 23.63 ± 3.704%
│ │
BM_SetIterate<Set<int>>/8....... │ 👍 -23.823% p=3.74e-06 │ 👍 -1.515% p=5.52e-05
baseline: │ 4.701 ± 6.589% │ 16.5 ± 0.000%
experiment: │ 3.581 ± 7.790% │ 16.25 ± 0.000%
│ │
BM_SetIterate<Set<int>>/16...... │ 👍 -4.502% p=1.37e-05 │ 👍 -1.020% p=3.31e-05
baseline: │ 3.124 ± 0.585% │ 12.25 ± 0.000%
experiment: │ 2.983 ± 0.625% │ 12.13 ± 0.000%
│ │
BM_SetIterate<Set<int>>/32...... │ 👍 -4.032% p=5.46e-06 │ 👍 0.617% p=1.96e-05
baseline: │ 2.957 ± 0.260% │ 10.13 ± 0.000%
experiment: │ 2.838 ± 0.434% │ 10.06 ± 0.000%
│ │
BM_SetIterate<Set<int>>/64...... │ 👍 -5.054% p=4.52e-06 │ 👎 0.321% p=1.37e-05
baseline: │ 2.937 ± 0.301% │ 9.75 ± 0.000%
experiment: │ 2.788 ± 1.143% │ 9.781 ± 0.000%
│ │
BM_SetIterate<Set<int>>/256..... │ 👍 -5.325% p=1.14e-05 │ 👎 1.073% p=6.58e-06
baseline: │ 2.916 ± 0.220% │ 9.469 ± 0.000%
experiment: │ 2.761 ± 0.142% │ 9.57 ± 0.000%
│ │
BM_SetIterate<Set<int>>/4096.... │ 👍 -4.865% p=4.52e-06 │ 👎 1.317% p=2.34e-05
baseline: │ 2.921 ± 0.194% │ 9.381 ± 0.000%
experiment: │ 2.779 ± 0.224% │ 9.504 ± 0.000%
│ │
BM_SetIterate<Set<int>>/65536... │ 👎 1.961% p=3.93e-05 │ 👎 1.332% p=1.49e-05
baseline: │ 4.015 ± 0.482% │ 9.375 ± 0.000%
experiment: │ 4.094 ± 0.613% │ 9.5 ± 0.000%
│ │
BM_SetIterate<Set<int>>/1048576. │ 👍 -4.843% p=1.14e-05 │ 👎 1.333% p=5.38e-06
baseline: │ 5.239 ± 0.144% │ 9.375 ± 0.000%
experiment: │ 4.986 ± 0.139% │ 9.5 ± 0.000%
│ │
BM_SetIterate<Set<int>>/16777216 │ 👍 0.840% p=0.0362 │ 👎 1.333% p=2.52e-06
baseline: │ 3.719 ± 1.420% │ 9.375 ± 0.000%
experiment: │ 3.688 ± 1.308% │ 9.5 ± 0.000%
│ │
BM_SetIterate<Set<int>>/56...... │ 👍 -2.857% p=9.53e-06 │ 👍 0.388% p=3.31e-05
baseline: │ 2.942 ± 0.439% │ 9.214 ± 0.000%
experiment: │ 2.858 ± 0.619% │ 9.179 ± 0.000%
│ │
BM_SetIterate<Set<int>>/224..... │ 👍 -2.161% p=2.34e-05 │ 👎 0.502% p=4.52e-06
baseline: │ 2.888 ± 0.347% │ 8.893 ± 0.000%
experiment: │ 2.826 ± 0.450% │ 8.938 ± 0.000%
│ │
BM_SetIterate<Set<int>>/3584.... │ 👍 -1.750% p=6.58e-06 │ 👎 0.793% p=2.34e-05
baseline: │ 2.89 ± 0.261% │ 8.792 ± 0.000%
experiment: │ 2.84 ± 0.411% │ 8.862 ± 0.000%
│ │
BM_SetIterate<Set<int>>/57344... │ ?? p=0.502 │ 👎 0.812% p=2.78e-05
baseline: │ 3.684 ± 4.246% │ 8.786 ± 0.000%
experiment: │ 3.644 ± 4.431% │ 8.857 ± 0.000%
│ │
BM_SetIterate<Set<int>>/917504.. │ 👍 -2.629% p=0.000148 │ 👎 0.813% p=3.08e-06
baseline: │ 4.372 ± 0.693% │ 8.786 ± 0.000%
experiment: │ 4.257 ± 0.210% │ 8.857 ± 0.000%
│ │
BM_SetIterate<Set<int>>/14680064 │ 👍 -2.927% p=0.0219 │ 👎 0.813% p=3.03e-06
baseline: │ 4.154 ± 3.286% │ 8.786 ± 0.000%
experiment: │ 4.032 ± 3.198% │ 8.857 ± 0.000%
│ │
```
Assisted-by: Antigravity with Opus
|
||
 Dana Jansens
|
49345352d6 |
Remove redundant use of typename (#7751)
clang-tidy 24 warns about these |
||
 Richard Smith
|
b300f36e6f |
Use inline constexpr where appropriate. (#6374)
This fixes various violations of C++'s One Definition Rule, where we accidentally gave the same static data member multiple definitions in different translation units. Clang happens to emit such definitions with weak linkage, which allows us to get away with this without link errors, but it's still formally incorrect. Also switch keyword order around for a handful of instances of `constexpr inline`, per agreement in open discussion. This happens to reduce the size of a `-c dbg` toolchain binary by 7.2 MiB, presumably by making more of our symbols and especially debug info discardable. |
||
 Jon Ross-Perkins
|
3f799bd987 |
Use explicit(false) for implicit construction (#6039)
Echoing what was added in #5608, updating existing uses. Unfortunately there's divergent behavior for operators versus constructors, so keeping the nolint on those. |
||
|
Jon Ross-Perkins
|
dbf12eb3fc |
Add a SameAsOneOf helper (#5490)
Trying to make repeated `std::same_as` easier to write. Calling it "concepts.h" because I figure we'll maybe have a couple more things like this. Was looking at this because I may add a couple more similar constructs. |
||
|
Dana Jansens
|
3d5d62e1c7 |
Allow making sets and maps with move-only keys and/or values (#4982)
When the key or value is move-only, then the set or map will be as well. |
||
|
Jon Ross-Perkins
|
2ea2166cf8 |
Update pre-commit (#4995)
`pre-commit autoupdate --freeze && pre-commit run -a` |
||
|
Jon Ross-Perkins
|
2fef1cb713 |
Switch to trailing returns in toolchain and related code. (#4919)
Also makes the style guide explicitly comment on void, but this was the intent IIRC because it matches Carbon's `-> ()` (and "always" versus "except for void", which we definitely went back and forth on). Includes adjusting function pointers, which I definitely forget this syntax works sometimes. Excludes utils/tree_sitter/src/scanner.c because it claims to be C, but really we should probably fix that to be cpp. |
||
|
Jon Ross-Perkins
|
82a346730a |
Fix clang-tidy issues (#4786)
Both of these I noticed from testing #4785, but they occur at head. ``` (elided)/execroot/_main/common/raw_hashtable.h:532:40: error: do not use nested 'std::max' calls, use an initializer list instead [modernize-min-max-use-initializer-list,-warnings-as-errors] 532 | static constexpr ssize_t Alignment = std::max<ssize_t>( | ^ 533 | {alignof(MetadataGroup), alignof(StorageEntry<KeyT, ValueT>)}); | ~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | static_cast<long>(alignof(MetadataGroup)) static_cast<long>(alignof(StorageEntry<KeyT, ValueT>)) (elided)/execroot/_main/toolchain/install/busybox_info_test.cpp:259:8: error: unused local variable 'usr_prefix' of type 'std::filesystem::path' [bugprone-unused-local-non-trivial-variable,-warnings-as-errors] 259 | auto usr_prefix = MakeInstallTree(dir_ / "usr"); | ^ (elided)/execroot/_main/toolchain/install/busybox_info_test.cpp:260:8: error: unused local variable 'usr_local_prefix' of type 'std::filesystem::path' [bugprone-unused-local-non-trivial-variable,-warnings-as-errors] 260 | auto usr_local_prefix = MakeInstallTree(dir_ / "usr/local"); | ^ ``` The std::max diagnostic seems a little confused, but the initializer list seems like it can be dropped without any loss. The unused locals diagnostic is correct. Neither of these seem like they should be newer than my last clang-tidy pass, maybe I just missed them in other sweeps. |
||
 
|
624950c62c |
Store hash in the probed_indices array in common/raw_hashtable.h to avoid its recomputation. (#4726)
Store hash in probed_indices array to avoid its recomputation. Benchmarks on ARM (altra, aarch64). ``` name old CYCLES/op new CYCLES/op delta BM_MapInsertSeq<Map<int, int>>/1 119 ± 2% 119 ± 1% ~ (p=0.961 n=55+54) BM_MapInsertSeq<Map<int, int>>/2 133 ± 1% 134 ± 1% ~ (p=0.342 n=56+57) BM_MapInsertSeq<Map<int, int>>/3 150 ± 1% 150 ± 1% ~ (p=0.856 n=56+57) BM_MapInsertSeq<Map<int, int>>/4 167 ± 2% 167 ± 2% ~ (p=0.430 n=56+57) BM_MapInsertSeq<Map<int, int>>/8 234 ± 5% 234 ± 3% ~ (p=0.957 n=57+57) BM_MapInsertSeq<Map<int, int>>/16 368 ± 4% 368 ± 4% ~ (p=0.762 n=57+57) BM_MapInsertSeq<Map<int, int>>/32 650 ± 4% 650 ± 4% ~ (p=0.955 n=57+57) BM_MapInsertSeq<Map<int, int>>/64 1.93k ± 4% 1.98k ± 4% +2.35% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/256 9.68k ± 5% 9.85k ± 3% +1.74% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/4096 177k ± 3% 163k ± 2% -8.17% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/65536 3.99M ± 3% 3.87M ± 4% -3.12% (p=0.000 n=56+56) BM_MapInsertSeq<Map<int, int>>/1048576 90.5M ± 5% 91.3M ± 6% +0.87% (p=0.025 n=55+55) BM_MapInsertSeq<Map<int, int>>/16777216 2.77G ± 8% 2.74G ± 9% ~ (p=0.076 n=57+57) BM_MapInsertSeq<Map<int, int>>/56 1.05k ± 5% 1.05k ± 5% ~ (p=0.727 n=57+57) BM_MapInsertSeq<Map<int, int>>/224 6.29k ± 5% 6.37k ± 4% +1.32% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/3584 124k ± 4% 109k ± 3% -12.46% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/57344 2.67M ± 4% 2.50M ± 4% -6.40% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/917504 65.3M ± 6% 65.8M ± 6% +0.89% (p=0.050 n=55+56) BM_MapInsertSeq<Map<int, int>>/14680064 2.17G ±10% 2.14G ± 9% -1.55% (p=0.032 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/1 122 ± 1% 122 ± 1% ~ (p=0.415 n=56+56) BM_MapInsertSeq<Map<int*, int*>>/2 136 ± 1% 136 ± 1% ~ (p=0.861 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/3 153 ± 1% 153 ± 1% ~ (p=0.607 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/4 170 ± 2% 174 ± 3% +2.34% (p=0.001 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/8 238 ± 4% 242 ± 3% +1.59% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/16 382 ± 4% 383 ± 4% ~ (p=0.977 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/32 701 ± 7% 682 ± 5% -2.69% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/64 2.13k ± 6% 2.09k ± 3% -1.89% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/256 10.3k ± 3% 10.2k ± 3% -0.94% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/4096 184k ± 2% 179k ± 2% -2.62% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/65536 3.63M ± 2% 3.68M ± 3% +1.22% (p=0.000 n=54+57) BM_MapInsertSeq<Map<int*, int*>>/1048576 129M ±10% 129M ±10% ~ (p=0.874 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/16777216 3.27G ±11% 3.24G ±10% ~ (p=0.451 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/56 1.18k ± 9% 1.10k ± 5% -6.52% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/224 6.76k ± 5% 6.59k ± 4% -2.55% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/3584 117k ± 2% 115k ± 3% -1.93% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/57344 2.22M ± 3% 2.24M ± 2% +0.87% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/917504 95.0M ± 8% 94.8M ± 9% ~ (p=0.894 n=55+57) BM_MapInsertSeq<Map<int*, int*>>/14680064 2.42G ±14% 2.40G ±13% ~ (p=0.852 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/1 124 ± 1% 124 ± 1% ~ (p=0.604 n=56+55) BM_MapInsertSeq<Map<int, llvm::StringRef>>/2 140 ± 1% 140 ± 1% ~ (p=0.181 n=56+56) BM_MapInsertSeq<Map<int, llvm::StringRef>>/3 158 ± 1% 158 ± 3% ~ (p=1.000 n=56+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/4 176 ± 2% 176 ± 3% ~ (p=0.125 n=56+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/8 247 ± 4% 247 ± 2% ~ (p=0.614 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/16 391 ± 3% 391 ± 2% ~ (p=0.993 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/32 690 ± 3% 691 ± 3% ~ (p=0.224 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/64 2.17k ± 3% 2.22k ± 3% +1.94% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/256 11.1k ± 3% 11.3k ± 3% +1.58% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/4096 204k ± 2% 193k ± 2% -5.65% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/65536 5.19M ± 3% 5.09M ± 3% -2.05% (p=0.000 n=56+56) BM_MapInsertSeq<Map<int, llvm::StringRef>>/1048576 124M ±10% 123M ± 6% ~ (p=0.626 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/16777216 3.30G ± 9% 3.25G ± 8% -1.39% (p=0.019 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/56 1.12k ± 3% 1.12k ± 3% ~ (p=0.482 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/224 7.04k ± 4% 7.14k ± 3% +1.36% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/3584 138k ± 2% 126k ± 2% -8.89% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/57344 3.48M ± 4% 3.34M ± 4% -3.93% (p=0.000 n=56+56) BM_MapInsertSeq<Map<int, llvm::StringRef>>/917504 84.4M ± 7% 84.9M ± 6% ~ (p=0.159 n=56+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/14680064 2.42G ± 9% 2.40G ±10% ~ (p=0.300 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/1 168 ± 0% 168 ± 0% ~ (p=0.555 n=56+55) BM_MapInsertSeq<Map<llvm::StringRef, int>>/2 208 ± 0% 208 ± 0% ~ (p=0.722 n=52+53) BM_MapInsertSeq<Map<llvm::StringRef, int>>/3 248 ± 0% 248 ± 0% ~ (p=0.248 n=53+54) BM_MapInsertSeq<Map<llvm::StringRef, int>>/4 288 ± 0% 288 ± 0% ~ (p=0.185 n=54+55) BM_MapInsertSeq<Map<llvm::StringRef, int>>/8 457 ± 0% 457 ± 0% ~ (p=0.665 n=53+53) BM_MapInsertSeq<Map<llvm::StringRef, int>>/16 867 ± 1% 867 ± 1% ~ (p=0.174 n=47+52) BM_MapInsertSeq<Map<llvm::StringRef, int>>/32 1.61k ± 3% 1.62k ± 4% ~ (p=0.402 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/64 4.96k ± 9% 4.89k ± 5% -1.37% (p=0.046 n=57+54) BM_MapInsertSeq<Map<llvm::StringRef, int>>/256 26.9k ± 8% 26.5k ± 8% -1.51% (p=0.004 n=56+55) BM_MapInsertSeq<Map<llvm::StringRef, int>>/4096 600k ± 3% 588k ± 2% -2.07% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/65536 13.9M ± 3% 13.5M ± 2% -2.99% (p=0.000 n=55+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/1048576 407M ± 7% 393M ± 5% -3.27% (p=0.000 n=56+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/16777216 10.2G ± 8% 9.9G ± 5% -3.50% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/56 2.81k ± 5% 2.81k ± 4% ~ (p=0.809 n=56+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/224 17.9k ± 6% 17.6k ± 5% -1.20% (p=0.035 n=57+52) BM_MapInsertSeq<Map<llvm::StringRef, int>>/3584 374k ± 3% 367k ± 3% -1.80% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/57344 8.64M ± 3% 8.53M ± 2% -1.29% (p=0.000 n=55+55) BM_MapInsertSeq<Map<llvm::StringRef, int>>/917504 247M ± 6% 244M ± 5% -1.19% (p=0.021 n=56+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/14680064 6.81G ± 8% 6.64G ± 6% -2.46% (p=0.000 n=57+57) ``` Benchmarks on x86 ``` name old cpu/op new cpu/op delta BM_MapInsertSeq<Map<int, int>>/1 32.9ns ± 3% 32.6ns ± 3% -0.84% (p=0.027 n=54+51) BM_MapInsertSeq<Map<int, int>>/2 35.9ns ± 3% 35.7ns ± 4% ~ (p=0.123 n=54+54) BM_MapInsertSeq<Map<int, int>>/3 39.7ns ± 3% 47.4ns ± 4% +19.40% (p=0.000 n=55+56) BM_MapInsertSeq<Map<int, int>>/4 52.7ns ± 3% 52.1ns ± 4% -1.22% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/8 78.1ns ± 3% 78.3ns ± 3% ~ (p=0.141 n=50+57) BM_MapInsertSeq<Map<int, int>>/16 135ns ± 3% 135ns ± 4% ~ (p=0.936 n=53+57) BM_MapInsertSeq<Map<int, int>>/32 249ns ± 3% 241ns ± 3% -3.28% (p=0.000 n=55+57) BM_MapInsertSeq<Map<int, int>>/64 631ns ± 3% 618ns ± 3% -2.21% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/256 2.62µs ± 3% 2.36µs ± 4% -10.02% (p=0.000 n=52+53) BM_MapInsertSeq<Map<int, int>>/4096 39.2µs ± 3% 37.9µs ± 4% -3.40% (p=0.000 n=57+56) BM_MapInsertSeq<Map<int, int>>/65536 972µs ± 3% 955µs ± 3% -1.76% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/1048576 16.2ms ± 4% 16.3ms ± 5% ~ (p=0.231 n=52+54) BM_MapInsertSeq<Map<int, int>>/16777216 651ms ± 3% 648ms ± 2% -0.42% (p=0.048 n=57+56) BM_MapInsertSeq<Map<int, int>>/56 418ns ± 3% 401ns ± 3% -4.10% (p=0.000 n=54+57) BM_MapInsertSeq<Map<int, int>>/224 1.79µs ± 3% 1.61µs ± 3% -10.20% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, int>>/3584 26.0µs ± 3% 24.9µs ± 4% -4.13% (p=0.000 n=57+56) BM_MapInsertSeq<Map<int, int>>/57344 560µs ± 3% 549µs ± 3% -2.11% (p=0.000 n=56+57) BM_MapInsertSeq<Map<int, int>>/917504 10.4ms ± 3% 10.4ms ± 3% ~ (p=0.805 n=56+56) BM_MapInsertSeq<Map<int, int>>/14680064 422ms ± 2% 421ms ± 3% ~ (p=0.269 n=57+56) BM_MapInsertSeq<Map<int*, int*>>/1 33.7ns ± 3% 33.7ns ± 3% ~ (p=0.620 n=55+55) BM_MapInsertSeq<Map<int*, int*>>/2 36.7ns ± 3% 36.5ns ± 3% ~ (p=0.160 n=55+56) BM_MapInsertSeq<Map<int*, int*>>/3 41.1ns ± 2% 41.0ns ± 4% ~ (p=0.284 n=54+56) BM_MapInsertSeq<Map<int*, int*>>/4 45.0ns ± 3% 53.9ns ± 4% +19.70% (p=0.000 n=57+56) BM_MapInsertSeq<Map<int*, int*>>/8 77.1ns ± 3% 80.9ns ± 4% +4.98% (p=0.000 n=55+57) BM_MapInsertSeq<Map<int*, int*>>/16 130ns ± 3% 136ns ± 4% +4.42% (p=0.000 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/32 244ns ± 3% 246ns ± 4% +0.95% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/64 620ns ± 3% 674ns ± 3% +8.83% (p=0.000 n=55+57) BM_MapInsertSeq<Map<int*, int*>>/256 2.93µs ± 3% 2.88µs ± 3% -1.73% (p=0.000 n=56+56) BM_MapInsertSeq<Map<int*, int*>>/4096 54.0µs ± 3% 50.8µs ± 4% -6.01% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/65536 1.18ms ± 2% 1.17ms ± 4% ~ (p=0.083 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/1048576 28.9ms ± 4% 29.1ms ± 5% +0.91% (p=0.007 n=55+56) BM_MapInsertSeq<Map<int*, int*>>/16777216 914ms ± 2% 919ms ± 3% +0.56% (p=0.015 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/56 404ns ± 3% 427ns ± 4% +5.60% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/224 1.88µs ± 3% 1.87µs ± 4% -0.68% (p=0.013 n=55+53) BM_MapInsertSeq<Map<int*, int*>>/3584 34.2µs ± 3% 32.9µs ± 4% -4.02% (p=0.000 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/57344 768µs ± 3% 756µs ± 3% -1.53% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int*, int*>>/917504 16.4ms ± 5% 16.5ms ± 7% ~ (p=0.303 n=56+57) BM_MapInsertSeq<Map<int*, int*>>/14680064 607ms ± 2% 613ms ± 3% +0.92% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/1 34.1ns ± 3% 34.2ns ± 4% ~ (p=0.288 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/2 37.4ns ± 3% 37.5ns ± 3% ~ (p=0.316 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/3 41.8ns ± 4% 49.1ns ± 3% +17.45% (p=0.000 n=57+56) BM_MapInsertSeq<Map<int, llvm::StringRef>>/4 54.6ns ± 3% 53.9ns ± 5% -1.35% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/8 81.4ns ± 3% 81.4ns ± 4% ~ (p=0.956 n=56+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/16 139ns ± 3% 139ns ± 3% ~ (p=0.754 n=57+56) BM_MapInsertSeq<Map<int, llvm::StringRef>>/32 256ns ± 3% 250ns ± 4% -2.32% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/64 705ns ± 4% 687ns ± 3% -2.56% (p=0.000 n=53+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/256 2.95µs ± 5% 3.05µs ± 3% +3.42% (p=0.000 n=52+55) BM_MapInsertSeq<Map<int, llvm::StringRef>>/4096 49.6µs ± 3% 50.8µs ± 4% +2.44% (p=0.000 n=55+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/65536 1.39ms ± 3% 1.40ms ± 3% +0.65% (p=0.004 n=57+56) BM_MapInsertSeq<Map<int, llvm::StringRef>>/1048576 37.7ms ± 4% 38.1ms ± 4% +1.07% (p=0.001 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/16777216 1.20s ± 3% 1.20s ± 3% +0.50% (p=0.040 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/56 432ns ± 3% 414ns ± 3% -3.99% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/224 1.92µs ± 4% 1.89µs ± 4% -1.48% (p=0.000 n=52+55) BM_MapInsertSeq<Map<int, llvm::StringRef>>/3584 31.5µs ± 4% 32.1µs ± 4% +1.89% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/57344 757µs ± 3% 748µs ± 3% -1.28% (p=0.000 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/917504 21.9ms ± 4% 22.1ms ± 5% ~ (p=0.096 n=57+57) BM_MapInsertSeq<Map<int, llvm::StringRef>>/14680064 735ms ± 3% 737ms ± 3% ~ (p=0.208 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/1 41.5ns ± 3% 41.4ns ± 4% ~ (p=0.790 n=54+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/2 50.6ns ± 4% 50.6ns ± 5% ~ (p=0.684 n=53+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/3 59.7ns ± 4% 59.4ns ± 4% ~ (p=0.277 n=55+53) BM_MapInsertSeq<Map<llvm::StringRef, int>>/4 68.5ns ± 5% 68.2ns ± 5% ~ (p=0.623 n=54+55) BM_MapInsertSeq<Map<llvm::StringRef, int>>/8 107ns ± 5% 107ns ± 9% ~ (p=0.359 n=54+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/16 200ns ± 6% 200ns ± 6% ~ (p=0.772 n=56+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/32 373ns ± 8% 371ns ± 7% ~ (p=0.541 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/64 1.11µs ± 9% 1.09µs ± 8% -2.09% (p=0.003 n=56+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/256 5.61µs ± 5% 5.48µs ± 7% -2.42% (p=0.000 n=54+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/4096 153µs ± 4% 147µs ± 6% -3.80% (p=0.000 n=54+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/65536 3.24ms ± 3% 3.10ms ± 3% -4.19% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/1048576 100ms ± 2% 98ms ± 3% -1.97% (p=0.000 n=56+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/16777216 2.45s ± 2% 2.40s ± 3% -2.09% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/56 637ns ± 8% 630ns ± 8% ~ (p=0.101 n=56+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/224 3.77µs ± 6% 3.68µs ± 6% -2.42% (p=0.000 n=56+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/3584 92.1µs ± 7% 88.4µs ± 6% -4.04% (p=0.000 n=57+56) BM_MapInsertSeq<Map<llvm::StringRef, int>>/57344 1.99ms ± 4% 1.92ms ± 3% -3.47% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/917504 62.1ms ± 4% 60.9ms ± 3% -1.93% (p=0.000 n=57+57) BM_MapInsertSeq<Map<llvm::StringRef, int>>/14680064 1.53s ± 3% 1.50s ± 3% -1.85% (p=0.000 n=57+57) ``` --------- Co-authored-by: Chandler Carruth <chandlerc@gmail.com> Co-authored-by: Carbon Infra Bot <carbon-external-infra@google.com> |
||
|
Chandler Carruth
|
a2af7ad8f0 |
Improve hashtable prefetching (#4585)
I had removed most but not all of the hashtable prefetching during development because I wasn't confident in the benchmarking results. However, I never revisited this once the benchmarking infrastructure improved and there were solid and stable results. This factors the two interesting prefetch patterns I've seen for this style of hashtable into helpers that are always called, and provides macros that can be used during the build to configure exactly which prefetch strategies are enabled. Benchmarking these and gaining confidence is very frustrating -- even now with the improved infrastructure, the noise is much higher than I would like. But it seems clear that *some* prefetching is a significant win. It also seems like enabling both results in too much prefetch traffic. And the entry group prefetch appears to be significantly more effective, both for the most interesting of the microbenchmarks and maybe most importantly for our compilation benchmarks. There, AMD is helped substantially and M1 seems to be helped some (although harder to measure). AMD server benchmark numbers: ``` name old cpu/op new cpu/op delta BM_CompileAPIFileDenseDecls<Phase::Lex>/256 35.0µs ± 2% 34.2µs ± 2% -2.40% (p=0.000 n=20+19) BM_CompileAPIFileDenseDecls<Phase::Lex>/1024 156µs ± 2% 151µs ± 2% -3.18% (p=0.000 n=20+19) BM_CompileAPIFileDenseDecls<Phase::Lex>/4096 625µs ± 1% 605µs ± 1% -3.22% (p=0.000 n=19+18) BM_CompileAPIFileDenseDecls<Phase::Lex>/16384 2.79ms ± 1% 2.69ms ± 2% -3.67% (p=0.000 n=17+19) BM_CompileAPIFileDenseDecls<Phase::Lex>/65536 12.1ms ± 1% 11.6ms ± 1% -4.30% (p=0.000 n=17+18) BM_CompileAPIFileDenseDecls<Phase::Lex>/262144 56.6ms ± 1% 53.8ms ± 1% -5.00% (p=0.000 n=18+17) BM_CompileAPIFileDenseDecls<Phase::Parse>/256 61.1µs ± 2% 61.7µs ± 1% +0.87% (p=0.000 n=19+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/1024 288µs ± 1% 290µs ± 1% +0.55% (p=0.004 n=20+20) BM_CompileAPIFileDenseDecls<Phase::Parse>/4096 1.16ms ± 1% 1.16ms ± 1% -0.54% (p=0.000 n=17+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/16384 4.98ms ± 1% 4.91ms ± 1% -1.39% (p=0.000 n=20+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/65536 20.9ms ± 1% 20.5ms ± 1% -1.86% (p=0.000 n=20+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/262144 92.1ms ± 1% 90.2ms ± 1% -2.12% (p=0.000 n=18+20) BM_CompileAPIFileDenseDecls<Phase::Check>/256 1.16ms ± 2% 1.16ms ± 1% ~ (p=0.931 n=19+19) BM_CompileAPIFileDenseDecls<Phase::Check>/1024 2.17ms ± 2% 2.16ms ± 1% ~ (p=0.247 n=20+19) BM_CompileAPIFileDenseDecls<Phase::Check>/4096 6.07ms ± 1% 6.04ms ± 1% -0.48% (p=0.007 n=19+19) BM_CompileAPIFileDenseDecls<Phase::Check>/16384 22.4ms ± 1% 22.2ms ± 1% -0.99% (p=0.000 n=20+19) BM_CompileAPIFileDenseDecls<Phase::Check>/65536 93.3ms ± 1% 92.2ms ± 1% -1.23% (p=0.000 n=20+18) BM_CompileAPIFileDenseDecls<Phase::Check>/262144 400ms ± 1% 391ms ± 1% -2.15% (p=0.000 n=20+18) ``` |
||
|
4845f40dff |
Switch CARBON_CHECK to a format string API (#4285)
This switches `DCHECK` and `FATAL` as well. The goal is to reduce the code size impact of these assertions so that we can keep more of them enabled. Currently, the largest cost I see from `CHECK` is not the actual check or the cold code itself, but actually the failure to inline trivial functions due to the presence of the cold code. This means that our goal isn't to reduce apparent code size in the final binary but the LLVM IR cost assessed for these routines in the inliner, which closely correlates with code size but is a bit different. As discussed in #4283, experimentation shows that a single function call with a minimal number of arguments is the lowest cost model for these. This is easily achieved with a format-string API that internally uses `llvm::formatv`. This PR is essentially the `CHECK` version of #4283. However, the check macros are substantially harder to make work with both format strings and streaming because they also take a condition. Also, unexpectedly, I was very successful at devising a regular expression based automated rewrite from the streaming to the format string form with only low 10s of manual fixes. This includes compacting strings broken up across lines, etc. Given how well that went, I've prepared this PR which just directly switches to the format string API and migrate everything to use it. One nice side-effect is that the format string approach ends up greatly simplifying the implementation here as well. This is ... *shockingly* effective. Parsing speeds up by more than 3% with just this change. And checking speeds up by **8%** with this change alone: ``` BM_CompileAPIFileDenseDecls<Phase::Parse>/256 86.3µs ± 1% 82.9µs ± 1% -3.94% (p=0.000 n=17+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/1024 431µs ± 1% 415µs ± 1% -3.76% (p=0.000 n=18+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/4096 1.77ms ± 1% 1.71ms ± 1% -3.18% (p=0.000 n=18+19) BM_CompileAPIFileDenseDecls<Phase::Parse>/16384 7.44ms ± 1% 7.17ms ± 2% -3.56% (p=0.000 n=18+20) BM_CompileAPIFileDenseDecls<Phase::Parse>/65536 30.7ms ± 1% 29.7ms ± 1% -3.15% (p=0.000 n=18+20) BM_CompileAPIFileDenseDecls<Phase::Parse>/262144 131ms ± 1% 127ms ± 1% -2.81% (p=0.000 n=18+18) BM_CompileAPIFileDenseDecls<Phase::Check>/256 878µs ± 2% 800µs ± 1% -8.91% (p=0.000 n=19+20) BM_CompileAPIFileDenseDecls<Phase::Check>/1024 1.88ms ± 2% 1.72ms ± 1% -8.56% (p=0.000 n=19+20) BM_CompileAPIFileDenseDecls<Phase::Check>/4096 5.78ms ± 2% 5.28ms ± 1% -8.70% (p=0.000 n=20+18) BM_CompileAPIFileDenseDecls<Phase::Check>/16384 21.9ms ± 1% 20.1ms ± 1% -8.02% (p=0.000 n=18+20) BM_CompileAPIFileDenseDecls<Phase::Check>/65536 90.4ms ± 2% 83.1ms ± 1% -8.04% (p=0.000 n=19+20) BM_CompileAPIFileDenseDecls<Phase::Check>/262144 381ms ± 2% 352ms ± 1% -7.79% (p=0.000 n=19+19) ``` --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com> |
||
|
Chandler Carruth
|
a8748f3e2d |
Key context improvements (#4095)
This injects a customization point for hashtable-specific equality testing that the key context uses by default. While this is rarely needed, there are LLVM types where it is necessary and it seems a good general tool to have to avoid unnecessary complexity from custom key contexts when a simple customization of equality is all that is required. This also adds a CRTP mixin for implementing a common pattern of key contexts where the context provides translation of some key types into another type, potentially using state. Rather than having to implement the entire key context API, code can derive from this template and simply provide a set of overloads for the types it wants to translate. Any key types used which can be passed to one of those overloads will get translated before following the same logic as the default key context. While this updates the only usage so far of this pattern, a subsequent PR will add several more users making the pattern worth abstracting here. |
||
|
Chandler Carruth
|
6310293330 |
Fix some subtle UB found by MSan. (#4093)
Technically, any small size buffer's lifetime has ended by the time we get to the base's destructor. This means its no longer valid to access table contents if stored there from the base destructor. We need to handle destruction in the table class instead. This ends up being a trivial change because the logic is already factored out, we just need to call it from a different point. |
||
|
Chandler CarruthandRichard Smith
|
af6312a3aa |
Add assignment support to the hashtables. (#4066)
This lets copy and move assignment work. While it's a bit suboptimal to do assignment with these tables, it still seems like an unreasonable burden to not allow the basics to work. Even the toolchain ended up doing this in a few places. --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Chandler Carruthandjosh11b
|
07fadc6474 |
Add growth API to the new hashtables. (#4044)
This adds two different growth APIs. This is instead of the more conventional STL `reserve` method. One allows users that aren't trying to grow in anticipation of an *exact* count of insertions, but generally trying to size the table to the correct ballpark with a power-of-two estimate. The other API allows pre-growing to allow a specific number of insertions to be performed without further growth. This API takes the maximum load factor and other implementation details into account. --------- Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com> |
||
|
Chandler CarruthandRichard Smith
|
3be57b71e0 |
Collect more detailed metrics on hashtables. (#4046)
Previously we just looked at the raw count of probed keys. Now, we compute the average and max of both the probe _distance_ measured in the number of _groups_ probed, and the number of probe _compares_ measured in the compares required _before_ finding the matching entry. This lets us understand the relative impact of probe-distance vs. tag collisions on a given set of benchmark keys. Some of this is motivated by considering additional optimization techniques similar to those used in Boost's table and the F14 table from Facebook/Meta. --------- Co-authored-by: Richard Smith <richard@metafoo.co.uk> |
||
|
Chandler Carruthandjosh11b
|
21a81bc59e |
Introduce custom hash table data structures. (#3940)
The hash table design is heavily based on Abseil's ["Swiss Tables"][swiss-tables] design. It uses an array of bytes storing metadata about each entry and an array of entries where each is a pair of key and value. The metadata byte consists of 7-bits of hash of the key (distinct from the bits used to index the table), and one bit indicating the presence of a special entry -- either empty or deleted. [swiss-tables]: https://abseil.io/about/design/swisstables There are a large range of optimizations and other nuanced aspects of this hash table design and implementation, a good point to understand that context is `raw_hashtable.h` which has an overview of the design and references to various other files for relevant details. --------- Co-authored-by: josh11b <15258583+josh11b@users.noreply.github.com> |