mirror of
https://github.com/carbon-language/carbon-lang.git
synced 2026-11-10 12:28:25 +00:00
Remove the hashtable seed (#7871)
Hashtables seed every hash with the address of a global,
`ComputeSeed()`, so that ASLR varies layout and iteration order between
runs. Debug builds already vary iteration order directly in `entries()`,
so the seed only matters in release builds. There it costs a register
and an XOR on the latency path of every integer or pointer hash, and it
redraws a table's collisions in each process.
Remove the seed from `HashValue`, `Hasher`, `CarbonHashValue`,
`CustomHashValue::Hash`, and `HashKey`. Hashing starts from a zero
buffer, so integer keys hash as `WeakMix(key)`. `global_addr_seed`
remains in debug builds to seed the iteration-order entropy.
`ClangDeclSignature` chained its hash through the seed, and now hashes
its fields with one `Hasher`.
The seed also kept some short-input paths from collapsing. Without it,
every `{i, 0, 0}` array of `int32_t` hashed the same, all-zero 4-8 byte
inputs lost their size, and the 17-32 byte path multiplied the size
sample by a byte-shifted copy of itself. Each `Mix` operand of the
short-input paths now XORs in a 32-bit constant from
`ImmediateRandomData` in place of a 64-bit chunk of `StaticRandomData`,
which they now only sample to encode sizes. `XorImmediate` keeps these
as `xor $imm32` on x86-64; without its `asm` barrier, a read feeding the
XOR is folded in instead and the constant ends up in a register that
loops hoist.
Cycles against trunk on Zen 5, from `bench_runner.py` with 48 runs per
side, each run at a random code layout, and table sizes of 64, 3584, and
57344:
- Integer and pointer hits: -3.0% to -5.7% in `Map`, -0.8% to -2.4% in
`Set`.
- Integer and pointer misses: up to -12.6%, except `Set<int*>` at +1.4%
(64) and +6.1% (3584).
- String keys: -2.8% to +2.1%, with `Map` updates at +1.1% to +1.6%.
- Hash latency, without layout randomization: integers and pointers
-9.8%, empty strings +5.4% (0.1 cycles), other strings not significant.
Same method on an Apple M1, with #7870 on both sides. Without it, this
change moves benchmark loop counters between `sp`- and `x29`-relative
stack slots, which alone makes `Set<int>` lookups up to 49% slower on
the M1.
- Integer and pointer hits: -1.0% to -10.3% in `Map`, -24.5% to +1.2% in
`Set`.
- Integer and pointer misses: -3.9% to -48.0%, except +7.2% to +59.3% at
3584, from longer probes (instructions +6.4% to +70.6%).
- Inserting 64 integer keys: +12.1% to +15.0%, from store memory-order
violations.
- String keys: -6.6% to +3.9%.
- Hash latency: integers and pointers -9.0% to -10.0%, empty strings
+48.8% (1 cycle).
Release builds now iterate a given table in the same order in every run.
Code that depends on that order still fails in debug builds.
Assisted-by: Claude Code
This commit is contained in:
1 parent
bf64992afb
commit
13ecad24be
17 files changed
+226
-251
No files matched your search
+139
-121
@@ -5,6 +5,7 @@
|
||||
#ifndef CARBON_COMMON_HASHING_H_
|
||||
#define CARBON_COMMON_HASHING_H_
|
||||
|
||||
#include <bit>
|
||||
#include <concepts>
|
||||
#include <string>
|
||||
#include <tuple>
|
||||
@@ -75,21 +76,16 @@ class HashCode : public Printable<HashCode> {
|
||||
uint64_t value_ = 0;
|
||||
};
|
||||
|
||||
// Computes a hash code for the provided value, incorporating the provided seed.
|
||||
//
|
||||
// The seed doesn't need to be of any particular high quality, but a zero seed
|
||||
// has bad effects in several places. Prefer the unseeded routine rather than
|
||||
// providing a zero here.
|
||||
// Computes a hash code for the provided value.
|
||||
//
|
||||
// This **not** a cryptographically secure or stable hash -- it is only designed
|
||||
// for use with in-memory hash table style data structures. Being fast and
|
||||
// effective for that use case is the guiding principle of its design.
|
||||
//
|
||||
// There is no guarantee that the values produced are stable from execution to
|
||||
// execution. For speed and quality reasons, the implementation does not
|
||||
// introduce any variance to defend against accidental dependencies. As a
|
||||
// consequence, it is strongly encouraged to use a seed that varies from
|
||||
// execution to execution to avoid depending on specific values produced.
|
||||
// There is no guarantee that the values produced are stable between versions
|
||||
// of this code. Within one build, the hash of a value is the same in every
|
||||
// execution: the function is unseeded, so hashtable layouts and collisions are
|
||||
// reproducible between runs. Code must not depend on the specific values.
|
||||
//
|
||||
// The algorithm used is most heavily based on [Abseil's hashing algorithm][1],
|
||||
// with some additional ideas and inspiration from the fallback hashing
|
||||
@@ -100,13 +96,13 @@ class HashCode : public Printable<HashCode> {
|
||||
// [2]: https://github.com/tkaitchuck/aHash/wiki/AHash-fallback-algorithm
|
||||
// [3]: https://docs.rs/fxhash/latest/fxhash/
|
||||
//
|
||||
// This hash algorithm does *not* defend against hash flooding. While it can be
|
||||
// viewed as "keyed" on the seed, it is expected to be possible to craft inputs
|
||||
// for some data types that cancel out the seed used and manufacture endlessly
|
||||
// colliding sets of keys. In general, this function works to be *fast* for hash
|
||||
// tables. If you need to defend against hash flooding, either directly use a
|
||||
// data structure with strong worst-case guarantees, or a hash table which
|
||||
// detects catastrophic collisions and falls back to such a data structure.
|
||||
// This hash algorithm does *not* defend against hash flooding. It is unkeyed,
|
||||
// and it is expected to be possible to craft inputs for some data types that
|
||||
// manufacture endlessly colliding sets of keys. In general, this function works
|
||||
// to be *fast* for hash tables. If you need to defend against hash flooding,
|
||||
// either directly use a data structure with strong worst-case guarantees, or a
|
||||
// hash table which detects catastrophic collisions and falls back to such a
|
||||
// data structure.
|
||||
//
|
||||
// This hash function is heavily optimized for *latency* over *quality*. Modern
|
||||
// hash tables designs can efficiently handle reasonable collision rates,
|
||||
@@ -147,7 +143,7 @@ class HashCode : public Printable<HashCode> {
|
||||
// `CarbonHashValue` with the following signature:
|
||||
//
|
||||
// ```cpp
|
||||
// auto CarbonHashValue(const YourType& value, uint64_t seed) -> HashCode;
|
||||
// auto CarbonHashValue(const YourType& value) -> HashCode;
|
||||
// ```
|
||||
//
|
||||
// The extension point needs to ensure that values that compare equal (including
|
||||
@@ -173,16 +169,6 @@ class HashCode : public Printable<HashCode> {
|
||||
//
|
||||
// [4]: https://en.wikipedia.org/wiki/Avalanche_effect
|
||||
template <typename T>
|
||||
inline auto HashValue(const T& value, uint64_t seed) -> HashCode;
|
||||
|
||||
// The same as the seeded version of `HashValue` but without callers needing to
|
||||
// provide a seed.
|
||||
//
|
||||
// Generally prefer the seeded version, but this is available if there is no
|
||||
// reasonable seed. In particular, this will behave better than using a seed of
|
||||
// `0`. One important use case is for recursive hashing of sub-objects where
|
||||
// appropriate or needed.
|
||||
template <typename T>
|
||||
inline auto HashValue(const T& value) -> HashCode;
|
||||
|
||||
// Object and APIs that eventually produce a hash code.
|
||||
@@ -199,8 +185,8 @@ inline auto HashValue(const T& value) -> HashCode;
|
||||
//
|
||||
// Example usage:
|
||||
// ```cpp
|
||||
// auto CarbonHashValue(const MyType& value, uint64_t seed) -> HashCode {
|
||||
// Hasher hasher(seed);
|
||||
// auto CarbonHashValue(const MyType& value) -> HashCode {
|
||||
// Hasher hasher;
|
||||
// hasher.HashTwo(value.x, value.y);
|
||||
// return static_cast<HashCode>(hasher);
|
||||
// }
|
||||
@@ -221,7 +207,6 @@ inline auto HashValue(const T& value) -> HashCode;
|
||||
class Hasher {
|
||||
public:
|
||||
Hasher() = default;
|
||||
explicit Hasher(uint64_t seed) : buffer(seed) {}
|
||||
|
||||
Hasher(Hasher&& arg) = default;
|
||||
Hasher(const Hasher& arg) = delete;
|
||||
@@ -311,6 +296,13 @@ class Hasher {
|
||||
static auto Read4(const std::byte* data) -> uint64_t;
|
||||
static auto Read8(const std::byte* data) -> uint64_t;
|
||||
|
||||
// Returns `value` XOR-ed with `imm` sign-extended to 64 bits, keeping `imm`
|
||||
// an immediate operand of the XOR on x86-64. When `value` comes directly from
|
||||
// a read, instruction selection otherwise folds the read into the XOR and
|
||||
// materializes `imm` in a register, which a loop then hoists and keeps live.
|
||||
// Hiding `value` from the optimizer prevents that.
|
||||
static auto XorImmediate(uint64_t value, int32_t imm) -> uint64_t;
|
||||
|
||||
// Similar to the `ReadN` functions, but supports reading a range of different
|
||||
// bytes provided by the size *without branching on the size*. The lack of
|
||||
// branches is often key, and the code in these routines works to be efficient
|
||||
@@ -355,7 +347,9 @@ class Hasher {
|
||||
// multiplication. However, it is not *necessary* as we do capture the
|
||||
// complete 128-bit result. Where reasonable, the caller should XOR random
|
||||
// data into operands before calling `Mix` to try and increase the
|
||||
// distribution of bits feeding the multiply.
|
||||
// distribution of bits feeding the multiply. The operands of each `Mix` use
|
||||
// `ImmediateRandomData` for this, and use `StaticRandomData` only to encode
|
||||
// a size.
|
||||
static auto Mix(uint64_t lhs, uint64_t rhs) -> uint64_t;
|
||||
|
||||
// An alternative to `Mix` that is significantly weaker but also lower
|
||||
@@ -392,10 +386,10 @@ class Hasher {
|
||||
|
||||
// Random data taken from the hexadecimal digits of Pi's fractional component,
|
||||
// written in lexical order for convenience of reading. The resulting
|
||||
// byte-stream will be different due to little-endian integers. These can be
|
||||
// used directly for convenience rather than calling `SampleRandomData`, but
|
||||
// be aware that this is the underlying pool. The goal is to reuse the same
|
||||
// single cache-line of constant data.
|
||||
// byte-stream will be different due to little-endian integers. This pool is
|
||||
// sampled with `SampleRandomData` to encode sizes into the hash, and is used
|
||||
// directly by the large-input routine. The goal is to reuse the same single
|
||||
// cache-line of constant data.
|
||||
//
|
||||
// The initializers here can be generated with the following shell script,
|
||||
// which will generate 8 64-bit values and one more digit. The `bc` command's
|
||||
@@ -417,6 +411,33 @@ class Hasher {
|
||||
0xc0ac'29b7'c97c'50dd, 0x3f84'd5b5'b547'0917,
|
||||
};
|
||||
|
||||
// Random data XOR-ed into the operands of `Mix` so that common inputs, such
|
||||
// as all-zero bytes, don't zero an operand, and to spread bits across the
|
||||
// multiply. These are 32-bit values, sign-extended on use: 32 bits of
|
||||
// random data is enough for this, and x86-64 can XOR a sign-extended 32-bit
|
||||
// value in as an immediate operand, where a full 64-bit constant costs an
|
||||
// extra instruction to materialize. They are also independent of
|
||||
// `StaticRandomData`, so they can't correlate with a sample of that pool on
|
||||
// the other operand of the same `Mix`.
|
||||
//
|
||||
// The values are successive 32-bit chunks of the hexadecimal digits of e's
|
||||
// fractional component. The initializers can be generated with the
|
||||
// following shell script:
|
||||
//
|
||||
// ```sh
|
||||
// echo 'obase=16; scale=40; e(1)' | bc -l \
|
||||
// | cut -c 3-34 | tr '[:upper:]' '[:lower:]' \
|
||||
// | sed -e "s/.\{4\}/&'/g" \
|
||||
// | sed -e "s/\(.\{4\}'.\{4\}\)'/0x\1,\n/g"
|
||||
// ```
|
||||
static constexpr std::array<int32_t, 4> ImmediateRandomData =
|
||||
std::bit_cast<std::array<int32_t, 4>>(std::array<uint32_t, 4>{
|
||||
0xb7e1'5162,
|
||||
0x8aed'2a6a,
|
||||
0xbf71'5880,
|
||||
0x9cf4'f3c7,
|
||||
});
|
||||
|
||||
// We need a multiplicative hashing constant for both 64-bit multiplicative
|
||||
// hashing fast paths and some other 128-bit folded multiplies. We use an
|
||||
// empirically better constant compared to Knuth's, Rust's FxHash, and others
|
||||
@@ -448,7 +469,7 @@ class Hasher {
|
||||
static constexpr uint64_t MulConstant = 0x79d5'f9e0'de1e'8cf5U;
|
||||
|
||||
private:
|
||||
uint64_t buffer;
|
||||
uint64_t buffer = 0;
|
||||
};
|
||||
|
||||
// A dedicated namespace for `CarbonHashValue` overloads that are not found by
|
||||
@@ -460,42 +481,37 @@ class Hasher {
|
||||
namespace InternalHashDispatch {
|
||||
|
||||
template <typename T>
|
||||
inline auto CarbonHashValue(llvm::ArrayRef<T> values, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(llvm::ArrayRef<T> values) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.HashArray(values);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
inline auto CarbonHashValue(llvm::ArrayRef<std::byte> bytes, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(llvm::ArrayRef<std::byte> bytes) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.HashSizedBytes(bytes);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
// Hashing implementation for `llvm::StringRef`. We forward all the other
|
||||
// string-like types that support heterogeneous lookup to this one.
|
||||
inline auto CarbonHashValue(llvm::StringRef value, uint64_t seed) -> HashCode {
|
||||
return CarbonHashValue(
|
||||
llvm::ArrayRef(reinterpret_cast<const std::byte*>(value.data()),
|
||||
value.size()),
|
||||
seed);
|
||||
inline auto CarbonHashValue(llvm::StringRef value) -> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(
|
||||
reinterpret_cast<const std::byte*>(value.data()), value.size()));
|
||||
}
|
||||
|
||||
inline auto CarbonHashValue(std::string_view value, uint64_t seed) -> HashCode {
|
||||
return CarbonHashValue(llvm::StringRef(value.data(), value.size()), seed);
|
||||
inline auto CarbonHashValue(std::string_view value) -> HashCode {
|
||||
return CarbonHashValue(llvm::StringRef(value.data(), value.size()));
|
||||
}
|
||||
|
||||
inline auto CarbonHashValue(const std::string& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
return CarbonHashValue(llvm::StringRef(value.data(), value.size()), seed);
|
||||
inline auto CarbonHashValue(const std::string& value) -> HashCode {
|
||||
return CarbonHashValue(llvm::StringRef(value.data(), value.size()));
|
||||
}
|
||||
|
||||
template <unsigned Length>
|
||||
inline auto CarbonHashValue(const llvm::SmallString<Length>& value,
|
||||
uint64_t seed) -> HashCode {
|
||||
return CarbonHashValue(llvm::StringRef(value.data(), value.size()), seed);
|
||||
inline auto CarbonHashValue(const llvm::SmallString<Length>& value)
|
||||
-> HashCode {
|
||||
return CarbonHashValue(llvm::StringRef(value.data(), value.size()));
|
||||
}
|
||||
|
||||
// Support types that are array-like by building an `llvm::ArrayRef` out of
|
||||
@@ -503,37 +519,32 @@ inline auto CarbonHashValue(const llvm::SmallString<Length>& value,
|
||||
// because that type supports building a synthetic array out of any single
|
||||
// element.
|
||||
template <typename T>
|
||||
inline auto CarbonHashValue(const std::vector<T>& arg, uint64_t seed)
|
||||
-> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg), seed);
|
||||
inline auto CarbonHashValue(const std::vector<T>& arg) -> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg));
|
||||
}
|
||||
template <typename T>
|
||||
inline auto CarbonHashValue(const llvm::SmallVectorImpl<T>& arg, uint64_t seed)
|
||||
-> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg), seed);
|
||||
inline auto CarbonHashValue(const llvm::SmallVectorImpl<T>& arg) -> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg));
|
||||
}
|
||||
template <typename T, size_t N>
|
||||
inline auto CarbonHashValue(const std::array<T, N>& arg, uint64_t seed)
|
||||
-> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg), seed);
|
||||
inline auto CarbonHashValue(const std::array<T, N>& arg) -> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg));
|
||||
}
|
||||
template <typename T, size_t N>
|
||||
inline auto CarbonHashValue(const T (&arg)[N], uint64_t seed) -> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg), seed);
|
||||
inline auto CarbonHashValue(const T (&arg)[N]) -> HashCode {
|
||||
return CarbonHashValue(llvm::ArrayRef(arg));
|
||||
}
|
||||
|
||||
template <typename... Ts>
|
||||
inline auto CarbonHashValue(const std::tuple<Ts...>& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(const std::tuple<Ts...>& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
std::apply([&](const auto&... args) { hasher.Hash(args...); }, value);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
template <typename T, typename U>
|
||||
inline auto CarbonHashValue(const std::pair<T, U>& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(const std::pair<T, U>& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.Hash(value.first, value.second);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
@@ -550,8 +561,8 @@ concept HasCustomHashValue = requires { CustomHashValue<T>::Hash; };
|
||||
// overload available for a particular type, either in this namespace or found
|
||||
// via ADL. Note that this should not be moved above any overloads.
|
||||
template <typename T>
|
||||
concept HasCarbonHashValue = requires(const T& value, uint64_t seed) {
|
||||
{ CarbonHashValue(value, seed) } -> std::same_as<HashCode>;
|
||||
concept HasCarbonHashValue = requires(const T& value) {
|
||||
{ CarbonHashValue(value) } -> std::same_as<HashCode>;
|
||||
};
|
||||
|
||||
// C++ guarantees this is true for the unsigned variants, but we require it for
|
||||
@@ -596,21 +607,21 @@ concept CanHashAsRawDataType = std::same_as<T, std::nullptr_t> ||
|
||||
// overloads, either here, or via ADL. Note that similar to
|
||||
// `HasCarbonHashValue`, this must not be moved above any of those overloads.
|
||||
template <typename T>
|
||||
inline auto DispatchImpl(const T& value, uint64_t seed) -> HashCode {
|
||||
inline auto DispatchImpl(const T& value) -> HashCode {
|
||||
if constexpr (HasCarbonHashValue<T>) {
|
||||
// If we have an explicit overload for `CarbonHashValue`, call it. This may
|
||||
// be provided above or via ADL, and is preferred as it represents an
|
||||
// explicit request for how the type is hashed.
|
||||
return CarbonHashValue(value, seed);
|
||||
return CarbonHashValue(value);
|
||||
} else if constexpr (HasCustomHashValue<T>) {
|
||||
// If we have an explicit specialization for `CustomHashValue`, call it.
|
||||
// This is a fallback explicit hashing path that doesn't require ADL or
|
||||
// being in this header.
|
||||
return CustomHashValue<T>::Hash(value, seed);
|
||||
return CustomHashValue<T>::Hash(value);
|
||||
} else if constexpr (CanHashAsRawDataType<T>) {
|
||||
// There was no explicit overload or specialization to call, but the type
|
||||
// allows us to hash it as raw data, do so.
|
||||
Hasher hasher(seed);
|
||||
Hasher hasher;
|
||||
hasher.HashRaw(MapToRawDataType(value));
|
||||
return static_cast<HashCode>(hasher);
|
||||
} else {
|
||||
@@ -626,18 +637,9 @@ inline auto DispatchImpl(const T& value, uint64_t seed) -> HashCode {
|
||||
|
||||
} // namespace InternalHashDispatch
|
||||
|
||||
template <typename T>
|
||||
inline auto HashValue(const T& value, uint64_t seed) -> HashCode {
|
||||
return InternalHashDispatch::DispatchImpl(value, seed);
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
inline auto HashValue(const T& value) -> HashCode {
|
||||
// When a seed isn't provided, use the last 64-bit chunk of random data. Other
|
||||
// chunks (especially the first) are more often XOR-ed with the seed and risk
|
||||
// cancelling each other out and feeding a zero to a `Mix` call in a way that
|
||||
// sharply increasing collisions.
|
||||
return HashValue(value, Hasher::StaticRandomData[7]);
|
||||
return InternalHashDispatch::DispatchImpl(value);
|
||||
}
|
||||
|
||||
constexpr auto HashCode::ExtractIndex() -> ssize_t { return value_; }
|
||||
@@ -689,6 +691,13 @@ inline auto Hasher::Read8(const std::byte* data) -> uint64_t {
|
||||
return result;
|
||||
}
|
||||
|
||||
inline auto Hasher::XorImmediate(uint64_t value, int32_t imm) -> uint64_t {
|
||||
#if defined(__x86_64__)
|
||||
__asm__("" : "+r"(value));
|
||||
#endif
|
||||
return value ^ static_cast<uint64_t>(static_cast<int64_t>(imm));
|
||||
}
|
||||
|
||||
inline auto Hasher::Read1To3(const std::byte* data, ssize_t size) -> uint64_t {
|
||||
// Use carefully crafted indexing to avoid branches on the exact size while
|
||||
// reading.
|
||||
@@ -745,23 +754,16 @@ inline auto Hasher::HashDense(uint64_t data) -> void {
|
||||
inline auto Hasher::HashDense(uint64_t data0, uint64_t data1) -> void {
|
||||
// When hashing two chunks of data at the same time, we XOR it with random
|
||||
// data to avoid common inputs from having especially bad multiplicative
|
||||
// effects. We also XOR in the starting buffer as seed or to chain. Note that
|
||||
// we don't use *consecutive* random data 64-bit values to avoid a common
|
||||
// compiler "optimization" of loading both 64-bit chunks into a 128-bit vector
|
||||
// and doing the XOR in the vector unit. The latency of extracting the data
|
||||
// afterward eclipses any benefit. Callers will routinely have two consecutive
|
||||
// data values here, but using non-consecutive keys avoids any vectorization
|
||||
// being tempting.
|
||||
// effects. We also XOR in the starting buffer to chain.
|
||||
//
|
||||
// XOR-ing both the incoming state and a random word over the second data is
|
||||
// done to pipeline with materializing the constants and is observed to have
|
||||
// better performance than XOR-ing after the mix.
|
||||
// XOR-ing both the incoming state and random data over the second data is
|
||||
// observed to have better performance than XOR-ing after the mix.
|
||||
//
|
||||
// This roughly matches the mix pattern used in the larger mixing routines
|
||||
// from Abseil, which is a more minimal form than used in other algorithms
|
||||
// such as AHash and seems adequate for latency-optimized use cases.
|
||||
buffer =
|
||||
Mix(data0 ^ StaticRandomData[1], data1 ^ StaticRandomData[3] ^ buffer);
|
||||
buffer = Mix(XorImmediate(data0, ImmediateRandomData[3]),
|
||||
XorImmediate(data1, ImmediateRandomData[2]) ^ buffer);
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
@@ -818,7 +820,7 @@ inline auto Hasher::Hash(const Ts&... values) -> void {
|
||||
auto map_value = []<typename T>(const T& value) -> uint64_t {
|
||||
if constexpr (HasCarbonHashValue<T> || HasCustomHashValue<T>) {
|
||||
// Use the top-level `HashValue` to re-dispatch to the custom
|
||||
// implementation with a fixed seed.
|
||||
// implementation.
|
||||
return static_cast<uint64_t>(HashValue(value));
|
||||
} else if constexpr (CanHashAsRawDataType<T>) {
|
||||
auto raw_value = MapToRawDataType(value);
|
||||
@@ -828,8 +830,7 @@ inline auto Hasher::Hash(const Ts&... values) -> void {
|
||||
if constexpr (sizeof(raw_value) <= 8) {
|
||||
return ReadSmall(raw_value);
|
||||
} else {
|
||||
// Use the top-level `HashValue` to pick up a good fixed seed and hash
|
||||
// this large object as raw data.
|
||||
// Use the top-level `HashValue` to hash this large object as raw data.
|
||||
return static_cast<uint64_t>(HashValue(raw_value));
|
||||
}
|
||||
} else {
|
||||
@@ -912,13 +913,14 @@ inline auto Hasher::HashRaw(const T& value) -> void {
|
||||
if constexpr (16 < sizeof(T) && sizeof(T) <= 32) {
|
||||
CARBON_MCA_BEGIN("fixed-32b");
|
||||
// Essentially the same technique used for dynamically sized byte sequences
|
||||
// of this size, but we start with a fixed XOR of random data.
|
||||
buffer ^= StaticRandomData[0];
|
||||
uint64_t m0 = Mix(Read8(data_ptr) ^ StaticRandomData[1],
|
||||
Read8(data_ptr + 8) ^ buffer);
|
||||
// of this size, but with a fixed XOR of random data in place of the size.
|
||||
uint64_t m0 =
|
||||
Mix(XorImmediate(Read8(data_ptr), ImmediateRandomData[3]),
|
||||
XorImmediate(Read8(data_ptr + 8), ImmediateRandomData[1]) ^ buffer);
|
||||
const std::byte* tail_16b_ptr = data_ptr + (sizeof(T) - 16);
|
||||
uint64_t m1 = Mix(Read8(tail_16b_ptr) ^ StaticRandomData[3],
|
||||
Read8(tail_16b_ptr + 8) ^ buffer);
|
||||
uint64_t m1 = Mix(
|
||||
XorImmediate(Read8(tail_16b_ptr), ImmediateRandomData[2]),
|
||||
XorImmediate(Read8(tail_16b_ptr + 8), ImmediateRandomData[1]) ^ buffer);
|
||||
buffer = m0 ^ m1;
|
||||
CARBON_MCA_END("fixed-32b");
|
||||
return;
|
||||
@@ -948,7 +950,11 @@ inline auto Hasher::HashSizedBytes(llvm::ArrayRef<std::byte> bytes) -> void {
|
||||
// Note that we don't drop to the `WeakMix` routine here because we want
|
||||
// to use sampled random data to encode the size, which may not be as
|
||||
// effective without the full 128-bit folded result.
|
||||
buffer = Mix(data ^ buffer, SampleAlignedRandomData(size - 1));
|
||||
//
|
||||
// The data is XOR-ed with random data so that all-zero data doesn't zero
|
||||
// the multiply and lose the size.
|
||||
buffer = Mix(XorImmediate(data, ImmediateRandomData[0]) ^ buffer,
|
||||
SampleAlignedRandomData(size - 1));
|
||||
CARBON_MCA_END("dynamic-8b");
|
||||
return;
|
||||
}
|
||||
@@ -972,12 +978,13 @@ inline auto Hasher::HashSizedBytes(llvm::ArrayRef<std::byte> bytes) -> void {
|
||||
if (size <= 16) {
|
||||
CARBON_MCA_BEGIN("dynamic-16b");
|
||||
// Similar to the above, we optimize primarily for latency here and spread
|
||||
// the incoming data across both ends of the multiply. Note that this does
|
||||
// have a drawback -- any time one half of the mix function becomes zero it
|
||||
// will fail to incorporate any bits from the other half. However, there is
|
||||
// exactly 1 in 2^64 values for each side that achieve this, and only when
|
||||
// the size is exactly 16 -- for smaller sizes there is an overlapping byte
|
||||
// that makes this impossible unless the seed is *also* incredibly unlucky.
|
||||
// the incoming data across both ends of the multiply. Any time one half of
|
||||
// the mix function becomes zero it fails to incorporate any bits from the
|
||||
// other half, so both halves are XOR-ed with random data: a sample of the
|
||||
// pool encoding the size, and a 32-bit constant. Otherwise the common case
|
||||
// of trailing zero bytes, such as `{x, 0, 0}` as an array of `int32_t`,
|
||||
// would zero the second half and hash every `x` the same. With the random
|
||||
// data, exactly 1 in 2^64 values for each half is a problem.
|
||||
//
|
||||
// Because this hash function makes no attempt to defend against hash
|
||||
// flooding, we accept this risk in order to keep the latency low. If this
|
||||
@@ -985,7 +992,8 @@ inline auto Hasher::HashSizedBytes(llvm::ArrayRef<std::byte> bytes) -> void {
|
||||
// the 16-byte case down the next tier of cost.
|
||||
uint64_t size_hash = SampleRandomData(size);
|
||||
auto data = Read8To16(data_ptr, size);
|
||||
buffer = Mix(data.first ^ size_hash, data.second ^ buffer);
|
||||
buffer = Mix(data.first ^ size_hash,
|
||||
XorImmediate(data.second, ImmediateRandomData[1]) ^ buffer);
|
||||
CARBON_MCA_END("dynamic-16b");
|
||||
return;
|
||||
}
|
||||
@@ -994,13 +1002,23 @@ inline auto Hasher::HashSizedBytes(llvm::ArrayRef<std::byte> bytes) -> void {
|
||||
CARBON_MCA_BEGIN("dynamic-32b");
|
||||
// Do two mixes of overlapping 16-byte ranges in parallel to minimize
|
||||
// latency. We also incorporate the size by sampling random data into the
|
||||
// seed before both.
|
||||
// buffer before both.
|
||||
//
|
||||
// The first operands use 32-bit constants rather than chunks of the pool.
|
||||
// The sample reads bytes 17 through 39 of the pool, and a chunk that
|
||||
// overlaps it is the same random data shifted by some bytes. Multiplying
|
||||
// one by the other produces structured collisions.
|
||||
//
|
||||
// Both also go through `XorImmediate`, which keeps the two products at the
|
||||
// same depth. Otherwise reassociation XORs the halves of `m0` together
|
||||
// first, and on x86-64 the high half of a multiply arrives a cycle after
|
||||
// the low half, lengthening the critical path by a cycle.
|
||||
buffer ^= SampleRandomData(size);
|
||||
uint64_t m0 = Mix(Read8(data_ptr) ^ StaticRandomData[1],
|
||||
uint64_t m0 = Mix(XorImmediate(Read8(data_ptr), ImmediateRandomData[3]),
|
||||
Read8(data_ptr + 8) ^ buffer);
|
||||
|
||||
const std::byte* tail_16b_ptr = data_ptr + (size - 16);
|
||||
uint64_t m1 = Mix(Read8(tail_16b_ptr) ^ StaticRandomData[3],
|
||||
uint64_t m1 = Mix(XorImmediate(Read8(tail_16b_ptr), ImmediateRandomData[2]),
|
||||
Read8(tail_16b_ptr + 8) ^ buffer);
|
||||
// Just an XOR mix at the end is quite weak here, but we prefer that for
|
||||
// latency over a more robust approach. Doing another mix with the size (the
|
||||
|
||||
@@ -158,9 +158,9 @@ struct HashBenchBase {
|
||||
uint64_t seed;
|
||||
|
||||
HashBenchBase() {
|
||||
// The real-world use case we care about is in a hash table where we'll mix
|
||||
// in some seed state, likely some ASLR address. To simulate this for
|
||||
// benchmarking, compute a seed from the address of a stack local variable.
|
||||
// The Abseil and LLVM hash tables mix in some seed state, likely some ASLR
|
||||
// address. To simulate this for benchmarking, compute a seed from the
|
||||
// address of a stack local variable.
|
||||
volatile char key;
|
||||
key = 42;
|
||||
// Rinse this through a volatile variable as well so returning it isn't
|
||||
@@ -171,10 +171,10 @@ struct HashBenchBase {
|
||||
}
|
||||
};
|
||||
|
||||
struct CarbonHashBench : HashBenchBase {
|
||||
struct CarbonHashBench {
|
||||
template <typename T>
|
||||
auto operator()(const T& value) -> uint64_t {
|
||||
return static_cast<uint64_t>(HashValue(value, seed));
|
||||
return static_cast<uint64_t>(HashValue(value));
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -14,8 +14,8 @@ namespace Carbon::InternalHashDispatch {
|
||||
|
||||
template <>
|
||||
struct CustomHashValue<llvm::APInt> {
|
||||
static auto Hash(llvm::APInt value, uint64_t seed) -> HashCode {
|
||||
Hasher hasher(seed);
|
||||
static auto Hash(llvm::APInt value) -> HashCode {
|
||||
Hasher hasher;
|
||||
if (LLVM_LIKELY(value.isSingleWord())) {
|
||||
hasher.Hash(value.getBitWidth(), value.getZExtValue());
|
||||
} else {
|
||||
@@ -29,12 +29,12 @@ struct CustomHashValue<llvm::APInt> {
|
||||
|
||||
template <>
|
||||
struct CustomHashValue<llvm::APFloat> {
|
||||
static auto Hash(llvm::APFloat value, uint64_t seed) -> HashCode {
|
||||
Hasher hasher(seed);
|
||||
static auto Hash(llvm::APFloat value) -> HashCode {
|
||||
Hasher hasher;
|
||||
// Hashing floating point numbers is complex and depends on the specific
|
||||
// internal semantics of `APFloat`, so delegate to the LLVM hashing
|
||||
// framework here. We re-hash the result to mix in our seed. All of this is
|
||||
// a bit inefficient, and we can revisit this to provide a dedicated
|
||||
// framework here. We re-hash the result to mix it into our hash. All of
|
||||
// this is a bit inefficient, and we can revisit this to provide a dedicated
|
||||
// implementation if it becomes a bottleneck.
|
||||
using llvm::hash_value;
|
||||
hasher.HashRaw(hash_value(value));
|
||||
|
||||
+5
-19
@@ -98,15 +98,6 @@ TEST(HashingTest, Integers) {
|
||||
}
|
||||
}
|
||||
|
||||
TEST(HashingTest, BasicSeeding) {
|
||||
auto unseeded_hash = HashValue(42);
|
||||
EXPECT_THAT(unseeded_hash, Ne(HashValue(42, 1)));
|
||||
EXPECT_THAT(unseeded_hash, Ne(HashValue(42, 2)));
|
||||
EXPECT_THAT(unseeded_hash, Ne(HashValue(42, 3)));
|
||||
EXPECT_THAT(unseeded_hash,
|
||||
Ne(HashValue(42, static_cast<uint64_t>(unseeded_hash))));
|
||||
}
|
||||
|
||||
TEST(HashingTest, Pointers) {
|
||||
int object1 = 42;
|
||||
std::string object2 =
|
||||
@@ -366,8 +357,8 @@ struct HashableType {
|
||||
// help ensure that the hashing framework doesn't accidentally override this.
|
||||
template <typename T>
|
||||
requires(std::same_as<T, HashableType>)
|
||||
friend auto CarbonHashValue(const T& value, uint64_t seed) -> HashCode {
|
||||
Hasher hasher(seed);
|
||||
friend auto CarbonHashValue(const T& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.Hash(value.x, value.y);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
@@ -468,11 +459,6 @@ TEST(HashingTest, TupleRecursion) {
|
||||
Eq(HashValue(std::tuple{a, c, a})));
|
||||
}
|
||||
|
||||
// The only significantly bad seed is zero, so pick a non-zero seed with a tiny
|
||||
// amount of entropy to make sure that none of the testing relies on the entropy
|
||||
// from this.
|
||||
constexpr uint64_t TestSeed = 42ULL * 1024;
|
||||
|
||||
auto ToHexBytes(llvm::StringRef s) -> std::string {
|
||||
RawStringOstream rendered;
|
||||
rendered << "{";
|
||||
@@ -637,7 +623,7 @@ auto AllByteStringsHashedAndSorted() {
|
||||
bytes[j] = (static_cast<uint64_t>(i) >> (8 * j)) & 0xff;
|
||||
}
|
||||
std::string s(std::begin(bytes), std::end(bytes));
|
||||
hashes.push_back({HashValue(s, TestSeed), s});
|
||||
hashes.push_back({HashValue(s), s});
|
||||
}
|
||||
|
||||
llvm::sort(hashes, [](const HashedString& lhs, const HashedString& rhs) {
|
||||
@@ -748,7 +734,7 @@ struct SparseHashTest : ::testing::Test {
|
||||
SetBitCount::Begin, std::min(bits, SetBitCount::End))) {
|
||||
if (set_bit_count == 0) {
|
||||
std::string s(byte_count, '\0');
|
||||
hashes.push_back({HashValue(s, TestSeed), std::move(s)});
|
||||
hashes.push_back({HashValue(s), std::move(s)});
|
||||
continue;
|
||||
}
|
||||
for (int begin_set_bit : llvm::seq_inclusive(0, bits - set_bit_count)) {
|
||||
@@ -793,7 +779,7 @@ struct SparseHashTest : ::testing::Test {
|
||||
if (has_end_byte_bits) {
|
||||
s[end_set_bit_byte_index] &= end_set_bit_byte;
|
||||
}
|
||||
hashes.push_back({HashValue(s, TestSeed), std::move(s)});
|
||||
hashes.push_back({HashValue(s), std::move(s)});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -79,10 +79,10 @@ auto HashtableEq(const LeftT& lhs, const RightT& rhs) -> bool;
|
||||
// llvm::ArrayRef<llvm::StringRef> interned_strings)
|
||||
// : interned_strings_(interned_strings) {}
|
||||
//
|
||||
// auto HashKey(llvm::StringRef s, uint64_t seed) const -> HashCode {
|
||||
// auto HashKey(llvm::StringRef s) const -> HashCode {
|
||||
// return HashValue(s);
|
||||
// }
|
||||
// auto HashKey(int index_key, uint64_t seed) const -> HashCode {
|
||||
// auto HashKey(int index_key) const -> HashCode {
|
||||
// return HashKey(interned_strings_[index_key]);
|
||||
// }
|
||||
//
|
||||
@@ -99,7 +99,7 @@ auto HashtableEq(const LeftT& lhs, const RightT& rhs) -> bool;
|
||||
// ```
|
||||
struct DefaultKeyContext {
|
||||
template <typename AnyKeyT>
|
||||
auto HashKey(const AnyKeyT& key, uint64_t seed) const -> HashCode;
|
||||
auto HashKey(const AnyKeyT& key) const -> HashCode;
|
||||
|
||||
template <typename AnyKeyT, typename TableKeyT>
|
||||
auto KeyEq(const AnyKeyT& lhs_key, const TableKeyT& rhs_key) const -> bool;
|
||||
@@ -138,7 +138,7 @@ class TranslatingKeyContext {
|
||||
}
|
||||
|
||||
template <typename AnyKeyT>
|
||||
auto HashKey(const AnyKeyT& key, uint64_t seed) const -> HashCode;
|
||||
auto HashKey(const AnyKeyT& key) const -> HashCode;
|
||||
|
||||
template <typename AnyKeyT, typename TableKeyT>
|
||||
auto KeyEq(const AnyKeyT& lhs_key, const TableKeyT& rhs_key) const -> bool;
|
||||
@@ -216,9 +216,8 @@ inline auto HashtableEq(const LeftT& lhs, const RightT& rhs) -> bool {
|
||||
}
|
||||
|
||||
template <typename AnyKeyT>
|
||||
auto DefaultKeyContext::HashKey(const AnyKeyT& key, uint64_t seed) const
|
||||
-> HashCode {
|
||||
return HashValue(key, seed);
|
||||
auto DefaultKeyContext::HashKey(const AnyKeyT& key) const -> HashCode {
|
||||
return HashValue(key);
|
||||
}
|
||||
|
||||
template <typename AnyKeyT, typename TableKeyT>
|
||||
@@ -229,13 +228,13 @@ auto DefaultKeyContext::KeyEq(const AnyKeyT& lhs_key,
|
||||
|
||||
template <typename DerivedT>
|
||||
template <typename AnyKeyT>
|
||||
auto TranslatingKeyContext<DerivedT>::HashKey(const AnyKeyT& key,
|
||||
uint64_t seed) const -> HashCode {
|
||||
auto TranslatingKeyContext<DerivedT>::HashKey(const AnyKeyT& key) const
|
||||
-> HashCode {
|
||||
const DerivedT& self = *static_cast<const DerivedT*>(this);
|
||||
if constexpr (requires { self.TranslateKey(key); }) {
|
||||
return HashValue(self.TranslateKey(key), seed);
|
||||
return HashValue(self.TranslateKey(key));
|
||||
} else {
|
||||
return HashValue(key, seed);
|
||||
return HashValue(key);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -121,9 +121,8 @@ TEST(HashtableKeyContextTest, HashtableEqAPFloat) {
|
||||
struct CustomHash {
|
||||
int x;
|
||||
|
||||
friend auto CarbonHashValue(const CustomHash& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
return HashValue(value.x + 42, seed);
|
||||
friend auto CarbonHashValue(const CustomHash& value) -> HashCode {
|
||||
return HashValue(value.x + 42);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -155,20 +154,15 @@ TEST(HashtableKeyContextTest, DefaultKeyContext) {
|
||||
EXPECT_FALSE(HashtableEq(zero_float, neg_zero_float));
|
||||
|
||||
// Also check hash dispatching.
|
||||
uint64_t seed = 1234;
|
||||
EXPECT_THAT(context.HashKey(42, seed), Eq(HashValue(42, seed)));
|
||||
EXPECT_THAT(context.HashKey(CustomHash{.x = 1234}, seed),
|
||||
Eq(HashValue(CustomHash{.x = 1234}, seed)));
|
||||
EXPECT_THAT(context.HashKey(one_64, seed), Eq(HashValue(one_64, seed)));
|
||||
EXPECT_THAT(context.HashKey(one_128, seed), Eq(HashValue(one_128, seed)));
|
||||
EXPECT_THAT(context.HashKey(one_64, seed),
|
||||
Ne(context.HashKey(one_128, seed)));
|
||||
EXPECT_THAT(context.HashKey(zero_float, seed),
|
||||
Eq(HashValue(zero_float, seed)));
|
||||
EXPECT_THAT(context.HashKey(neg_zero_float, seed),
|
||||
Eq(HashValue(neg_zero_float, seed)));
|
||||
EXPECT_THAT(context.HashKey(zero_float, seed),
|
||||
Ne(context.HashKey(neg_zero_float, seed)));
|
||||
EXPECT_THAT(context.HashKey(42), Eq(HashValue(42)));
|
||||
EXPECT_THAT(context.HashKey(CustomHash{.x = 1234}),
|
||||
Eq(HashValue(CustomHash{.x = 1234})));
|
||||
EXPECT_THAT(context.HashKey(one_64), Eq(HashValue(one_64)));
|
||||
EXPECT_THAT(context.HashKey(one_128), Eq(HashValue(one_128)));
|
||||
EXPECT_THAT(context.HashKey(one_64), Ne(context.HashKey(one_128)));
|
||||
EXPECT_THAT(context.HashKey(zero_float), Eq(HashValue(zero_float)));
|
||||
EXPECT_THAT(context.HashKey(neg_zero_float), Eq(HashValue(neg_zero_float)));
|
||||
EXPECT_THAT(context.HashKey(zero_float), Ne(context.HashKey(neg_zero_float)));
|
||||
}
|
||||
|
||||
struct TestTranslatingKeyContext
|
||||
@@ -191,13 +185,12 @@ TEST(HashtableKeyContextTest, TranslatingKeyContext) {
|
||||
|
||||
TestTranslatingKeyContext context = {.array = values};
|
||||
|
||||
uint64_t seed = 1234;
|
||||
EXPECT_THAT(context.HashKey(0, seed), Eq(HashValue(one_64, seed)));
|
||||
EXPECT_THAT(context.HashKey(1, seed), Eq(HashValue(two_64, seed)));
|
||||
EXPECT_THAT(context.HashKey(2, seed), Eq(HashValue(one_128, seed)));
|
||||
EXPECT_THAT(context.HashKey(3, seed), Eq(HashValue(two_128, seed)));
|
||||
EXPECT_THAT(context.HashKey(4, seed), Eq(HashValue(one_64, seed)));
|
||||
EXPECT_THAT(context.HashKey(5, seed), Eq(HashValue(one_64, seed)));
|
||||
EXPECT_THAT(context.HashKey(0), Eq(HashValue(one_64)));
|
||||
EXPECT_THAT(context.HashKey(1), Eq(HashValue(two_64)));
|
||||
EXPECT_THAT(context.HashKey(2), Eq(HashValue(one_128)));
|
||||
EXPECT_THAT(context.HashKey(3), Eq(HashValue(two_128)));
|
||||
EXPECT_THAT(context.HashKey(4), Eq(HashValue(one_64)));
|
||||
EXPECT_THAT(context.HashKey(5), Eq(HashValue(one_64)));
|
||||
|
||||
EXPECT_TRUE(context.KeyEq(one_64, 0));
|
||||
EXPECT_TRUE(context.KeyEq(one_64, 4));
|
||||
|
||||
@@ -8,9 +8,12 @@
|
||||
|
||||
namespace Carbon::RawHashtable {
|
||||
|
||||
#ifndef NDEBUG
|
||||
// A global variable whose address seeds the iteration entropy pool. This allows
|
||||
// ASLR to introduce some variation in debug iteration order when enabled via
|
||||
// the code model for globals.
|
||||
volatile std::byte global_addr_seed{1};
|
||||
|
||||
#ifndef NDEBUG
|
||||
std::atomic<HashCode> entropy_hash =
|
||||
Carbon::HashValue(reinterpret_cast<uint64_t>(&global_addr_seed));
|
||||
#endif
|
||||
|
||||
+9
-21
@@ -78,6 +78,10 @@
|
||||
// - The storage for an entry is an internal type that should not be exposed to
|
||||
// users, and instead only the underlying keys and values.
|
||||
//
|
||||
// - Keys are hashed with the unseeded `HashValue`; there is no per-table seed.
|
||||
// This makes a table's layout, and any collisions in it, reproducible
|
||||
// between runs. Debug builds vary the iteration order in `entries()` instead.
|
||||
//
|
||||
// - The hash addressing and probing occurs over *groups* of slots rather than
|
||||
// individual entries. When inserting a new entry, it can be added to the
|
||||
// group it hashes to as long it is not full, and can even replace a slot with
|
||||
@@ -868,19 +872,6 @@ class TableImpl : public InputBaseT {
|
||||
//
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
// Computes a seed that provides a small amount of entropy from ASLR where
|
||||
// available with minimal cost. The priority is speed, and this computes the
|
||||
// entropy in a way that doesn't require loading from memory, merely accessing
|
||||
// entropy already available without accessing memory.
|
||||
inline auto ComputeSeed() -> uint64_t {
|
||||
// A global variable whose address is used as a seed. This allows ASLR to
|
||||
// introduce some variation in hashtable ordering when enabled via the code
|
||||
// model for globals.
|
||||
extern volatile std::byte global_addr_seed;
|
||||
|
||||
return reinterpret_cast<uint64_t>(&global_addr_seed);
|
||||
}
|
||||
|
||||
#ifndef NDEBUG
|
||||
// A pool of entropy used to vary the iteration order of hashtables in debug
|
||||
// builds. It is seeded from ASLR where available.
|
||||
@@ -980,7 +971,7 @@ auto ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::LookupEntry(
|
||||
CARBON_DCHECK(local_size > 0);
|
||||
|
||||
uint8_t* local_metadata = metadata();
|
||||
HashCode hash = key_context.HashKey(lookup_key, ComputeSeed());
|
||||
HashCode hash = key_context.HashKey(lookup_key);
|
||||
auto [hash_index, tag] = hash.ExtractIndexAndTag<7>();
|
||||
|
||||
EntryT* local_entries = entries_data();
|
||||
@@ -1057,8 +1048,7 @@ auto ViewImpl<InputKeyT, InputValueT, InputKeyContextT>::ComputeMetricsImpl(
|
||||
for (ssize_t byte_index : present_matched_range) {
|
||||
++metrics.key_count;
|
||||
ssize_t index = group_index + byte_index;
|
||||
HashCode hash =
|
||||
key_context.HashKey(local_entries[index].key(), ComputeSeed());
|
||||
HashCode hash = key_context.HashKey(local_entries[index].key());
|
||||
auto [hash_index, tag] = hash.ExtractIndexAndTag<7>();
|
||||
ProbeSequence s(hash_index, local_size);
|
||||
metrics.probed_key_count +=
|
||||
@@ -1257,7 +1247,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::InsertImpl(
|
||||
|
||||
uint8_t* local_metadata = metadata();
|
||||
|
||||
HashCode hash = key_context.HashKey(lookup_key, ComputeSeed());
|
||||
HashCode hash = key_context.HashKey(lookup_key);
|
||||
auto [hash_index, tag] = hash.ExtractIndexAndTag<7>();
|
||||
|
||||
// We re-purpose the empty control byte to signal no insert is needed to the
|
||||
@@ -1383,8 +1373,7 @@ BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::GrowToAllocSizeImpl(
|
||||
for (ssize_t byte_index : present_matched_range) {
|
||||
++count;
|
||||
ssize_t index = group_index + byte_index;
|
||||
HashCode hash =
|
||||
key_context.HashKey(old_entries[index].key(), ComputeSeed());
|
||||
HashCode hash = key_context.HashKey(old_entries[index].key());
|
||||
EntryT* new_entry = InsertIntoEmpty(hash);
|
||||
new_entry->MoveFrom(std::move(old_entries[index]));
|
||||
}
|
||||
@@ -1799,8 +1788,7 @@ auto BaseImpl<InputKeyT, InputValueT, InputKeyContextT>::GrowToNextAllocSize(
|
||||
CARBON_DCHECK(new_metadata[old_index | old_size] ==
|
||||
old_metadata[old_index]);
|
||||
}
|
||||
HashCode hash =
|
||||
key_context.HashKey(old_entries[old_index].key(), ComputeSeed());
|
||||
HashCode hash = key_context.HashKey(old_entries[old_index].key());
|
||||
ssize_t old_hash_index = hash.ExtractIndexAndTag<7>().first &
|
||||
ComputeProbeMaskFromSize(old_size);
|
||||
if (LLVM_UNLIKELY(old_hash_index != group_index)) {
|
||||
|
||||
@@ -372,9 +372,8 @@ auto DumpHashStatistics(llvm::ArrayRef<T> keys) -> void {
|
||||
constexpr int GroupShift = llvm::ConstantLog2<GroupSize>();
|
||||
|
||||
size_t mask = ComputeProbeMaskFromSize(expected_size);
|
||||
uint64_t salt = ComputeSeed();
|
||||
auto get_hash_index = [mask, salt](auto x) -> ssize_t {
|
||||
auto [hash_index, _] = HashValue(x, salt).template ExtractIndexAndTag<7>();
|
||||
auto get_hash_index = [mask](auto x) -> ssize_t {
|
||||
auto [hash_index, _] = HashValue(x).template ExtractIndexAndTag<7>();
|
||||
return (hash_index & mask) >> GroupShift;
|
||||
};
|
||||
|
||||
@@ -408,9 +407,8 @@ auto DumpHashStatistics(llvm::ArrayRef<T> keys) -> void {
|
||||
for (auto i : llvm::ArrayRef(grouped_key_indices[max_group_index])
|
||||
.take_front(2 * GroupSize)) {
|
||||
auto k = keys[i];
|
||||
auto hash = static_cast<uint64_t>(HashValue(k, salt));
|
||||
auto hash = static_cast<uint64_t>(HashValue(k));
|
||||
llvm::errs() << " key: " << k
|
||||
<< " salt: " << llvm::formatv("{0:x16}", salt)
|
||||
<< " hash: " << llvm::formatv("{0:x16}", hash) << "\n";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -79,9 +79,8 @@ struct LowZeroBitInt {
|
||||
constexpr auto operator<=>(const LowZeroBitInt& rhs) const
|
||||
-> std::strong_ordering = default;
|
||||
|
||||
friend auto CarbonHashValue(const LowZeroBitInt& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
return HashValue(value.shifted_value, seed);
|
||||
friend auto CarbonHashValue(const LowZeroBitInt& value) -> HashCode {
|
||||
return HashValue(value.shifted_value);
|
||||
}
|
||||
|
||||
template <typename H>
|
||||
|
||||
@@ -38,8 +38,8 @@ struct TestData : Printable<TestData> {
|
||||
return lhs.value <=> rhs.value;
|
||||
}
|
||||
|
||||
friend auto CarbonHashValue(TestData data, uint64_t seed) -> HashCode {
|
||||
return Carbon::HashValue(data.value, seed);
|
||||
friend auto CarbonHashValue(TestData data) -> HashCode {
|
||||
return Carbon::HashValue(data.value);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -74,9 +74,8 @@ struct MoveOnlyTestData : Printable<TestData> {
|
||||
const MoveOnlyTestData& rhs)
|
||||
-> std::strong_ordering = default;
|
||||
|
||||
friend auto CarbonHashValue(const MoveOnlyTestData& data, uint64_t seed)
|
||||
-> HashCode {
|
||||
return Carbon::HashValue(data.value, seed);
|
||||
friend auto CarbonHashValue(const MoveOnlyTestData& data) -> HashCode {
|
||||
return Carbon::HashValue(data.value);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -92,8 +91,8 @@ inline auto CarbonHashtableEq(int lhs, const MoveOnlyTestData& rhs) -> bool {
|
||||
// fails to be used.
|
||||
struct TestKeyContext : DefaultKeyContext {
|
||||
template <typename KeyT>
|
||||
auto HashKey(const KeyT& key, uint64_t seed) const -> HashCode {
|
||||
Hasher hash(seed);
|
||||
auto HashKey(const KeyT& key) const -> HashCode {
|
||||
Hasher hash;
|
||||
// Inject some other data to the hash.
|
||||
hash.HashRaw(42);
|
||||
hash.HashRaw(HashValue(key));
|
||||
@@ -108,8 +107,8 @@ struct TestKeyContext : DefaultKeyContext {
|
||||
template <int TagBits, bool FixIndexBits, bool FixTagBits, uint64_t FixedVal>
|
||||
struct FixedHashKeyContext : DefaultKeyContext {
|
||||
template <typename KeyT>
|
||||
auto HashKey(const KeyT& key, uint64_t seed) const -> HashCode {
|
||||
HashCode original_hash = HashValue(key, seed);
|
||||
auto HashKey(const KeyT& key) const -> HashCode {
|
||||
HashCode original_hash = HashValue(key);
|
||||
auto raw_hash = static_cast<uint64_t>(original_hash);
|
||||
|
||||
constexpr uint64_t TagMask = (1U << TagBits) - 1;
|
||||
|
||||
@@ -685,8 +685,8 @@ struct StateKey {
|
||||
friend auto operator==(const StateKey& lhs, const StateKey& rhs)
|
||||
-> bool = default;
|
||||
|
||||
friend auto CarbonHashValue(const StateKey& key, uint64_t seed) -> HashCode {
|
||||
Hasher hasher(seed);
|
||||
friend auto CarbonHashValue(const StateKey& key) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.Hash(key.stack, key.closer_inserted);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
@@ -93,16 +93,12 @@ struct ClangDeclSignature : public Printable<ClangDeclSignature> {
|
||||
auto operator==(const ClangDeclSignature& rhs) const -> bool = default;
|
||||
|
||||
// Hashing for ClangDeclSignature.
|
||||
friend auto CarbonHashValue(const ClangDeclSignature& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
HashCode code =
|
||||
HashValue(std::tuple{value.kind, value.num_params,
|
||||
static_cast<int8_t>(value.self_passing_mode)},
|
||||
seed);
|
||||
for (auto mode : value.passing_modes) {
|
||||
code = HashValue(static_cast<int8_t>(mode), static_cast<uint64_t>(code));
|
||||
}
|
||||
return code;
|
||||
friend auto CarbonHashValue(const ClangDeclSignature& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.Hash(value.kind, value.num_params,
|
||||
static_cast<int8_t>(value.self_passing_mode),
|
||||
llvm::ArrayRef(value.passing_modes));
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -139,9 +135,8 @@ struct ClangDeclKey : public Printable<ClangDeclKey> {
|
||||
auto operator==(const ClangDeclKey& rhs) const -> bool = default;
|
||||
|
||||
// Hashing for ClangDecl. See common/hashing.h.
|
||||
friend auto CarbonHashValue(const ClangDeclKey& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
return HashValue(std::tuple{value.decl, value.signature_id}, seed);
|
||||
friend auto CarbonHashValue(const ClangDeclKey& value) -> HashCode {
|
||||
return HashValue(std::tuple{value.decl, value.signature_id});
|
||||
}
|
||||
|
||||
// The Clang declaration pointing to the Clang AST.
|
||||
|
||||
@@ -137,9 +137,8 @@ using DeclaredFacetTypeStore =
|
||||
CanonicalValueStore<DeclaredFacetTypeId, DeclaredFacetType, Tag<CheckIRId>>;
|
||||
|
||||
// See common/hashing.h.
|
||||
inline auto CarbonHashValue(const DeclaredFacetType& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(const DeclaredFacetType& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.HashArray(llvm::ArrayRef(value.extend_constraints));
|
||||
hasher.HashArray(llvm::ArrayRef(value.self_impls_constraints));
|
||||
hasher.HashArray(llvm::ArrayRef(value.extend_named_constraints));
|
||||
|
||||
@@ -44,9 +44,8 @@ struct EntityName : public Printable<EntityName> {
|
||||
}
|
||||
|
||||
// Hashing for EntityName. See common/hashing.h.
|
||||
friend auto CarbonHashValue(const EntityName& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
friend auto CarbonHashValue(const EntityName& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.HashRaw(value.IdentityKey());
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
@@ -705,8 +705,8 @@ class InstBlockStore
|
||||
};
|
||||
|
||||
// See common/hashing.h.
|
||||
inline auto CarbonHashValue(const Inst& value, uint64_t seed) -> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(const Inst& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.HashRaw(value);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
@@ -28,9 +28,8 @@ using StructTypeFieldsStore =
|
||||
BlockValueStore<StructTypeFieldsId, StructTypeField, Tag<CheckIRId>>;
|
||||
|
||||
// See common/hashing.h. Supports canonicalization of fields.
|
||||
inline auto CarbonHashValue(const StructTypeField& value, uint64_t seed)
|
||||
-> HashCode {
|
||||
Hasher hasher(seed);
|
||||
inline auto CarbonHashValue(const StructTypeField& value) -> HashCode {
|
||||
Hasher hasher;
|
||||
hasher.HashRaw(value);
|
||||
return static_cast<HashCode>(hasher);
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user