Comparing Hashing strategies
Overview
Chronicle has a number of implementations for hashing, including City and Murmur. It also has it's own Vanilla Hash, but how was this tested?What is Vanilla Hash?
Vanilla Hash is designed to be as simple as possible and be optimised for the Orthogonal Bits test (See below) This was compared with City 1.1 and Murmur 3 hashing strategies.This is the 99%tile latencies for filling the 64 byte/256 byte buffer with new data and generating a 64bit hash. JMH was used to perform the measurements. See Main64bytes and Main256bytes
Hashing
Strategy

64
byte 99%tile

256
byte 99%tile


Vanilla 
67 ns

112 ns

City 1.1 
90 ns

182 ns

Murmur 3 
104 ns

211 ns

The full test of results are here.
What tests can you do to check a hashing strategy is good?
There is a number of simple tests you can do. The tests cannot identify a good hash, but they can show a hash to be a poor one. Passing one test could mean it will fail another.
In each case multiple tests are run with different random starting points. A score is taken for the 99%th percentile, i.e. the worst 1%. This is because you don't need a hash which works some of the time, or on average. You need one which works most of the time. (In all cases you can invent a pathological case where any specific hash will break down)
For consistency, lower scores are better. The test should be constructed in such as that a score of 0 indicates the test is broken.
In each test, using an input of 8,192 bits, or 1024 KB, one bit at a time is toggled. From these inputs, 8,192 x 64bit hashes are generated.
For the random tests however, a sequence of random 64bit values were taken. These are useful to get an idea of what a good number is for the hashing strategies tested.
Mask of Hash Score
In this test, each hash is modulus by 16,384 (double the number of hashes) and the number of collisions is reported. Most hashing strategies did well for this test.
Avalanche Score
In this test, each hash is compared to the previous hash (with the previous bit toggled) to see how likely any given bit will be flipped. The ideal is 50% and the sum of difference to 50% is taken with the worst 1% reported.
Speed in latency
In this test, the time it takes to perform the hash is recorded and the worst 1% latency reported.
Orthogonal Bits
The purpose of this tests is to ensure all the hashes have bits which are different as different as possible to every other hash produced. Think of the 8 Queens Problem, except for 64bit numbers. The ideal is that every number has the same number of bits different to every other number and this is as high as possible.
In this test, every hash is compared to every other hash. A count of the number of bits which are different is taken. If the number of different bits is less than 18, this is given a penalty score of 2^(17n). The less bits which are different the greater the penalty on an exponential scale. If any of the 8K hashes comapred to the other 8K hashes are different in less than 5 bits, this is a failure even if all the other pairs are fine.
I have called it an Orthogonal Bits test as you can model a 64bit number as a 64 dimensional vector of bits. Ideally you want the angle between all the hashes produced as high as possible.
Of all the tests, this one shows the highest difference between the String.hashCode() with HashMap.hash(int) and the other hashing strategies.
Testing String.hashCode()
String.hashCode() is a very poor hash, especially for the lower bits. It is standard and cannot be changed or break backward compatibility. However, this doesn't have to be a problem as HashMap uses an agitate function which brings down some of the higher bits to randomise the lower ones.
int hash(int h) { // This function ensures that hashCodes that differ only by // constant multiples at each bit position have a bounded // number of collisions (approximately 8 at default load factor). h ^= (h >>> 20) ^ (h >>> 12); return h ^ (h >>> 7) ^ (h >>> 4); }
Results
The CheckMain class run a suit of tests on each hashing strategy.
VANILLA
Orthogonal bits: 99%tile score: 6066
Speed: The 99%tile for latency was 0.223 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1815
CITY_1_1
Orthogonal bits: 99%tile score: 7395
Speed: The 99%tile for latency was 0.267 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1817
MURMUR_3
Orthogonal bits: 99%tile score: 7524
Speed: The 99%tile for latency was 0.378 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1815
STRING32
Orthogonal bits: 99%tile score: 295906433
Speed: The 99%tile for latency was 1.580 us
Avalanche: The 99%tile of the drift from 50% was 1.02%
Mask of Hash: 99%tile collisions: 1814
STRING64
Orthogonal bits: 99%tile score: 1939167
Speed: The 99%tile for latency was 1.520 us
Avalanche: The 99%tile of the drift from 50% was 0.61%
Mask of Hash: 99%tile collisions: 1816
STRING32_WITHOUT_AGITATE
Orthogonal bits: 99%tile score: 879390386
Speed: The 99%tile for latency was 1.573 us
Avalanche: The 99%tile of the drift from 50% was 3.53%
Mask of Hash: 99%tile collisions: 6593
RANDOM
Orthogonal bits: 99%tile score: 7444
Speed: The 99%tile for latency was 0.058 us
Avalanche: The 99%tile of the drift from 50% was 0.53%
Mask of Hash: 99%tile collisions: 1817
SECURE_RANDOM
Orthogonal bits: 99%tile score: 7449
Speed: The 99%tile for latency was 0.861 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1816
SEEDED_VANILLA
Orthogonal bits: 99%tile score: 6000
Speed: The 99%tile for latency was 0.219 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1814
SEEDED_CITY_1_1
Orthogonal bits: 99%tile score: 7313
Speed: The 99%tile for latency was 0.270 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1813
SEEDED_MURMUR_3
Orthogonal bits: 99%tile score: 7404
Speed: The 99%tile for latency was 0.359 us
Avalanche: The 99%tile of the drift from 50% was 0.53%
Mask of Hash: 99%tile collisions: 1810
SEEDED_VANILLA
Orthogonal bits: 99%tile score: 6000
Speed: The 99%tile for latency was 0.219 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1814
SEEDED_CITY_1_1
Orthogonal bits: 99%tile score: 7313
Speed: The 99%tile for latency was 0.270 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1813
SEEDED_MURMUR_3
Orthogonal bits: 99%tile score: 7404
Speed: The 99%tile for latency was 0.359 us
Avalanche: The 99%tile of the drift from 50% was 0.53%
Mask of Hash: 99%tile collisions: 1810
Conclusions
The Vanilla, City and Murmur hashers were the fastest.
While String.hashCode() is simple, the multiplication operation on a per character basis is expensive. By comparison all the others process 8 bytes at a time using longs. See STRINGS32_WITHOUT_AGITATE compared with STRING32. HashMap uses the later.
The 32bit String hashCode() even with the agitate performed poorly on the Avalanche test. In SMHasher where this test comes from a score over 1% was considered a failure.
The Mask of Hash tests, while simple appears to be performed well in all cases. The exception being the String.hashCode() which as mentioned doesn't have very random low bits.
What I found interesting is how different the orthogonal test score were. The first three hash stratgies were again consistently low. Even the 64bit version of String.hashCode() has a high change of producing hashes with less than 18 bits different, in fact a lot of the bits are the same.
Disclaimer
Vanilla Hash was optimised for the Orthogonal Bits test. As such it is no surprise that it gets a slightly better result. This doesn't mean that Vanilla Hash is better than City or Murmur. It may just mean it is best for the Orthogonal Bits test.
Comments
Post a Comment