Comparing Hashing strategies

Overview

Chronicle has a number of implementations for hashing, including City and Murmur.  It also has it's own Vanilla Hash, but how was this tested?

What is Vanilla Hash?

Vanilla Hash is designed to be as simple as possible and be optimised for the Orthogonal Bits test (See below)  This was compared with City 1.1 and Murmur 3 hashing strategies.

This is the 99%tile latencies for filling the 64 byte/256 byte buffer with new data and generating a 64-bit hash. JMH was used to perform the measurements. See Main64bytes and Main256bytes

Hashing Strategy	64 byte 99%tile	256 byte 99%tile
Vanilla	67 ns	112 ns
City 1.1	90 ns	182 ns
Murmur 3	104 ns	211 ns

The full test of results are here.

What tests can you do to check a hashing strategy is good?

There is a number of simple tests you can do. The tests cannot identify a good hash, but they can show a hash to be a poor one. Passing one test could mean it will fail another.

In each case multiple tests are run with different random starting points.  A score is taken for the 99%th percentile, i.e. the worst 1%.  This is because you don't need a hash which works some of the time, or on average.  You need one which works most of the time. (In all cases you can invent a pathological case where any specific hash will break down)

For consistency, lower scores are better. The test should be constructed in such as that a score of 0 indicates the test is broken.

In each test, using an input of 8,192 bits, or 1024 KB, one bit at a time is toggled.  From these inputs, 8,192 x 64-bit hashes are generated.

For the random tests however, a sequence of random 64-bit values were taken. These are useful to get an idea of what a good number is for the hashing strategies tested.

Mask of Hash Score

In this test, each hash is modulus by 16,384 (double the number of hashes) and the number of collisions is reported.  Most hashing strategies did well for this test.

Avalanche Score

In this test, each hash is compared to the previous hash (with the previous bit toggled) to see how likely any given bit will be flipped.  The ideal is 50% and the sum of difference to 50% is taken with the worst 1% reported.

Speed in latency

In this test, the time it takes to perform the hash is recorded and the worst 1% latency reported.

Orthogonal Bits

The purpose of this tests is to ensure all the hashes have bits which are different as different as possible to every other hash produced.  Think of the 8 Queens Problem, except for 64-bit numbers.  The ideal is that every number has the same number of bits different to every other number and this is as high as possible.

In this test, every hash is compared to every other hash. A count of the number of bits which are different is taken.  If the number of different bits is less than 18, this is given a penalty score of 2^(17-n).  The less bits which are different the greater the penalty on an exponential scale. If any of the 8K hashes comapred to the other 8K hashes are different in less than 5 bits, this is a failure even if all the other pairs are fine.

I have called it an Orthogonal Bits test as you can model a 64-bit number as a 64 dimensional vector of bits.  Ideally you want the angle between all the hashes produced as high as possible.

Of all the tests, this one shows the highest difference between the String.hashCode() with HashMap.hash(int) and the other hashing strategies.

Testing String.hashCode()

String.hashCode() is a very poor hash, especially for the lower bits. It is standard and cannot be changed or break backward compatibility.  However, this doesn't have to be a problem as HashMap uses an agitate function which brings down some of the higher bits to randomise the lower ones.

int hash(int h) {
    // This function ensures that hashCodes that differ only by
    // constant multiples at each bit position have a bounded
    // number of collisions (approximately 8 at default load factor).
    h ^= (h >>> 20) ^ (h >>> 12);
    return h ^ (h >>> 7) ^ (h >>> 4);
}

Results

The CheckMain class run a suit of tests on each hashing strategy.

VANILLA

Orthogonal bits: 99%tile score: 6066

Speed: The 99%tile for latency was 0.223 us

Avalanche: The 99%tile of the drift from 50% was 0.55%

Mask of Hash: 99%tile collisions: 1815

CITY_1_1

Orthogonal bits: 99%tile score: 7395

Speed: The 99%tile for latency was 0.267 us

Avalanche: The 99%tile of the drift from 50% was 0.55%

Mask of Hash: 99%tile collisions: 1817

MURMUR_3

Orthogonal bits: 99%tile score: 7524

Speed: The 99%tile for latency was 0.378 us

Avalanche: The 99%tile of the drift from 50% was 0.54%

Mask of Hash: 99%tile collisions: 1815

STRING32

Orthogonal bits: 99%tile score: 295906433

Speed: The 99%tile for latency was 1.580 us

Avalanche: The 99%tile of the drift from 50% was 1.02%

Mask of Hash: 99%tile collisions: 1814

STRING64

Orthogonal bits: 99%tile score: 1939167

Speed: The 99%tile for latency was 1.520 us

Avalanche: The 99%tile of the drift from 50% was 0.61%

Mask of Hash: 99%tile collisions: 1816

STRING32_WITHOUT_AGITATE

Orthogonal bits: 99%tile score: 879390386

Speed: The 99%tile for latency was 1.573 us

Avalanche: The 99%tile of the drift from 50% was 3.53%

Mask of Hash: 99%tile collisions: 6593

RANDOM

Orthogonal bits: 99%tile score: 7444

Speed: The 99%tile for latency was 0.058 us

Avalanche: The 99%tile of the drift from 50% was 0.53%

Mask of Hash: 99%tile collisions: 1817

SECURE_RANDOM

Orthogonal bits: 99%tile score: 7449

Speed: The 99%tile for latency was 0.861 us

Avalanche: The 99%tile of the drift from 50% was 0.54%

Mask of Hash: 99%tile collisions: 1816

SEEDED_VANILLA
Orthogonal bits: 99%tile score: 6000
Speed: The 99%tile for latency was 0.219 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1814

SEEDED_CITY_1_1
Orthogonal bits: 99%tile score: 7313
Speed: The 99%tile for latency was 0.270 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1813

SEEDED_MURMUR_3
Orthogonal bits: 99%tile score: 7404
Speed: The 99%tile for latency was 0.359 us
Avalanche: The 99%tile of the drift from 50% was 0.53%
Mask of Hash: 99%tile collisions: 1810

Note: Seeded Vanilla Hash is part of Chronicle Enterprise

Conclusions

The Vanilla, City and Murmur hashers were the fastest.  

While String.hashCode() is simple, the multiplication operation on a per character basis is expensive.  By comparison all the others process 8 bytes at a time using longs.  See STRINGS32_WITHOUT_AGITATE compared with STRING32.  HashMap uses the later.

The 32-bit String hashCode() even with the agitate performed poorly on the Avalanche test. In SMHasher where this test comes from a score over 1% was considered a failure.

The Mask of Hash tests, while simple appears to be performed well in all cases. The exception being the String.hashCode() which as mentioned doesn't have very random low bits.

What I found interesting is how different the orthogonal test score were.  The first three hash stratgies were again consistently low. Even the 64-bit version of String.hashCode() has a high change of producing hashes with less than 18 bits different, in fact a lot of the bits are the same.

Disclaimer

Vanilla Hash was optimised for the Orthogonal Bits test.  As such it is no surprise that it gets a slightly better result.  This doesn't mean that Vanilla Hash is better than City or Murmur. It may just mean it is best for the Orthogonal Bits test.