Lies, statistics and vendors
Overview Reading performance results supplied by vendors is a skill in itself. It can be difficult to compare numbers from different vendors on a fair basis, and even more difficult to estimate how a product will behave in your system. Lies and statistics One of the few quotes from University I remember goes roughly like this Peak Performance - A manufacture's guarantee not to exceed a given rating -- Computer Architecture, A Quantitative Approach. (1st edition) At first this appears rather cynical, but over the years I have come to the conclusion this is unavoidable and once you accept this you can trust the numbers you get in if you see them a new light. Why is it so hard to give a trustworthy performance number? There are many challenges in giving good performance numbers. Most vendors try harder to give trustworthy numbers but it is not as easy as it looks. Latencies and throughputs don't follow a normal distributi...