Vanilla Java

Overview Yahoo Cloud Service Benchmark is a reasonably widely used benchmarking tool for testing key value stores for a significant number of key e.g 100 million, and a modest number of clients i.e. served from one machine. In this article I look at how a test of 100 million * 1 KB key/values performed using Chronicle Map on a single machine with 128 GB memory, dual Intel E5-2650 v2 @ 2.60GHz, and six Samsung 840 EVO SSDs. The 1 KB value consists of ten fields of 100 byte Strings.  For a more optimal solution, primitive numbers would be a better choice. While the SSDs helped, the peak transfer rate was 700 MB/s which could be supported by two SATA SSD drives. These benchmarks were performed using the latest version at the time of the report, Chronicle Map 2.0.6a-SNAPSHOT. Micro-second world. Something which confounds me when reading benchmarks about key-value stores is that they start with the premise that performance is really important.  IMHO, about 90% o...

Overview Essentialism isn't just flawed, it is flawed at many levels, some of which should be familiar to technologists.  I have read a number of arguments against essentialism, however for me they miss a number of points which relate to technology, an area I claim to know more about.  In technology you can measurably demonstrate the performance difference between two solutions, and yet it is difficult to convince people to let go of their biases. What is Essentialism (from Google) "a belief that things have a set of characteristics that make them what they are, and that the task of science and philosophy is their discovery and expression; the doctrine that essence is prior to existence. the view that all children should be taught on traditional lines the ideas and methods regarded as essential to the prevalent culture. the view that categories of people, such as women and men, or heterosexuals and homosexuals, or members of ethnic groups, have intrinsically different...

A really hard interview question would be something like this int i = Integer.MAX_VALUE; i += 0.0f; int j = i; System.out.println(j == Integer.MAX_VALUE); // true Why does this print true? At first glace, the answer seem obvious, until you realise that if you change  int i for long i things get weird long i = Integer.MAX_VALUE; i += 0.0f; int j = (int) i; System.out.println(j == Integer.MAX_VALUE); // false System.out.println(j == Integer.MIN_VALUE); // true What is going on you might wonder? When did Java become JavaScript? Let me start by explaining why long gives a such a strange result. An important detail about += is that it does an implicit cast.  You might think that a += b; is the same as a = a + b; and basically it is except with a subtle difference which most of the time doesn't matter; a = (typeOf(a)) (a + b); Another subtle feature of addition is the the result is the "wider" of the two types.  ...

Overview I was recently asked to compare the performance of Kafka with Chronicle Queue.  No two products are exactly alike, and performing a fair comparison is not easy.  We can try to run similar tests and see what results we get. This test is based on Apache Kafka Performance Results What was the test used? One area Kafka tests is multi-threaded performance.  In tests we have done, it is neither better or worse to use more more threads (up to the number CPUs you have).  We didn't benchmark this here. All tests use one producer. Another difference, is that we flush to disk periodically by time rather than by count.  Being able to say you are never behind by more than X milli-seconds is often more useful than say 600 messages, as you don't know how long those messages could have been waiting there.  For our tests, we look at flush periods of between 1 ms and 10 ms.  In Kafka's tests, they appears to be every 3 ms approximately. The mess...

Background Between my partner and I, we have six daughters, and as they have grown I have thought more interested in their long term future, the role of women in society, the way technology will change our lives and in particular the role of women in technology. On the last topic, all the articles I have read have been written my women.  In this post, I hope to outline my experiences in this regard. Women face many challenges, most typical of male dominated professions as well as some specific to the technology field.  I believe that there will be a time when IT, like teaching and accounting, will have more women than men. I can understand it is frustrating as a woman in technology to be treated as a novelty.  Perhaps my experience can illustrate why that might be. One thing I have learnt over the years is that while I am happy to talk technology all day, there is one time to stop, in social situations when women are present.  This doesn't come from men, bu...

Overview Java 8 has added features such as lambdas and type inference. This makes the language less verbose and cleaner, however it comes with more side effects as you don't have to be as explicit in what you are doing. The return type of a lambda matters Java 8 infers the type of a closure.  One way it does this is to look at the return type (or whether anything is returned)  This can have a surprising side effect.  Consider this code. ExecutorService es = Executors.newSingleThreadExecutor(); es.submit(() -> {     try(Scanner scanner = new Scanner(new FileReader("file.txt"))) {         String line = scanner.nextLine();         process(line);     }     return null; }); This code compiles fine. However, the line return null;  appears redundant and you might be tempted to remove it.  However if you remove the line, you get an error. Error:(12, 39) java: un...

Overview Vendors typically publish numbers they are happy with, and avoid telling you about a product's weaknesses.  However, behind the numbers is a dirty secret if you know where to look. Why don't we use GPUs for everything? Finding problems which naturally scale to thousands of data points/tasks is easy for some problems, and very hard for others. GPUs are designed for computing large vector operations.  However, what is "large" and why does it matter? Say you have a GPU with 1024 cores. This means it can process a vector of length 1024 all at once and 2048 double the time.  But what happen if we only have a vector of 100, or 10 or 1. The inconvenient answer is it take the same amount of time because you can’t make use of all of your cores at once.  You get only 10%, 1% or just 0.1% of the peak performance.  If you want to get best efficiency, you want a problem which has many thousands of values which can be processed concurrently.  If...