Kafka Benchmark on Chronicle Queue

Overview

I was recently asked to compare the performance of Kafka with Chronicle Queue.  No two products are exactly alike, and performing a fair comparison is not easy.  We can try to run similar tests and see what results we get.

This test is based on Apache Kafka Performance Results

What was the test used?

One area Kafka tests is multi-threaded performance.  In tests we have done, it is neither better or worse to use more more threads (up to the number CPUs you have).  We didn't benchmark this here. All tests use one producer.

Another difference, is that we flush to disk periodically by time rather than by count.  Being able to say you are never behind by more than X milli-seconds is often more useful than say 600 messages, as you don't know how long those messages could have been waiting there.  For our tests, we look at flush periods of between 1 ms and 10 ms.  In Kafka's tests, they appears to be every 3 ms approximately.

The message size used was 200 bytes in each case, and we explored the difference writing batches of 1, 2, 5 and 10 messages at once made.  We also tried 200 messages in a batch and the performance was similar to batches of 10.

We only tested writing to SSD disks for persistence.  Note: Chronicle is broker-less.

The results

The result of this test show you the message rate in terms of MB/s.  This is a reasonable way to describe the performance as the message size can vary, but you will get a similar amount of bandwidth, especially over 1 KB message sizes.

device flush period (ms) 1 2 5 10
ssd.ext4 1 236 MB/s 300 MB/s 340 MB/s 363 MB/s
ssd.ext4 3 378 MB/s 483 MB/s 556 MB/s 583 MB/s
ssd.ext4 10 495 MB/s 595 MB/s 687 MB/s 705 MB/s
tmpfs na 988 MB/s 1317 MB/s 1680 MB/s 1847 MB/s

We also tested "writing" to a tmpfs file system.  This is much faster as there is no actual writes to a device performed.

Conclusions

It isn't possible to draw a direct comparison with Kafka as it is a broker based system as must send ever message over TCP.  Chronicle can replicate over TCP, however it doesn't have to, and if you want to maximise performance you will use a high speed network, the fastest being the memory bus of your server.

You can run similar tests and get exceptional results.  If you need to handle bursts of hundreds of MB/s, Chronicle may be a better solution.

Comments

  1. Rüdiger Möller2 October 2014 at 18:24

    Type in title.
    I often wonder why stuff is hyped that much. Actually 50MB/sec is underwhelming, This would not even saturate a 1GB network. Can you post numbers of chronicle via TCP ? Starting around ~200MB/sec usually encoding/decoding (not covered here) becomes a bottleneck [except one uses seriously handcrafted or flatbuffer alike encodings].

    ReplyDelete

Post a Comment

Popular posts from this blog

Java is Very Fast, If You Don’t Create Many Objects

Image
  You still have to watch how many objects you create. This article looks at a benchmark passing events over TCP/IP at 4 billion events per minute using the net.openhft.chronicle.wire.channel package in Chronicle Wire and why we still avoid object allocations..  One of the key optimisations is creating almost no garbage. Allocation is a very cheap operation and collection of very short-lived objects is also very cheap. Does this really make a difference? What difference does one small object per event (44 bytes) make to the performance in a throughput test where GC pauses are amortised? While allocation is as efficient as possible, it doesn’t avoid the memory pressure on the L1/L2 caches of your CPUs and when many cores are busy, they are contending for memory in the shared L3 cache.  Results Benchmark on a Ryzen 5950X with Ubuntu 22.10. JVM Vendor, Version No objects Throughput, Average Latency* One object per event Throughput, Average Latency* Azul Zulu 1.8.0_322 60.6 ...
Read more

System wide unique nanosecond timestamps

A Unique Identifier can be very useful for tracing. Those ids are even more useful when they contain a high-resolution timestamp.  Not only do they record the time of an event, but if unique can help trace events as they pass through the system. Such unique timestamps however can be expensive depending on how they are implemented.   This post explores a lightweight means of producing a unique, monotonically increasing system-wide nano-second resolution timestamp available in our open-source library. Uses for Unique Identifiers Unique identifiers can be useful to associate with a piece of information so that information can be referred to later unambiguously. This could be an event, a request, an order id, or a customer id. They can naturally be used as a primary key in a database or key/value store to retrieve that information later. One of the challenges of generating these identifiers is avoiding creating duplicates while not having an increasing cost.  You could ...
Read more

What does Chronicle Software do?

Image
Overview Chronicle Software is about simplifying fast data.  It is a suite of libraries to make it easier to write, monitor and tune data processing systems where performance and scalability are concerned. But its free, how do you make money, through support? We offer premium support . However, we often hear from users for the first time about a year after they have it in production.  Most users find they can support the software themselves.  It is only after a year or so, they have questions or concerns about where to take the product next.  In many cases, it is to a problem we have already solved and they just need to upgrade their software or use a new module we have added. As we don't know who is using most of our software, we can't advise them on how best to use our software and what updates or enhancements they would benefit from. We need users to contact us and ask questions.  We have a free forum , and we respond to 50% of questions in 2 hours...
Read more