I have seen a number of tests where a JVM has 10K threads. However, what happens if you go beyond this?
My recommendation is to consider having more servers once your total reaches 10K. You can get a decent server for $2K and a powerful one for $10K.
Creating threads gets slower
The time it takes to create a thread increases as you create more thread. For the 32-bit JVM, the stack size appears to limit the number of threads you can create. This may be due to the limited address space. In any case, the memory used by each thread's stack add up. If you have a stack of 128KB and you have 20K threads it will use 2.5 GB of virtual memory.
Bitness
Stack Size
Max threads
32-bit
64K
32,073
32-bit
128K
20,549
32-bit
256K
11,216
64-bit
64K
stack too small
64-bit
128K
32,072
64-bit
512K
32,072
Note: in the last case, the thread stacks total 16 GB of virtual memory.
You still have to watch how many objects you create. This article looks at a benchmark passing events over TCP/IP at 4 billion events per minute using the net.openhft.chronicle.wire.channel package in Chronicle Wire and why we still avoid object allocations.. One of the key optimisations is creating almost no garbage. Allocation is a very cheap operation and collection of very short-lived objects is also very cheap. Does this really make a difference? What difference does one small object per event (44 bytes) make to the performance in a throughput test where GC pauses are amortised? While allocation is as efficient as possible, it doesn’t avoid the memory pressure on the L1/L2 caches of your CPUs and when many cores are busy, they are contending for memory in the shared L3 cache. Results Benchmark on a Ryzen 5950X with Ubuntu 22.10. JVM Vendor, Version No objects Throughput, Average Latency* One object per event Throughput, Average Latency* Azul Zulu 1.8.0_322 60.6 M event/s, 528
I wrote an article on low latency microservices almost five years ago now. Chronicle Software has worked with a number of tier-one investment banks to implement and support those systems. What has changed in that time and what lessons have we learnt? Read this article and learn what we learned after five years of developing and supporting low latency microservices. Separation of Concerns Give Better Testability Microservices repeatedly demonstrated that testing and debugging business components were much easier with simple, stand-alone components with clear contracts between microservices. Unit tests were still used to start with. However, in 2017 we moved almost entirely to behavior-driven development of microservices. Unit tests are still used for lower-level libraries and utilities. As our microservices are all based on Kappa Architecture , all our behaviour driven tests are modeled as a series of events in and out of the service. An input test might look like this --- oms : O
There are things you can do in Java you rarely see, generally because there is no use for it. However, there are some unusual things in Java that could be surprisingly useful. Chronicle Software uses a number of different usual patterns in it’s low-level libraries most developers wouldn’t generally come across. One of them is a class that extends Throwable but isn’t an Error or an Exception. StackTrace Extends Throwable package net.openhft.chronicle.core; /** * Throwable created purely for the purposes of reporting a stack trace. * This is not an Error or an Exception and is not expected to be thrown or caught. */ public class StackTrace extends Throwable { public StackTrace() { this ( "stack trace" ); } public StackTrace(String message) { this (message, null ); } public StackTrace(String message, Throwable cause) { super (message + " on " + Thread. currentThread ().getName(), cause); } public static StackTrace forThread(Thread t) {
Comments
Post a Comment