Posts

Showing posts from June, 2022

Event-Driven Order Processing Program

Image
  Following the   Hello World example   of a simple, independently deployable real-time Event-Driven Microservice, this article looks at a more realistic example of an Order Processor with a New Order Single in and an Execution Report out.  A  New Order Single  is a standard message type for the order of one asset in the FIX protocol used widely by financial institutions such as banks. The reply is typically one or more  Execution  Report s updating the status of that order. Some Background on Fintech In fintech, when one organisation wishes to purchase an asset or commodity from another, they send an order.  The other organisation sends a message to notify if the order was successful; this message is called an execution report. You could think of it a bit like a trade receipt. These orders and execution reports are transmitted electronically, using a data format standardised by Financial Information eXchange (FIX). There are many different o...

Efficient Memory Mapping for Terabyte Sparse Files in Java

Image
  On Linux, you can create  sparse files , where only the pages (of 4 KiB) that are touched utilise either memory or disk space. This allows you to memory map large virtual regions without worrying about wasted memory or disk In this program, you can see it reserves 8 TiB (8,192 GiB) Figure 1.  Test 1: Sparse file Tip:  x << y means x × 2 y  therefore  1L << 10 = 1 KiB (1024 bytes),  1L << 20 = 1 MiB (1024 2  bytes),  1L << 30 = 1 GiB (1024 3  bytes),  1L << 40 = 1 TiB etc Using multiples of 10 for the shift makes them easier to read.  64L << 20 is 64 × 2 20  = 64 × 1024 2  = 64 MiB.  The virtual memory size of the above process is just over 8192 GiB at 8200.7 GiB, but the RSS (Resident Set Size) is only  122,060  KB, or 122  MB. Figure 2. RES for Test 1 On disk, the extents reported are 8 TiB, however the amount of disk (and memory) actually used is just 20 KiB....