Event Driven Hello World Program

 Event-driven microservices can be straightforward to describe before they are implemented, tested and maintained. They are also highly responsive to new information in real-time, with latencies in Java of below 10 microseconds 99.99% of the time, depending on the functionality of the small, independently deployable microservice. 

 

In this introductory article, we use an example event-driven Hello World program (a programming paradigm where the program flow is determined by events) to step through behaviour-driven development, where we describe the behaviour the business needs first as test data and write a very simple microservice which turns input events like this.

say: Hello World

Into outputs like this, by adding an exclamation point

say: Hello World!  # <- adds an exclamation point

All the code for this example is available on GitHub.

When modelling Event-Driven systems, a useful pattern is to have event-driven core systems with gateways connecting to external systems that might not be event-driven. To keep a clear separation of concern, business logic such as making a decision based on market data, or processing an order, is placed in the event-driven microservices, as these are the easiest to test, with the gateways connecting to external clients and systems being as thin as possible, so they are only concerned with acting as adapters and avoid containing significant business logic. 

Domain-Driven Design is focused on determining the requirements of domain experts. Their requirements are further divided into event-driven microservices. Where the information is passed as a series of events between the microservices.

The requirements for each internal microservices can be described in YAML for Behaviour-Driven Development.


Figure 1- Gateways connect internal services to external systems

All examples are in the Chronicle-Queue-Demo/hello-world module.

A Simple Event-Driven Contract

We model events as asynchronous method calls without arguments or one-to-many arguments e.g.

public interface Says {
   void say(String words);
}

This is the simplest Hello World example to get started. We can add to this interface other event types (methods) with multiple parameters. Parameters don’t have to be just primitives; they can also be complex data structures such as Data Transfer Objects.

There is no assumption about how the events produced by the microservice will be processed. It might be recorded but otherwise ignored, for now, processed immediately by a single microservice or read by multiple downstream microservices sometime later.  Thus, it doesn’t return a value. Any results will be emitted as events from the respective event handlers. In programming, an event handler is a callback routine that can operate asynchronously.

External Event Producers and Consumers

Often we need to integrate with the client’s external systems. As this is a simple “Hello World” example, let’s imagine that instead of external systems connected via gateways, we have a simple program that reads input from the console to provide upstream events and another simple program to write to the console, acting as a downstream gateway.

public class SaysInput {
   public static void input(Says says) throws IOException {
       BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
       for (String line; ((line = br.readLine()) != null); )
           says.say(line);
   }
}


public class SaysOutput implements Says {
   public void say(String words) {
       System.out.println(words);
   }
}

These can be integrated easily as the output of one is wired to the input of the other.

public class RecordInputToConsoleMain {
   public static void main(String[] args) throws IOException {
       // Writes text in each call to say(line) to the console
       final Says says = new SaysOutput();
       // Takes each line input and calls say(line) each time
       SaysInput.input(says);
   }
}

We can also record everything the producer performs to YAML to build tests later.

public class RecordInputAsYamlMain {
   public static void main(String[] args) throws IOException {
       // obtains a proxy that writes to the PrintStream the method calls and their  arguments
       final Says says = Wires.recordAsYaml(Says.class, System.out);
       // Takes each line input and calls say(theLine) each time
       SaysInput.input(says);
   }
}

Use the following to replay the output from a file.

public class ReplayOutputMain {
   public static void main(String[] args) throws IOException {
    // Reads the content of a Yaml file specified in args[0] and feeds it to SaysOutput.
     Wires.replay(args[0], new SaysOutput());
   }
}

Unit Tests for the RecordAsYaml and Replay Methods

The following unit tests were developed to test the functionality of recordAsYaml and replay methods in isolation and verify if they work as suggested above.  Having lots of text in unit tests is cumbersome, and in the next section, you can see how this text can be taken from files.

public class WiresTest extends WireTestCommon {
@Test
public void recordAsYaml() {
   ByteArrayOutputStream baos = new ByteArrayOutputStream();
   PrintStream ps = new PrintStream(baos);
   Says says = Wires.recordAsYaml(Says.class, ps);
   says.say("One");
   says.say("Two");
   says.say("Three");


   assertEquals("" +
           "---\n" +
           "say: One\n" +
           "...\n" +
           "---\n" +
           "say: Two\n" +
           "...\n" +
           "---\n" +
           "say: Three\n" +
           "...\n",
           new String(baos.toByteArray(), StandardCharsets.ISO_8859_1));
}


@Test
public void replay() throws IOException {
   ByteArrayOutputStream baos = new ByteArrayOutputStream();
   PrintStream ps = new PrintStream(baos);
   Says says = Wires.recordAsYaml(Says.class, ps);
   says.say("zero");
   Wires.replay("=" +
           "---\n" +
           "say: One\n" +
           "...\n" +
           "---\n" +
           "say: Two\n" +
           "...\n" +
           "---\n" +
           "say: Three\n" +
           "...\n",says);


   assertEquals("" +
           "---\n" +
           "say: zero\n" +
           "...\n" +
           "---\n" +
           "say: One\n" +
           "...\n" +
           "---\n" +
           "say: Two\n" +
           "...\n" +
           "---\n" +
           "say: Three\n" +
           "...\n", new String(baos.toByteArray(), StandardCharsets.ISO_8859_1));
}


interface Says {
   void say(String word);
}
}

By recording and replaying using YAML, our microservices are written, tested and debugged easily without any involvement of the messaging layer.

Let’s add a microservice as a data processor class that can have one or more event types. This microservice gets input events as text messages, adds an exclamation mark, and relays them to the output gateway.

public class AddsExclamation implements Says {
   private final Says out;

   public AddsExclamation(Says out) {
       this.out = out;
   }

   public void say(String words) {
       this.out.say(words + "!");
   }
}


Figure 2- A microservice that adds exclamation marks to input messages.

A Single-Threaded Event-Driven Process

We can combine these all stages in one process, one thread. While this is unlikely to be useful in production, putting microservices into a single thread makes it easier to test and debug.

public class DirectWithExclamationMain {
   public static void main(String[] args) throws IOException {
       SaysInput.input(new AddsExclamation(new SaysOutput()));
   }
}

Testing a Single Event-Driven Service

Instead of embedding large amounts of text in a test, we can read resource files. This makes them easier to read and maintain.

public class AddsExclamationTest {
   @Test
   public void say() throws IOException {
YamlTester yt = YamlTester.runTest(AddsExclamation.class, "says");
assertEquals(yt.expected(), yt.actual());
   }
}


Let’s update the input to see how easy it is to maintain this test. I will change the second input to Hello World and run the test again.

src/test/resources/says/in.yaml
---
say: One
...
---
say: Hello World
...
---
say: Three
...

Not only does the test fail, I can also click on the differences to see why.

At this point, I can either fix the test or accept the change by copying and pasting the actual result over the expected result in the out.yaml file.

In the next post, we will see how to implement more realistic example processing orders and automate many microservice tests from the configuration.  This provides a basis for creating highly performant, deterministic, redundant microservices.

Conclusion

This article shows the outline of creating and testing a simple microservice, which provides the basis for microservices which are easy to deploy and maintain.

Comments

Post a Comment

Popular posts from this blog

Java is Very Fast, If You Don’t Create Many Objects

Image
  You still have to watch how many objects you create. This article looks at a benchmark passing events over TCP/IP at 4 billion events per minute using the net.openhft.chronicle.wire.channel package in Chronicle Wire and why we still avoid object allocations..  One of the key optimisations is creating almost no garbage. Allocation is a very cheap operation and collection of very short-lived objects is also very cheap. Does this really make a difference? What difference does one small object per event (44 bytes) make to the performance in a throughput test where GC pauses are amortised? While allocation is as efficient as possible, it doesn’t avoid the memory pressure on the L1/L2 caches of your CPUs and when many cores are busy, they are contending for memory in the shared L3 cache.  Results Benchmark on a Ryzen 5950X with Ubuntu 22.10. JVM Vendor, Version No objects Throughput, Average Latency* One object per event Throughput, Average Latency* Azul Zulu 1.8.0_322 60.6 M event/s, 528
Read more

System wide unique nanosecond timestamps

A Unique Identifier can be very useful for tracing. Those ids are even more useful when they contain a high-resolution timestamp.  Not only do they record the time of an event, but if unique can help trace events as they pass through the system. Such unique timestamps however can be expensive depending on how they are implemented.   This post explores a lightweight means of producing a unique, monotonically increasing system-wide nano-second resolution timestamp available in our open-source library. Uses for Unique Identifiers Unique identifiers can be useful to associate with a piece of information so that information can be referred to later unambiguously. This could be an event, a request, an order id, or a customer id. They can naturally be used as a primary key in a database or key/value store to retrieve that information later. One of the challenges of generating these identifiers is avoiding creating duplicates while not having an increasing cost.  You could record every identi
Read more

Comparing Approaches to Durability in Low Latency Messaging Queues

Image
  A significant feature of Chronicle Queue Enterprise is support for TCP replication across multiple servers to ensure the high availability of application infrastructure. I generally believe that replicating data to a secondary system is faster than syncing to disk, assuming the round trip network delay wasn’t high due to quality networks and co-located redundant servers. This is the first time I have benchmarked it with a realistic example. Little’s Law and Why Latency Matters In many cases, the assumption is that the latency won't be a problem as long as throughput is high enough. However, latency is often a key factor in why the throughput isn’t high enough. Little’s law states, “ the long-term average number  L  of customers in a  stationary  system is equal to the long-term average effective arrival rate  λ  multiplied by the average time  W  that a customer spends in the system”. In computer terminology, the level of concurrency or parallelism a system has to support must be
Read more