Microservices Architecture Patterns: Event Sourcing and Event-Driven…

We begin the fifth installment of this architecture patterns series. In case you missed any previous articles, here are the links:

Today, we continue discussing communication and coordination between microservices, focusing on Event Sourcing and Event-Driven Architecture (EDA). In an upcoming post, we will conclude this section with CQRS (Command Query Responsibility Segregation), BFF (Backend for Frontend), and Outbox.

But we won’t stop there! There’s much more content to cover in future posts, where we will explore additional patterns and an example of scalability and resource management with auto-scaling.

Event Sourcing

The Event Sourcing pattern is a software design technique that involves storing a sequential record of all changes occurring in an application's state as a series of immutable events. Each event represents a unique and atomic change in the application's state and is stored sequentially in an event log, often referred to as a "journal" or "event log." This approach allows reconstructing the current state of the application at any point in time by replaying all events from the beginning up to the desired moment.

Characteristics of Event Sourcing

Immutable event log: All events representing changes in the application's state are immutable and stored sequentially in an event log.
State reconstruction: The current state of the application is rebuilt from the sequence of events by applying each event in chronological order.
Complete change history: The event log provides a full history of all changes that have occurred in the application over time.
Scalability and fault tolerance: The Event Sourcing pattern facilitates scalability and fault tolerance, allowing the application's state to be reproduced and reconstructed at any time.

Advantages of Event Sourcing

Auditing and tracking: Provides a complete record of all actions and changes made in the application, facilitating auditing and tracking.
Reproducibility: Enables the application's state to be replayed at any time, making debugging and troubleshooting easier.
Flexibility and evolution: Allows adding new functionalities retrospectively and adapting to changing business requirements by introducing new event types.
Consistency and atomicity: Events are atomic and applied consistently, ensuring the integrity of the application's state.

Challenges of the Event Sourcing Pattern

Implementation complexity: Implementing Event Sourcing can be more complex than traditional approaches due to the need to manage the event log and reconstruct the application's state.
Event log scalability: The size of the event log can grow significantly over time, posing challenges in terms of storage and performance.
Eventual consistency: Reconstructing the application's state from events can take time, leading to eventual consistency rather than immediate consistency.
Event modeling: Requires careful design of the event model to accurately represent all relevant changes in the application's state.

In summary, the Event Sourcing pattern is a powerful technique for efficiently and scalably managing an application's state. When implemented correctly, it can provide greater transparency, flexibility, and evolutionary capability compared to more traditional approaches. However, it also presents challenges in terms of implementation complexity and event log scalability.

Example

This would be a base event representing any action that occurs in our system:

import java.time.LocalDateTime;

public abstract class Event {
    private final LocalDateTime timestamp;

    public Event() {
        this.timestamp = LocalDateTime.now();
    }

    public LocalDateTime getTimestamp() {
        return timestamp;
    }
}

Specific events for actions such as:

Making a purchase: Each time a customer makes a purchase, a purchase event is generated containing all relevant information about the transaction, such as the purchase ID, purchased products, total amount, customer information, etc.

public class PurchaseEvent extends Event {
    private final String purchaseId;
    // Other relevant fields
    
    public PurchaseEvent(String purchaseId) {
        this.purchaseId = purchaseId;
        // Initialize other fields
    }
    
    // Getters for relevant fields
}

Payment Processing: When a payment for a purchase is processed, a payment event is generated containing details about the payment transaction, such as the purchase ID, payment method used, amount paid, etc.

public class PaymentEvent extends Event {
    private final String paymentId;
    // Other relevant fields
        
    public PaymentEvent(String paymentId) {
        this.paymentId = paymentId;
        // Initialize other fields
    }
        
    // Getters for relevant fields
}

Product Shipment: Once the purchased products are shipped, a shipping event is generated that records the shipping details, such as the purchase ID, shipped products, shipping address, etc.

public class ShippingEvent extends Event {
    private final String shipmentId;
    // Other relevant fields
            
    public ShippingEvent(String shipmentId) {
        this.shipmentId = shipmentId;
        // Initialize other fields
    }
            
    // Getters for relevant fields
}

Next, we create an event log where we will store all the events occurring in the system. In a real-world scenario, appropriate persistence techniques should be used to log data, such as queues, messaging brokers, etc.

import java.util.ArrayList;
import java.util.List;

public class EventLog {
    private final List<Event> events;

    public EventLog() {
        this.events = new ArrayList<>();
    }

    public void addEvent(Event event) {
        events.add(event);
    }

    public List<Event> getEvents() {
        return events;
    }
}

And finally, in our application service, we can log events whenever a relevant action occurs in the system.

For making a purchase:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class PurchaseService {
    private final EventLog eventLog;

    @Autowired
    public PurchaseService(EventLog eventLog) {
        this.eventLog = eventLog;
    }

    public void makePurchase(String purchaseId) {
        // Logic to process the purchase
        
        // Log the purchase event
        PurchaseEvent purchaseEvent = new PurchaseEvent(purchaseId);
        eventLog.addEvent(purchaseEvent);
    }
}

For processing the payment:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class PaymentService {
    private final EventLog eventLog;

    @Autowired
    public PaymentService(EventLog eventLog) {
        this.eventLog = eventLog;
    }

    public void processPayment(String paymentId) {
        // Logic to process the payment
        
        // Log the payment event
        PaymentEvent paymentEvent = new PaymentEvent(paymentId);
        eventLog.addEvent(paymentEvent);
    }
}

And for shipping:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class ShippingService {
    private final EventLog eventLog;

    @Autowired
    public ShippingService(EventLog eventLog) {
        this.eventLog = eventLog;
    }

    public void shipProducts(String shipmentId) {
        // Logic to ship the products
        
        // Log the shipping event
        ShippingEvent shippingEvent = new ShippingEvent(shipmentId);
        eventLog.addEvent(shippingEvent);
    }
}

We can see that, in a very simple way, a basic event sourcing system can be implemented. This approach allows us to achieve order traceability, enabling us to track which orders have been successfully completed and identify those that encountered issues. By integrating a reporting service or dashboard, we can gain valuable insights into the order processing flow.

EDA

Event-Driven Architecture (EDA) is an architectural style that focuses on the generation, detection, processing, and response to events occurring within a system or across distributed systems. In EDA, systems are designed to be highly reactive to events, enabling asynchronous and decoupled communication between system components.

Characteristics of Event-Driven Architecture

Events: Events are messages that represent an action or notification occurring within the system. They can be generated by users, applications, devices, or any other source and can be consumed by one or more components to trigger specific actions.
Asynchronous Communication: In EDA, communication between components occurs asynchronously through event exchanges. This allows components to communicate efficiently and in a decoupled manner, facilitating scalability and system flexibility.
Decoupling: Components in an event-driven system are decoupled from one another, meaning they can operate independently and are not directly dependent on other components.

Advantages of Event-Driven Architecture

Flexibility: EDA allows for greater flexibility in system design and evolution, as components can be modified or replaced without affecting others.
Scalability: Asynchronous communication and decoupling between components improve system scalability, enabling independent scaling of components as needed.
Reactivity: EDA helps systems be highly reactive to changes and events occurring in the environment, facilitating the implementation of real-time and high-availability systems.
Pattern Detection: EDA makes it easier to detect patterns and make event-driven decisions, allowing systems to be more intelligent and adaptive.

Challenges of Event-Driven Architecture

Design Complexity: Requires careful system and event design to ensure components communicate effectively and events are processed correctly.
Event Management: Managing events can be complex, especially in distributed systems with large event volumes. A robust infrastructure is needed to efficiently handle event exchange, processing, and storage.
Consistency and Coherence: Asynchronous communication can introduce challenges in maintaining data consistency and coherence across system components.
Debugging and Monitoring: Monitoring and debugging event-driven systems can be more complex due to the asynchronous nature of communication and the large number of events generated and processed.

In summary, event-driven architecture is a powerful architectural approach that focuses on generating, detecting, processing, and responding to events within or across distributed systems. It offers numerous advantages, such as decoupling, scalability, and reactivity, but also presents challenges in terms of system design, management, and monitoring.

Example

Let's assume we have a system where users can place orders and receive notifications when their products are shipped. We will use Kafka to send events related to orders and notifications.

First, we need to configure Kafka in our project using Spring Boot:

import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;

@Configuration
@EnableKafka
public class KafkaConfig {
    // Kafka configuration
}

Now, we define an event producer that will send order events to a Kafka topic:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

@Component
public class OrderEventProducer {

    private static final String TOPIC = "order-events";

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendOrderEvent(String orderId) {
        kafkaTemplate.send(TOPIC, orderId);
    }
}

Next, we create an event consumer that will listen to the Kafka topic and process order events:

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

@Component
public class OrderEventListener {

    @KafkaListener(topics = "order-events", groupId = "order-group")
    public void receiveOrderEvent(String orderId) {
        // Process the order event
        System.out.println("Order received: " + orderId);
        // Logic to handle the order, such as sending notifications, updating order status, etc.
    }
}

Finally, in our application service where an order is placed, we can send an order event to the event producer:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class OrderService {

    private final OrderEventProducer orderEventProducer;

    @Autowired
    public OrderService(OrderEventProducer orderEventProducer) {
        this.orderEventProducer = orderEventProducer;
    }

    public void placeOrder(String orderId) {
        // Logic to place the order
        
        // Send an order event
        orderEventProducer.sendOrderEvent(orderId);
    }
}

When an order is placed in the system, an order event is sent to the Kafka topic using the event producer. The event consumer listens to the Kafka topic and processes the order events, such as sending notifications or updating the order status.

This is just a simple example to illustrate how to implement an event-driven architecture (EDA) system using Java and Kafka. Depending on your specific requirements, you can add more logic and functionality to your system.

Below is a diagram illustrating an example of an event-driven architecture (EDA) for other e-commerce use cases:

This example illustrates three event producer components and the events they generate. We also see an event router that captures and filters events, then directs these events to the relevant event consumer.

Event-driven architecture enables the system to react to changes from various sources during peak demand periods without blocking the application or over-provisioning resources.

¿Event Sourcing = EDA?

No! Event Sourcing and EDA are related concepts, but they are distinct in the field of software engineering.

Event Sourcing is a software design pattern that stores a sequential record of all changes occurring in an application's state as a series of immutable events. Each event represents a unique and atomic change in the application's state and is stored sequentially in an event log. The primary goal of Event Sourcing is to maintain a complete and auditable history of all state changes and allow for the reconstruction of the application's current state at any given moment by replaying all events.

Event-driven architecture (EDA) is an architectural approach that focuses on the exchange of events between components in a distributed system. In an event-driven system, components can emit events when specific actions or state changes occur, and other components can react to those events and execute corresponding actions. EDA emphasizes asynchronous communication and independence between system components, enabling greater flexibility, scalability, and decoupling.

Among their main differences:

Primary focus: Event Sourcing focuses on recording and storing events to maintain a complete history of the application's state, whereas EDA focuses on asynchronous communication between system components through event exchange.
Scope of application: Event Sourcing is primarily used for managing an application's internal state, while EDA applies to communication between systems, microservices, or components within a distributed system.
Nature of events: In Event Sourcing, events represent changes in the application's state and are stored immutably. In EDA, events can represent various actions or notifications occurring in the system and may be consumed by multiple components to trigger specific actions.

In summary, while Event Sourcing and event-driven architecture share similarities in their use of events, they have different focuses and objectives in software system design and implementation. Do you have any questions or comments? Let us know below!

Yavé Guadaño

Since 2006, I’ve been dedicated to this profession (at Paradigma Digital since 2009). My career has been shaped more by people than by projects. With hard work and dedication, anything can be achieved. Always connected to technology, I love creating PoCs for new trends and staying up-to-date. I currently work at Paradigma as a Software Architect.

View more of Yavé.