Event Meshes and Multicloud
Digital transformation is leading to the accelerated adoption of cloud-native applications. These applications typically microservices style architecture that span across multiple zones to provide for scalability. Most cloud-native applications even span across multiple cloud providers in order to complete complex business processes such as in banking, logistics, and tele-health. Such complex business process requires orchestration among multiple microservices (sometimes in hundreds), multi zones, multi cloud providers and multiple data centers.
Event driven architecture (EDA) style is central to that orchestrated execution of business processes that are compartmentalized by business contexts and implemented through microservices. The EDA also plays a key role around data in the cloud-native space, that is bounded to microservices, and will need to be managed as different segment of transactions are processed across multiple microservices.
Event 101
An event is a change or action, in an enterprise system that produces a message. This message is then published in a event sourcing system in an enterprise. This message may be consumed by other systems in the enterprise which might trigger further action.
For example, consider this transaction: A bank account holder trying to transfer money to a foreign account. When this transaction is initiated, it constitutes an event which produces message about this transaction by such and such user is being initiated. The fraud and risk systems in the bank, that consumes these messages, might trigger action to check if the transaction to foreign bank is within regulatory compliance limits and/or if the transaction is fraudulent.
There are several tools that support event and messaging based architecture. However, Apache Kafka, the open source software, is the most popular event / messaging system. Some of its differentiators include its ability to horizontally scale, high throughput and ability to process high volume events with very low latency. Also Apache Kafka implements the messaging stream architecture with Advanced Message Queuing Protocol (AMQP) with features like transaction management, clustering of APIs etc. In addition, Red Hat has made an enterprise grade event streaming software (AMQ Streams) with optimized Apache Kafka solution directly on Open Shift container platform. Openshift with Knative eventing for Kubernetes makes it a highly desirable container platform towards a solid implementation of microservices based applications on containers with end to end
In essence, in a large enterprise, there are hundreds of thousands of events / messages are published and consumed while enterprise business processes are orchestrated. In such a large amount of events are produced and consumed — particularly serving applications across multiple business domains, zones, data centers and cloud providers, how to manage such a large stream of events?
Enter Event Mesh!
An event mesh is a dynamic infrastructure that propagates events across disparate a broad range of producers and consumers within the enterprise boundary (on premises data centers and cloud providers) as well as partner ecosystem.
Such a mesh infrastructure is needed to transport, monitor, manage and operate very high volumes of events flowing through heterogenous application topology with multiple application language run times (Java, C++, Python …), using multiple application integration/interoperation protocols (JMS, FTP, HTTP, SOAP, RPC …) spread over multiple cloud environments (IBM, AWS, Azure, GCP …). Such an infrastructure also enable the basis of separation of concerns with respect to interoperability among the portfolio of applications so the app developers focus on basic functions and business logic of the applications.
The combination of an EDA and event mesh establishes a simplified environment with regard to application development leaving the app developers need only be concerned about the core applications.
Conclusion:
EDA is central and key technology strategy for large enterprises adopting cloud-native technologies particularly with hybrid / multicloud ecosystems. Horizontally scalable and reliable event messaging systems (such as Apache Kafka, Red Hat AMQ Streams that supports Knative event patterns) helps you design highly distributed application topology. When such an event / messaging system is distributed across multiple locations and cloud providers, an event mesh is a required event management infrastructure that comes with key set of features that supports such multiple interaction patterns and reduces complexity for developers of modern applications that implement near real-time loosely coupled design patterns.