Microservices: Event-driven Patterns

 

Event driven microservices being applied to data pattern.

• Event driven microsercvice pattern requires one or more steps based on the invocation from specific event these can be as simple as writing a tuple to a database or complex as a series of mediation process. That are required when a specific system event occurs when working this model. The key is one or more steps
• Those events, however triggered from a single event. we don’t usually build event driven microservices based on several events being fired before kicking of the workflow it usually one single event. Event driven mocroservices architecture fired from successful check all other data is in a proper state
• Towards end Goal: All of the steps are point of the same goal
• Each piece plays it parts. each step should be distinct and required, These isolation steps allow the components, to the work there are assign to do and nothing more, but nothing less. The isolation and definition of what the role is allow the developers to build systems that the function as expected. because there can be a concrete definition each of steps

Choreographed Events

Choreographed event as the name implies are much like a dance choreography. choreographer lays out the framework for the dance. but each dancer does his or her part independent of the choreographers instructions.in a event driven asynchronous microservice pattern that uses choreographed events, the same holds true. The choreographer is played by the initializer of the event , but each step is on its own  from there on out

This often can be called a call tree. each step does some work and passes the message down the chain. there is no centralized controller of the choreographed events. They just cascade down the pipeline. In a pipe, the output from one command is passed to another. this is similar to how choreographed events work. while the full output of step isn’t passed to the next step that often some trigger will passed with sufficient data to the next step for its to do its work and so on down the call chain

There are two distinct use cases

Distinct System

• When you have different systems in play. few of us work in massive teams. that have tentacles into the entire operation of our company. As such, silos are often created within the system. when you are in this model, but you have an event that impacts many systems it usually make the most sense to fire event and allow the cascade to occur.
• The initial event execution will trigger the remote system the start its own event chain as they in turn may call in other systems through events to do the same
• The risk of having any form of coupling when teams are distinct are often too great for productivity to warrant any other model in these cases
• Separate teams have different ways of doing things different languages and often different processes. choreography makes lot of sense in these scenario

Alternative Cascades

• In this use cases each step may have result may have that triggered more than one next step sometime at once. As such the choreography sets the wheels in motion and each step along the way does its work and primes the pump for any alternatives step that may follow
• The reality is that each step trigger the downstream step via a message, not a direct call in pure asynchronous microservices model.

In Choreographed events We start with with out message broker. we then have an event producer. This is what choreographs the event driven message. Now, in this case, our choreographed event has three unique steps. All of these components produce message for the broker, consume them, or both. The message is produced by the event producer. It is sent to the broker. Its then read by the first step, and the work is done. A new message is sent back to the broker. That message is ready by the next step and executed on. It then created a new message and send it back to the message broker. The third step then reads the message and executes on it. That the event producer isn’t involved after the initial production of the message.

Benefits and Tradeoffs 

• Increased performance:

Increased performance over orchestration, because don’t have a centralized orchestrater. The steps don’t funnel through a single process. As such you could increase performance by off loading the steps to the message broker Each step can be optimized for it solve function which is the work to be performed

• Reduced Cost:

Also because of performance in code complexity have a reduced total cost of ownership

• Reduced reliability:

There is no central place to handle error states there is more chance that is a single event will fail to fire everywhere that its needed. Need to really ensure that error tracking and DLQs exist in the message broker to indicate need to address errors in the workflow

• Reduced Observability:

Because there is no centralized orchestrater, remove single point for Observability. This can make determining the status of the event that much more difficult. Have to look everywhere until find the state of the current message to really knows whats going on.

Orchestrated Events

The command and control they offer make sense lot of architectural decision. There are attractive to software architect and developer. Much more common practice

Centralized command control. More often than not, we know the steps that are involved in the process and as such. we have ability to invoke the steps as we needed to and oftentimes poll for a result. Because we are in asynchronous pattern the polling result is standard either from the source executing the step or may be from a database of some other state store. The orchestrater knows which events are ready to fire and while handling the calling downstream via message system.

Still based on the steps. This model is however based on isolated steps

Each step still has a job to do. This isolated steps are distinct. seldom, It ever do steps need to know about other steps in the overall orchestration.

Some Use Cases

• Sequential Processing:

The need to dispatch messages intended for remote workers. depends on the current state of the system. This model all communications is asynchronous but the producer is the original message becomes a consumer on the response.

• Command workflow:

You can make direct calls bu often the blocking nature of those calls hinders the throughput of the system.As such you can dispatch the event to a worker process the worker does its work and when its done send response message back to the indented command and control center

It can dispatch another message to system to another worker providing the connection details of the newly created resources. created many at once through a single orchestrater call, since the call has become non-blocking asynchronous messages.

• Response Required:

Anytime when doing asynchronous messaging and you need a response be it for sequential processing, command workflows or other use cases. the orchestrater can be the point where this response structure exists.  It will usually aggregate all of the response in to a master status that is available to any system interested. Most often the event originator Each step can respond to the orchestrater via message as needed but the master response will be aggregated in this model.

.

Orchestrated model Once again, we start with a message broker. we then have an orchestrater. It knows about the message broker. we also have an event producer. It will make an asynchronous restful call over HTTP. We then have several steps. each of the steps knows about message broker. The event producer makes a call to the orchestrater. It creates a message and sends it to the broker. The broker then exposes the message for the first step which receives the message and does work. when its done, it sends the message back to the message broker, which the orchestrater receives. All the while the poller may be pulling the orchestrater for a response. This process continues until all of our work is done. When finished the polling will succeed, and entire works is done for our system.

 

Benefits and Tradeoffs

• Reduced performance:

Because you have a centralized orchestrater the steps funnel through the single process as such decrease performance. Now the orchestrater must have a state machine of some kind to keep track of the status responses and code paths that handle the downstream actions.

• Increased Cost:

This increases total cost of ownership. The orchestrater needs to be beefier and often needs to have much more complex code. These cost in both operation and developer efficiencies.

• Increased reliability:

The orchestrater can use it state to resubmit jobs. that need to be processed you can also build in much more robust error handling for outlier conditions.

• Increased Obeservability:

The centralized controller becomes the sweet spot for Observability while you still have disjointed processes that aren’t solved in this model for Observability this central point can provide more logging and metrics. That can help an operation team.

Hybrid Events

In this model, you still have a centralized command and control structure for your system as a whole. But when the need arises, you can dispatch choreographed events to remote systems. They in turn can either stay choreographed or convert to an internal command and control center structure. Works get done, and if you do it right, the original command control knows when everything is finished. either directly inline or via some other pulling mechanism. The centralized command and control directs all traffic internally to the system of record, but also kicks off choreographed workloads in remote systems. In the end the remote system is responsible in some way to alert the centralized command and control structure that its finished. You can get a good mix of benefits here.

Contracts

No matter which event model you choose ,Contracts between your system via the message broker are key. The contracts must be well documented. There is no room for error when yo have disjointed processes. Mainly because you don’t need help making things more complex. a well documented contract of the message will help to alleviate this concern. The contract must be passive to change. Changes can and do happen, but they must not break downstream or upstream system in the process. Contracts must be enforced rigorously.  In order to be resistant to change in efficient in your preprocessing, you must read the contract and honor it. Furthermore, should you find a breaking change is you how your component is called, you send it to the DLQ quickly and efficiently to trigger alerting.

 

Thanks