Following on from the previous post dedicated to the new microservices that also work in Netflix’s filter bubbles, we will conclude this who is who with these new players in town.

Hystrix

What is it? Hystrix is a library that implements the CicuitBreaker pattern. Hystrix lets us manage the interactions between services in distributed systems adding latent logic and fault tolerance. Its purpose is to improve the global reliability of the system, to achieve this Hystrix isolates the access points of the microservices, therefore impeding failures in waterfall through the applications different components, providing an alternative ’’fall back’’, managing timeouts, thread pools…

How does it work? Hystrix encapsulates the requests made to “external” systems in order to manage different aspects of these, such as: timeouts, success and failure statistics, traffic lights, error management logic, spread of errors in waterfall…

So for example if a request to a service reaches a certain level (20 failures in 5 seconds by default), Hystrix will open the circuit so that more requests won’t be sent to the previously mentioned service, which will impede the spread of errors in waterfall.

What does it provide?

• It encapsulates the requests to “external” systems and runs them inside a separate thread. This is an example of the “command pattern”.

• It is in charge of cancelling requests that have exceeded the timeout that has been established.

• It manages traffic lights/ thread pools for each request to an “external” system, in a way that if there aren’t any threads available, the request will be rejected instead of being put in a queue, therefore avoiding unnecessary waiting time (“failing fast instead of queueing”).

• It manages the spread of errors in waterfall.

• It measures statistics of successful and failed requests and of timeouts.

• It manages requests so that in the event of an external system exceeding a defined error quota, more requests won’t be sent to that system.

• It manages the fallback execution if there is an error in a request in order to protect the user from the failure.

• It provides a dashboard that integrates captured metrics, where you can observe the number of successful or failed requests, as well as timeouts, number of threads, number of hosts, percentile of response time… in real time each one of the requests managed by Hystrix. These real-time metrics allow us to optimize the ‘’time to discovery’’ in the event of failures.

• The metrics data generated by Hystrix can be interpreted by Turbine, in order to add up all the requests managed by Hystrix of all the microservices of our ecosystem, in just one dashboard.

Ribbon

What is it? Ribbon is a library, designed for communication between processes in the Cloud, that carries out loading balance on the client's side. Each one of the balancers is part of a set of components that work together in order to communicate with a remote server on demand. One of Ribbon’s key points is the possibility of integrating it with Eureka in order to discover the different instances a microservice has. Although Ribbon can work by itself, in this article we focus on its integration with Eureka.

How does it work? When we need to call upon a microservice, firstly we will define our request to the previously mentioned microservice, identifying it by the name it has been registered with in Eureka. It will not be necessary to identify a certain instance of the microservice, nor the machine in which it’s found, its IP or port.

Ribbon will use the microservice’s name that we have indicated, and check Eureka’s register in order to recover the number of instances from that microservice and what machines they are in. With that information, Ribbon will run the loading balance algorithm that is set (Round Robin by default), in order to determine which instance of the microservice to call upon. Once the instance has been decided, Ribbon will encapsulate the request inside a “Hystrix Command” and carry it out. The request management will then be done by Hystrix.

What does it provide?

• Removal of the number of existing instances, which is called upon and its location: when a microservice’s name is called upon, we don’t need to know the number of available instances, which of them we are calling upon nor the location (IP or machine name) of the machine it’s running in.
• It has different implementations for balance algorithms: round robin, based on response time, random... besides it allows you to implement your own algorithm.
• Zone management: in cloud environments (as AWS), Ribbon is able to distinguish between different zones and it can be configured to only call upon microservices from its own zone, therefore reducing network cost and response time. Besides, it can detect the health of the zone’s instances and avoid the “most problematic” zone dynamically.
• Thanks to its integration with Eureka, it can detect which microservice instances are down and not send them requests.
• It allows you to configure retries for the requests automatically in case of failure as well as to set timeouts.
• It allows you to make “warm-up” requests so that the microservices are available when actual requests start.
• It integrates with Hystrix to encapsulate its requests in a “Hystrix Command”, which gives us the advantages of Hystrix.

Zuul

What is Zuul? Zuul is an “edge service” that allows dynamic routing, load balancing, monitoring, and request security. In a practical sense, Zuul is a server made up of filters, each of which is oriented to a functionality such as the previously mentioned.

How does it work? In a microservices arquitecture Zuul will normally be configured as the entry point to the ecosystem and will be in charge of routing, balancing, securing… the requests that the microservices receive.

Every request sent to Zuul will go through the filters that make it up and which, depending on the request’s characteristics, can for example reject it for security issues, register it with the purpose of monitoring, and redirect it to a specific microservice instance… according to the configured filters or those which we have implemented.

Zuul uses Ribbon by default in order to locate, through Eureka, the microservice instances to which it will redirect the requests it will run inside a “Hystrix Command”, integrating all the architecture’s components and making the most of all the advantages the springcloud ecosystem provides.

Zuul also allows us to configure routing to microservices through properties, so that a microservice doesn’t have to be necessarily called on by the name it has been registered with in Eureka. However, this routing functionality doesn’t use the other elements of the architecture like Ribbon and Hystrix.

What does it provide?

• It provides a system capable of reacting fast, changing its behavior depending on the different challenging situations it might face: this is achieved through the different filters Zuul is made up of.
• It provides a series of filters to handle different situations: authentication and security filters, monitoring filters, dynamic routing filters, load testing filters, resource management or static response filters, multi-region management filters...
• It allows the implementation of personalized filters and the upload of these without restarting the server, which allows load rebalancing, request redirecting and cancelling changing the behavior of the app in real time without restarting the server... This is achieved thanks to Zuul’s ability to compile and run filters without stopping the app. So, for example, a filter could be implemented and from a certain moment redirect a small amount of requests to a microservice instance with a new version so that you can test it.
• When all the requests are moved to the microservices ecosystem through Zuul, it can carry out common tasks such as CORS management, authentication which, this way, won’t have to be carried out later in each of the microservices.

There are other key elements of this new type of architectures left such as logs management, deployment management, monitoring, containers… but we will leave those for another time.

In conclusion

As we have seen, all of these components are designed to integrate with each other, making up a new high availability, fault tolerant architecture… which has been tested in a highly complex and demanding environment such as the one Netflix faces every day, with thousands of millions of requests. An architecture that fully integrates Cloud Computing’s concepts and philosophy.

An architecture that Pivotal has decided to include in Spring. Without a doubt this has favored it, allowing it be become a reference point, at the cutting edge of the new technology wave cloud brings. So don’t lose sight of it, because we are going to be hearing a lot about microservices and Spring Cloud.

Sources

• Documentation from Spring Cloud Netflix
• Eureka’s development official Wiki
• Cloud­-config’s development official website
• Hystrix’s development official Wiki
• Ribbon’s development official Wiki
• Zuul’s development official wiki
• Turbine’s development official wiki
• Asgard’s development official wiki
• CircuitBreaker by Martin Fowler
• Command Pattern
• Scalable Microservices at Netflix. Challenges and Tools of the Trade por Shudir Tonse