In today’s fast-paced digital landscape, orchestrating the flow of data and requests is akin to managing the traffic of a bustling metropolis. Two critical components in this digital thoroughfare are API Gateways and Load Balancers. While both play pivotal roles in optimizing system performance and enhancing security, understanding their unique functionalities is key to navigating the complexities of modern software architecture effectively.

So what exactly are API Gateway and Load Balancer?

An API Gateway is a middleware component that acts as a single entry point for clients to access multiple backend services or APIs, whereas a Load Balancer is a networking device or software component responsible for distributing incoming network traffic across multiple servers or resources in a balanced manner.

API Gateways and Load Balancers serve different purposes.

You would typically use an API Gateway when you want to do

  1. Request Routing: Directing incoming requests to the appropriate backend service based on predefined rules or configurations.

  2. Authentication and Authorization: Enforcing security measures by authenticating clients and authorizing access to specific APIs or resources based on defined policies.

  3. Rate Limiting: Controlling the rate of incoming requests to prevent overload on backend services and ensure fair usage of resources.

API Gateway has a few other features like Protocol Translation, Logging and Monitoring, etc. which are some of the more advanced concepts so for the sake of simplicity, we are not considering them in this article.

Now, you would use a Load Balancer when you want to do

  1. Traffic Distribution: Distributing incoming requests evenly across multiple servers or backend resources to prevent any single server from becoming overwhelmed and to ensure optimal utilization of available resources.

  2. Horizontal Scaling: Facilitating horizontal scalability by enabling the addition or removal of backend servers dynamically to accommodate changes in traffic volume or system requirements.

You can also use Load Balancers to have Session Affinity, Health Monitoring, etc. which are again some of the more advance concepts.

Now, when should we use an API Gateway and a Load Balancer?

By now we have established that API Gateway and Load Balancer solves completely different problems.

Let us say that you have an e-commerce site and you are running it on a micro-service architecture, where you have three services, Catalogue, User, and Payment

The Catalogue service manages all operations associated with displaying SKUs, maintaining cart states, and related functionalities. Meanwhile, the User service oversees tasks concerning user management, such as storing user information and managing account status. The Payment service is responsible for handling operations related to payment processing, including redirecting to the appropriate payment gateway and managing refunds.

To streamline the routing process, requests are routed based on their paths, with those starting with /catalogue directed to the Catalogue service, requests beginning with /user directed to the User service, and requests starting with /payment directed to the Payment service. This routing setup ensures efficient handling of requests and effective separation of concerns within the system architecture.

This is exactly something you would want an API Gateway to do.

Now, in the above example, let us assume that the Catalogue is seeing more traffic spikes compared to the User or Payment service (which is typically common in any e-commerce site).

What we can do now to handle the traffic spikes is to add more servers to the Catalogue service. To do this, we need a Load Balancer.

We would first create an auto-scalling group, which is nothing but a fancy name for a group of servers, then we would create an auto-scalling policy, which would have rules like if the total CPU utilization exceeds more than 75% add more servers to the pool (auto-scalling group) and attach this policy to the load balancer.

Now, it is the load balancer’s responsibility to determine which server from the pool would perform an incoming request and this determining of which server to use would depend on the algorithm that the load balancer is using. There are many different such algorithms like Round Robin, IP Hash, Least Connection, etc. We will not be covering the algorithms in-depth in this article.

How do both of them fit in an architecture?

Typically, architects go for a micro-service when they want to scale individual services independently. In that case, the typical system design would be

Your DNS record would point to the API Gateway and API Gateway would route the traffic to individual load balancers based on the URL route prefix, like /catalogue, /user, or /payment. Now, individual load balancers would route the request to the relevant server for performing the requested operation and return a response back to the client.

API Gateway and Load Balancer Working together diagram

Conclusion

In the world of digital systems, think of API Gateways and Load Balancers as superheroes with different powers but who work together to keep everything running smoothly.

API Gateways are like the friendly bouncers at the entrance of a club. They make sure only the right people get in and that everyone follows the rules. They’re great at directing traffic to different parts of the club and keeping an eye on how many people are coming in at once.

Load Balancers, on the other hand, are like traffic cops on a busy street. They make sure cars (or in this case, requests) are spread out evenly among different lanes (or servers). If one lane gets too crowded, they’ll send some cars to another lane to keep things moving smoothly.

So, when should we call on our superhero duo? Well, imagine you have a big shopping website with different sections for products, users, and payments. API Gateways are perfect for directing customers to the right part of the site, like the product catalog or the checkout page. Load Balancers come to the rescue when one part of the site gets really popular, making sure all the servers can handle the extra visitors without crashing.

Together, API Gateways and Load Balancers make sure our digital world keeps humming along, even when things get busy. They’re like the dynamic duo of the internet, working behind the scenes to make sure everything runs smoothly for us all.