Load Balancers in System Design

What are Load Balancers?

A load balancer is a software or a hardware device that sits between clients and a set of servers and basically has a balancing workload across resources. Not only do load balancers save our server or multiple servers from overloading, but it also increases the throughput of your system.

Let us start from the beginning.

In this system, we have several clients sending requests to a single server. When the number of clients increases significantly, the server experiences an overload, leading to failure in the system.

So, how do we handle it?

We scale our system. The first way is to vertically scale our system or increase the power of our server. But, there's only so much that we can do about it to increase a single machine's power. Another way is to scale the system horizontally, meaning to add more machines or servers to our system.

Now, how will clients know to issue requests to the servers in a balanced way? This is where load balancers come into play.

A load balancer is a server that sits between the clients and a set of servers and basically has the job of balancing workload across resources.

Load Balancers can also be between servers and databases, or when you are dealing with a website, you might have a load balancing at the DNS layer.

There is something called DNS Round Robin, which is basically a type of load balancing that happens at the DNS layer where a single domain name gets multiple IP addresses. A DNS query is made to get the domain name's IP address, gets multiple IP addresses, and is associated with that domain name to be returned uniformly.

It is also important here to know that there are different load balancers; namely, some load balancers are software load balancers, and there are hardware load balancers. Hardware load balancersare literally physical machines that are dedicated to loading balancing. The main difference between hardware load balancers and software load balancers is that you can do more with a software load balancer. You have more power for customization and scaling with software load balancers. With hardware load balancers, you are limited to the hardware that you are given with.

How load balancers select the servers to send traffic to?

There are a lot of different ways to do that. One might imagine one way is to redirect (if you want to) randomly. It is possible, in this case, that one server gets overloaded.

Another way that the load balancers often select servers following a Round Robin approach is actually the technique we were talking about earlier, where we mentioned the DNS Round Robin type of balancing.

So, Round Robin is basically a method that goes through all of your servers in one order (for example, it goes from one to last and then again to one).

A more complex type of Round Robin strategy that you can have your load balancer follow is what is called a weighted Round Robin, where you can basically place weights on specific servers meaning you would still follow the order of going to first one first and then second, third, …. the again to the first.

Another way for a load balancer to select servers is based on performance or load. This strategy is a little more involved; Basically, your load balancer performs health checks on your servers to know how much traffic a server is handling at any given time, how long a server is taking to respond to the traffic…

We can also select servers by IP-based selection strategy where basically your load balancer gets a request from clients; it will hash the clients' IP address and, depending on the value of the hash, redirect the traffic accordingly. IP-based load balancing can be beneficial if you have got caching going in your server.

One final method of server selection for load balancers that can be very useful is a path-based server selection strategy. With this strategy, a load balancer distributes requests to the server according to the path of the requests. So, basically, what this means is that all requests related to one process will go to one server, say “running code,” and all requests related to another process, say “payments,” will go to another server, and so on…

The interesting thing about all this is that it might make sense to have multiple load balancers that use different server selection strategies in a system.

server selection strategy in load balancer

We welcome your comments

Subscribe Our Newsletter

Get well-designed application and interview centirc content on ds-algorithms, machine learning, system design and oops. Content will be delivered weekly.