Distributed System in System Design

Introduction

Nowadays, almost every large scale application is based on distributed computing. Technological expansion and the increasing use of highly scalable systems have paved the way for Distributed Computing, and hence Distributed Systems have became a necessity. It is the need of the hour to have the knowledge of distributed systems as it is very crucial and instrumental to design fault-tolerant, highly scalable and low latency services. This blog will introduce you to the fundamentals of Distributed Systems, their functioning, and how they are applicable in real-world systems using some of the exciting examples. So, let us dive in and learn how to deal with trade-offs while designing our system.

What is a Distributed System?

A Distributed System is a collection of many independent machines that work together by coordinating and communicating with each other to achieve a common goal. These systems operate concurrently, offer high scalability, and fail independently without affecting the application’s performance. Distributed System works in such a fashion that it appears as a single entity to the end-user. Distributed Systems allow Resource Sharing, Concurrency, Fault-Tolerance, Scalability, and Transparency.

Why Distributed System?

Distributed Systems are a collection of a large number of individual machines working together to achieve a common objective. They are made up of machines capable of connecting over a network, having local memory, and communicating messages. Hence it is evident that such systems consist of a huge amount of complexity. So the common question is- Why are we using Distributed Systems if they are so hard to manage, so hard to deploy, and even harder to debug because of its extremely complex behavior? :|

The most significant advantage of adopting Distributed Computing in almost every software is its ability to provide highly scalable services. Distributed Systems allow us to add as many servers and hence offers Horizontal Scaling. Traditional systems generally scale using Vertical Scaling, i.e., by increasing the single machine’s complexity by upgrading it multiple times. Such Vertical scalable services are not compatible when the service is operating on a vast scale because they are expensive and moreover they are more prone to single point of failure. However, horizontal scaling allow us scaling indefinitely without any worry. If there is any performance degradation, you only need to add more machines, which makes the service extremely fast without much overhead cost compared to Vertical Scaling.  The figure below describes how it cost a company to use Vertical vs. Horizontal Scaling.

Along with scalability, Distributed Systems offer low-latency and fault-tolerant Services. Most of the highly scalable services are read-heavy, and this might decrease the performance of any system. To deal with this, there is a facility in distributed computing to replicate the servers. This offers highly available services and also ensures fault-tolerant behavior.

However, one can efficiently deal with the service’s low performance by using replication, but there is also a specific limit to which this is convenient. With distributed systems, there is always a trade-off between complexity and performance. To further increase performance, distributed systems offer another way to scale the service by sharding the databases. With sharding, one can easily split the main server into many smaller servers called Shards and achieve higher performance by distributing the load and achieve highly available and latency-free services.

How Distributed System Works?

Distributed Systems heavily rely on network calls and communication from all the individual components. A well-established network is the utmost requirement of the Distributed System to connect with each component and offer highly scalable services. The messages need to be communicated reliably. There should be a mechanism of detecting any failure node so that if the node fails, the System should synchronize properly without affecting other functionalities. The distributed systems fall into any one of the four architectural modes:

  1. Client-Server Model: Most of the traditional architecture falls under this category. There is a server to which all the requests are made from the clients. Resource Sharing is one of the best examples of the Client-Server model.
  2. Three-Tier: In such architecture, a central server manages all the requests and services and acts as a middle layer between the communication. The middle tier accepts the request, does some pre-processing, and forwards it to the server for further processing.
  3. Multi-Tier: Such architectures are used when an application needs to forward requests to various network services. Here the application servers interact both with the presentation tiers and data tiers.
  4. Peer-to-Peer: There are no centralized machines required in this architecture. Each entity behaves as an independent server and performs their roles. Responsibilities are distributed among various servers called peers, and they cooperate to achieve a common goal.

Examples which are heavily relied on Distributed Systems are:

  1. Telecommunication Networks
  2. Parallel Processing
  3. Real-Time Distributed Services
  4. Distributed Databases and Caches
  5. Distributed File System and Stream Processing Services

Advantages of Distributed System

Nowadays, systems require high availability, consistency, scalability, and they need to be fault-tolerant. Hence the vast majority of applications rely on a distributed system. The key benefits of using Distributed Systems are:

Reliability:

Distributed Systems remain available most of the time irrespective of the failure of any particular server of the System. If one server fails down, the service remains operational.

Scalability:

Distributed Systems offer extensive scalability. The advantage of adding a large number of servers allows such systems to achieve horizontal scalability.

Low Latency:

Distributed Systems offer high-speed service because of the Replication of servers and servers’ location close to users, hence reducing the query time.

Disadvantages of Distributed System

It is essential to know the various challenges that one may encounter while using any system. This will helps in dealing with trade-offs. Distributed Systems offer high performance and scalability, but they are highly complex. The shortcomings of Distributed Systems are:

Complexity:

Distributed Systems are highly complex. Although using a large number of machines, the system can become scalable, but it increases the service’s complexity. There will be more messages, more network calls, more machines, and hence it becomes very tedious to manage such systems. 

Network Failure:

Distributed Systems have heavily relied on network calls for communications and the transfer of information or data. In case of network failure, message mismatch or incorrect ordering of segments leads to communication failure and eventually deteriorates its application’s overall performance.

Consistency:

Because of a highly complex nature, it becomes too challenging to synchronize the application states and manage the data integrity in the service.

Management:

Many functions, such as load balancing, monitoring, increased intelligence, logging, etc., need to be added to prevent the distributed systems’ failures.

Conclusion

Distributed Systems are a necessity of modern applications that are heavily relied on distributed computing. Distributed systems enabled modern systems to offer highly scalable, reliable, and fast services. Such systems can support many requests and compute jobs compared to a single standard system by spreading workloads and requests. Although there are some trade-offs and challenges, Distributed Systems can transform the world with its services and applications and as a result of which almost every application have Distributed System as their one of the major component. Distributed systems are the necessity of the modern world as new machines need to be added, and applications need to scale to deal with all the technological advancements and better services.

We welcome your comments

Subscribe Our Newsletter

Get well-designed application and interview centirc content on ds-algorithms, machine learning, system design and oops. Content will be delivered weekly.