Single Server Architecture

To kick things off, let’s envision a straightforward setup where all components, including the database, cache, and web application, reside on a single server.

Users access websites via domain names, such as api.mysite. The request is directed to the Domain Name System (DNS), which retrieves the corresponding IP address for the user's browser. Subsequently, Hypertext Transfer Protocol (HTTP) requests are sent straight to the web server, which responds with HTML pages or JSON data for rendering.

Database Considerations

A single server setup quickly becomes insufficient. To effectively manage web traffic and database operations, we need to introduce a dedicated database server.

When selecting a database, you can choose between relational and non-relational databases:

• Relational Databases: Known as RDBMS or SQL databases, these include popular systems like MySQL, Oracle, and PostgreSQL. They organize and store data in tables and rows, allowing for join operations via SQL.
• Non-Relational Databases: Referred to as NoSQL databases, examples include CouchDB, Neo4j, and Amazon DynamoDB. These databases are categorized into key-value stores, document stores, column stores, and graph stores. They may be suitable for applications requiring low latency or dealing with unstructured data.

Scaling Approaches: Vertical vs. Horizontal

Vertical Scaling involves enhancing a server’s capabilities by adding more resources (CPU, RAM, etc.). While effective for low-traffic scenarios, it has limitations:

• There’s a hard cap on resources; servers cannot be endlessly upgraded.
• A single point of failure (SPOF) exists: if the server goes down, so does the entire application.

Horizontal Scaling, on the other hand, allows for adding more servers to the infrastructure, which is essential for large-scale applications. A load balancer can manage this scaling effectively.

The first video, "System Design: Scale System From Zero To Million Users," provides insights into scaling strategies and practical applications.

Load Balancer Implementation

A load balancer distributes incoming traffic evenly across multiple web servers, enhancing security by allowing users to connect only to the load balancer's public IP. This setup protects the private IPs used for server communication.

In this configuration, if one server encounters issues, traffic can be rerouted seamlessly to another server, ensuring continuous operation. Additional web servers can be added to handle spikes in traffic, further improving resilience.

Database Replication Techniques

Database replication establishes a primary/replica relationship among databases, facilitating improved performance and high availability.

In this architecture:

• The primary database handles write operations while replicas are designated for read operations.
• With a higher demand for read operations, systems typically maintain more replicas than primaries.

Advantages of this setup include enhanced performance through distributed read operations and increased availability, allowing for uninterrupted access even if one database goes offline.

Caching Strategies

To optimize performance, implementing a caching layer to store frequently accessed data can significantly reduce database load.

A cache acts as temporary storage for expensive responses, which can accelerate data retrieval and improve overall system efficiency. The cache tier is crucial for reducing database requests and can be scaled independently.

Content Delivery Network (CDN)

A CDN is a network of geographically distributed servers that cache static content, enhancing load times for users based on their proximity to the servers.

The second video, "System design interview: Scale to 1 million users," explores the practical aspects of designing scalable systems effectively.

Considerations for CDN Usage

When leveraging a CDN, it’s essential to consider factors such as cost, cache expiry settings, and how the system handles CDN failures. Proper management ensures that your application remains responsive, even during outages.

Stateless Web Tier Design

To facilitate horizontal scaling, it's vital to offload session data from the web tier into a persistent data store, allowing multiple web servers to access shared state data.

This design simplifies the architecture, making it more robust and scalable, thus enabling auto-scaling based on user traffic.