High Availability Architecture Patterns

What High Availability Really Means

High availability (HA) isn't just about having two servers. It's a comprehensive approach to designing systems that remain operational despite component failures. The industry standard is measured in "nines":

Availability %    | Downtime per Year | Use Case
------------------+-------------------+------------------
99% (two nines)   | 3.65 days         | Internal tools
99.9% (three)     | 8.76 hours        | Standard web apps
99.99% (four)     | 52.6 minutes      | E-commerce, SaaS
99.999% (five)    | 5.26 minutes      | Financial, healthcare

For most Indian businesses, 99.99% (four nines) is the sweet spot—achievable without astronomical costs.

Identifying Single Points of Failure

Before designing HA, audit your current architecture for SPOFs:

Common SPOFs in Traditional Indian Hosting:

User --> Single DNS Provider --> Single Load Balancer --> Single Database | | | SPOF #1 SPOF #2 SPOF #3 ```

Every component in your request path that has no redundancy is a potential outage. Map every path from user to data and back.

Pattern 1: Active-Passive Failover

The simplest HA pattern. One server handles traffic while another stands by, ready to take over.

                    +-------------+
                    |   Virtual   |
                    |     IP      |
                    +------+------+
                           |
              +------------+------------+
              |                         |
        +-----v-----+           +------v-----+
        |  Active    |  Heartbeat|  Passive   |
        |  Server    |<-------->|  Server    |
        |  (Primary) |          |  (Standby) |
        +-----+-----+           +------+-----+
              |                         |
        +-----v-----+           +------v-----+
        |  Primary   |---sync-->|  Standby   |
        |  Database  |          |  Database  |
        +-----------+           +------------+

Configuration with Keepalived

# /etc/keepalived/keepalived.conf (Primary)
vrrp_script chk_nginx {
    script "/usr/bin/killall -0 nginx"
    interval 2
    weight 50
}

vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1

authentication { auth_type PASS auth_pass ServerRaja2024 }

virtual_ipaddress { 10.0.1.100/24 }

track_script { chk_nginx }

notify_master "/etc/keepalived/notify.sh MASTER" notify_backup "/etc/keepalived/notify.sh BACKUP" notify_fault "/etc/keepalived/notify.sh FAULT" } ```

**Pros**: Simple, cost-effective, easy to understand **Cons**: Standby server is idle (wasted resources), failover time is 5-30 seconds

Pattern 2: Active-Active with Load Balancing

Both servers handle traffic simultaneously. If one fails, the other absorbs the full load.

                    +-------------+
                    | Load Balancer|
                    |  (Nginx/HA) |
                    +------+------+
                           |
              +------------+------------+
              |                         |
        +-----v-----+           +------v-----+
        |  Server A  |           |  Server B  |
        |  (Active)  |           |  (Active)  |
        |  50% load  |           |  50% load  |
        +-----+-----+           +------+-----+
              |                         |
              +------------+------------+
                           |
                    +------v------+
                    |  Shared DB  |
                    |  Cluster    |
                    | (Primary +  |
                    |  Replica)   |
                    +-------------+

Nginx Active-Active Configuration

upstream app_servers {
    least_conn;
    server 10.0.1.10:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=10s;
}

server { listen 80; location / { proxy_pass http://app_servers; proxy_next_upstream error timeout http_502 http_503; } } ```

**Pros**: No wasted resources, instant failover, better performance under normal load **Cons**: Requires stateless application design or shared session storage

Pattern 3: Multi-Region Active-Active

For mission-critical applications, deploy across multiple geographic regions.

    +--------------+     +--------------+     +--------------+
    |  Mumbai DC   |     |  Chennai DC  |     |  Pune DC     |
    |              |     |              |     |              |
    | +----------+ |     | +----------+ |     | +----------+ |
    | | App Tier | |     | | App Tier | |     | | App Tier | |
    | +----------+ |     | +----------+ |     | +----------+ |
    | +----------+ |     | +----------+ |     | +----------+ |
    | | DB Primary| |     | |DB Replica| |     | |DB Replica| |
    | +----------+ |     | +----------+ |     | +----------+ |
    +------+-------+     +------+-------+     +------+-------+
           |                    |                    |
           +--------------------+--------------------+
                        |
               +--------v--------+
               |   Global DNS    |
               |  (Geo-routing)  |
               +-----------------+

Database Replication Across Regions

-- PostgreSQL streaming replication setup
-- On primary (Mumbai):
ALTER SYSTEM SET wal_level = 'replica';
ALTER SYSTEM SET max_wal_senders = 5;
ALTER SYSTEM SET wal_keep_size = '1GB';

-- pg_hba.conf - Allow replication connections -- host replication replicator 10.0.2.0/24 md5 -- host replication replicator 10.0.3.0/24 md5

-- On replica (Chennai): primary_conninfo = 'host=mumbai-db.serverraja.com user=replicator password=secure_pass' ```

Pattern 4: Database High Availability

Databases are typically the hardest component to make highly available.

MySQL Group Replication

# /etc/mysql/mysql.conf.d/group_replication.cnf
[mysqld]
plugin_load_add = 'group_replication.so'
group_replication_group_name = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
group_replication_start_on_boot = OFF
group_replication_local_address = "10.0.1.10:33061"
group_replication_group_seeds = "10.0.1.10:33061,10.0.1.11:33061,10.0.1.12:33061"
group_replication_single_primary_mode = ON

Redis Sentinel for Cache HA

         +--------------+
         |   Sentinel   |
         |   Monitor    |
         +------+-------+
                |
    +-----------+-----------+
    |           |           |
+---v---+ +----v----+ +---v---+
|Redis  | | Redis   | |Redis  |
|Primary| |Replica 1| |Replica|
|       | |         | |   2   |
+-------+ +---------+ +-------+

Failure Detection and Automated Recovery

HA is only as good as your failure detection. Implement multi-level monitoring:

# Health check with dependency verification
def comprehensive_health_check():
    checks = {
        "application": check_app_responsive(),
        "database": check_db_connection(),
        "cache": check_redis_connection(),
        "disk": check_disk_space(threshold=90),
        "memory": check_memory_usage(threshold=85),
        "replication_lag": check_replication_lag(max_lag_seconds=30)
    }

all_healthy = all(checks.values()) status = "healthy" if all_healthy else "degraded"

return { "status": status, "checks": checks, "timestamp": datetime.utcnow().isoformat() } ```

Real-World Example: Payment Gateway HA

A Bangalore-based payment gateway achieved 99.995% availability:

**Architecture**: Active-active across 2 datacenters with automatic failover.

**Key decisions**: - Database: MySQL Group Replication with automatic primary election - Application: Stateless microservices behind load balancers - Sessions: Redis Sentinel cluster - DNS: Health-checked failover with 60-second TTL

**Failover test results**: - Server failure: Recovery in 3 seconds (load balancer health check) - Database primary failure: Recovery in 15 seconds (automatic election) - Full datacenter failure: Recovery in 90 seconds (DNS failover)

Cost vs. Availability Trade-offs

Higher availability costs more. Here's a framework for decision-making:

Availability Target | Redundancy Level | Cost Multiplier
--------------------+-----------------+----------------
99.9%               | N+1             | 1.5x
99.99%              | N+2, Multi-AZ   | 2.5x
99.999%             | Multi-Region    | 4x+

For Indian businesses, start with 99.9% availability and upgrade based on customer requirements and SLA commitments.

Implementation Checklist

1. Audit current architecture for single points of failure 2. Define availability targets based on business requirements 3. Implement health checks and monitoring before adding redundancy 4. Start with active-passive for simplicity 5. Graduate to active-active as traffic grows 6. Test failover scenarios regularly (chaos engineering) 7. Document runbooks for every failure scenario 8. Review and update architecture quarterly

High availability is a journey. Start with the basics—redundant load balancers and database replicas—then evolve toward multi-region active-active as your business demands it.

Key Takeaways

**Audit for single points of failure first** — one load balancer, one database server, or one network path can negate all other redundancy investments.
**Start with active-passive failover** (via Keepalived or similar) for simplicity, then graduate to active-active with load balancing as traffic and team expertise grow.
**Database HA requires its own strategy** — MySQL Group Replication and Redis Sentinel handle automatic failover, but they add complexity that needs testing before production reliance.
**Health checks must verify dependencies**, not just process existence — a web server returning 200 while the database is down is not actually "healthy."
**Test failover scenarios regularly** — a failover that has never been tested is a failover that will fail in production; schedule quarterly drills for every redundancy layer.

High Availability Architecture Patterns: Building Resilient Systems on Cloud