Understanding Network Latency, Bandwidth and Throughput

Network performance is described by three key metrics: latency, bandwidth, and throughput. They measure different aspects of network behavior and affect application performance in distinct ways.

Latency

Latency is the time it takes for a packet to travel from source to destination, measured in milliseconds (ms). It represents the delay in the network.

Factors affecting latency: - Physical distance (speed of light through fiber) - Number of hops (routers) between source and destination - Processing time at each hop - Queuing delays during congestion - Protocol overhead (TCP handshake adds latency)

Typical latency values: - Same data center: < 1 ms - Same country: 10 to 30 ms - Cross-continent: 50 to 150 ms - Around the world: 200 to 350 ms

For interactive applications (web browsing, API calls), latency directly affects user experience. Each HTTP request requires at least one round trip, so a 100 ms latency adds 100 ms to every request.

Bandwidth

Bandwidth is the maximum data transfer rate of a network link, measured in bits per second (bps). It represents the capacity of the connection.

Bandwidth is determined by: - The physical medium (fiber, copper, wireless) - The network equipment (switches, routers) - The provisioned service level

Common bandwidth levels: - 1 Gbps: standard server connection - 10 Gbps: high-performance server connection - 100 Gbps: backbone network links

Throughput

Throughput is the actual data transfer rate achieved, measured in bits per second. It is always less than or equal to bandwidth.

Throughput is affected by: - Available bandwidth - Network congestion - Latency (especially for TCP, where throughput is limited by window size / latency) - Packet loss and retransmissions - Protocol overhead

The Relationship

A useful analogy: bandwidth is the width of a highway, latency is the speed limit, and throughput is the actual traffic flow.

For TCP connections, throughput is approximately limited by: TCP Window Size / Latency. A connection with 64 KB window size and 100 ms latency achieves at most 64 KB / 0.1 s = 640 KB/s, even if bandwidth is much higher.

TCP window scaling (enabled by default on modern systems) increases the window size to mitigate this limitation.

Impact on Applications

Web applications: high latency increases page load time. Each HTTP request requires a round trip, and TLS requires additional round trips for the handshake.

Database connections: latency between the application server and database adds to every query. Co-locating application and database servers in the same data center minimizes this.

File transfers: throughput determines how quickly large files can be transferred. High bandwidth with high latency may not achieve full throughput without TCP window scaling.

Measurement

Latency: ping target-host (measures round-trip time) Bandwidth: iperf3 (measures maximum throughput between two hosts) Throughput: real-world transfer tests or monitoring tools

Optimizing Network Performance

• Minimize latency by choosing geographically close servers
• Use CDNs for static content to reduce user-to-server latency
• Ensure TCP window scaling is enabled
• Monitor throughput vs bandwidth to detect congestion
• Use connection pooling to reduce the impact of latency on database queries

Understanding these three metrics and their relationships helps you make informed infrastructure decisions and diagnose performance issues. ## Key Takeaways

• Latency is the time for a packet to travel between two points; bandwidth is the maximum capacity of the link; throughput is the actual achieved transfer rate
• For TCP connections, throughput is bounded by TCP window size divided by latency — high latency can prevent you from using available bandwidth without window scaling
• Minimize latency by placing application and database servers in the same data center and using CDNs for static content closer to users
• Measure with ping (latency), iperf3 (bandwidth), and real-world transfer tests (throughput), then monitor these metrics over time to detect degradation early