The fundamental idea of packet networks like the Internet is to transport variable rate data flows over a shared infrastructure. Individual end points may generate little or no traffic for minutes or hours on end and then suddenly want to send or receive a large data set as quickly as possible. Packet networks efficiently share common transport facilities among multiple users but how does this sharing actually work? Yesterday I posted some actual, relatively current data from three small ISPs. Today, let me show you how averaging traffic from many individual end points produces a completely stable core network flow.
Each of the graphs below shows successive five minutes measurements during a 24 hour period, but the first graph reflects a few hundred subscribers while the final graph represents the average of many, many millions of subscribers.
In the first graph, the blue line reflects outbound traffic from a few hundred subscribers. While the average is just over 1 Mbps, the peak (at ~1615 hrs) is 6.9 Mbps for a peak-to-average of nearly 7-to-1. The green bars represent inbound traffic with an average of 5.6 Mbps and a peak of nearly 16.8 Mbps for a peak-to-average of 3-to-1. Some of this is time-of-day dependency but a lot of it, particularly the 6.91 Mbps spike at 1615 hrs, is the result of averaging only a few hundred subscribers.
Here's a traffic measurement at AMS-IX, the largest Internet Exchange in the world. With an average of nearly 1 Gbps, the peak-to-average is a modest 1.43-to-1 and it's almost entirely due to time of day. The largest change from one 5 minute measurement to the next appears to be about 140 Mbps so the short term peak-to-average is perhaps 1.1-to-1.
Finally, here a measurement of all the traffic flowing through AMS-IX – over 822 Gbps peak. Here the time-of-day variation remains but there are virtually no statistical fluctuations. The short term peak-to-average is almost 1-to-1.
To put this in economic terms, for an Internet backbone link that runs at many Gbps your daily traffic profile is completely stable and you can guarantee zero packet loss merely by providing 10%-20% extra capacity above your daily peak.
But if you are running a small ISP, both the capacity you need per subscriber and the extra "headroom" for unanticipated peaks must be substantially larger. To get a handle on what is required we also need to look at shorter intervals (shorter than five minutes). But more on that in a subsequent post.