Broadband capacity - the impact of averaging many subscribers

The fundamental idea of packet networks like the Internet is to transport variable rate data flows over a shared infrastructure.  Individual end points may generate little or no traffic for minutes or hours on end and then suddenly want to send or receive a large data set as quickly as possible.  Packet networks efficiently share common transport facilities among multiple users but how does this sharing actually work?  Yesterday I posted some actual, relatively current data from three small ISPs.  Today, let me show you how averaging traffic from many individual end points produces a completely stable core network flow.

Each of the graphs below shows successive five minutes measurements during a 24 hour period, but the first graph reflects a few hundred subscribers while the final graph represents the average of many, many millions of subscribers.

In the first graph, the blue line reflects outbound traffic from a few hundred subscribers.  While the average is just over 1 Mbps, the peak (at ~1615 hrs) is 6.9 Mbps for a peak-to-average of nearly 7-to-1.  The green bars represent inbound traffic with an average of 5.6 Mbps and a peak of nearly 16.8 Mbps for a peak-to-average of 3-to-1.  Some of this is time-of-day dependency but a lot of it, particularly the 6.91 Mbps spike at 1615 hrs, is the result of averaging only a few hundred subscribers.

 

Here's a traffic measurement at AMS-IX, the largest Internet Exchange in the world.  With an average of nearly 1 Gbps, the peak-to-average is a modest 1.43-to-1 and it's almost entirely due to time of day.  The largest change from one 5 minute measurement to the next appears to be about 140 Mbps so the short term peak-to-average is perhaps 1.1-to-1.

 

Finally, here a measurement of all the traffic flowing through AMS-IX – over 822 Gbps peak.  Here the time-of-day variation remains but there are virtually no statistical fluctuations.  The short term peak-to-average is almost 1-to-1.

 

To put this in economic terms, for an Internet backbone link that runs at many Gbps your daily traffic profile is completely stable and you can guarantee zero packet loss merely by providing 10%-20% extra capacity above your daily peak.  

But if you are running a small ISP, both the capacity you need per subscriber and the extra "headroom" for unanticipated peaks must be substantially larger.  To get a handle on what is required we also need to look at shorter intervals (shorter than five minutes).  But more on that in a subsequent post.

Comments

[Rudolf]

You could have a look at the work done by Aiko Pras, Michiel Mandjes and Remco van de Meent of Twente University on this topic. Van de Meent's Phd thesis was on the topic of link dimensioning. to have a look at some of his work see here http://eprints.eemcs.utwente.nl/6869/01/TR-CTIT-06-56.pdf

[DGentry]

A maxim I heard a long time ago in the IEEE Ethernet standards effort:

"The most effective form of QoS is overprovisioning."

[brough]

Thanks Rudolph, thats a very interesting article, and directly relevant to what I was going to write about next.

[Juliet Bradman]

Thanks you so much for sharing this useful information with us.I think people should know about it.There are many people who don't know these broadband capacity.

[Frankbulk]

I work for a small ISP and see the same thing every day -- the links with a few dozen or hundreds of users is much more varied than our aggregated upstream. Similarly, other ISPs in our bandwidth-buying group who are much smaller have much greater variations in average to peak burst, which gives them the greater benefit in our aggregate purchasing than ourselves, which is more steady. They benefit more because the can meet the burst but are billed only on their average usage.

In summary, the smaller the number of end nodes, the greater the cost because (a) peak bandwidth per subscriber ratio and (b) lower purchase volumes.

[Juliet Waugh]

One thing I would like tell you that I really enjoyed reading your post that was too informative.I think this is a great information about broadband.