Why there's no Internet QoS and likely never will be

I thought savvy networking folks understood why IP QoS hasn't been widely deployed in the Internet, but a recent discussion surprised me.  Then I looked at the Wikipedia article on Quality of Service and was further surprised. 

The Wikipedia article describes what QoS is, how it works and the applications that could benefit from QoS, but says almost nothing about problems or issues with IP QoS and doesn't discuss why QoS has never been deployed in the Internet backbone.  The only item in the Wikipedia article even touching such issues is a reference to an Internet2 study by Benjamin Teitelbaum and Stanislav Shalunov.  But that reference didn't mention its title, Why Premium IP Service Has Not Deployed (and Probably Never Will), and, until I corrected it, gave an invalid link.  I'm not quite organized to write appropriate paragraphs for Wikipedia, but let me set out some information and some references.

References on QoS Issues

The 2002 paper by Teitelbaum and Shalunov, referenced above, highlights the difficulty of incremental deployment and some of the serious economic challenges associated with QoS.  A 2003 essay by Dan Bricklin, Why We Don't Need QOS: Trains, Cars, and Internet Quality of Service, explains the roots of those economic challenges. [I did add this reference to the Wikipedia article.]

Beyond these two general papers, let me suggest specific reasons why VoIP users and multi-player gamers may benefit from specific QoS measures at their end of a broadband access link, but are unlikely to pay for QoS from their ISPs and any kind of QoS in the Internet backbone.

Internet Core

Once you get beyond the access network, every link in the Internet – local, regional, national or international – is carrying multiplexed traffic from many, many users.  Multiplexing many bursty flows results in relatively predictable traffic flows on such links. Traffic volumes vary by time of day, but these links don't saturate, except as a result of poor engineering or forecasting on the part of the ISP or failures in other parts of the network causing rerouted traffic.  Either case generates a rapid response from any ISP that expects to remain in business.

In short, except at the edges of the access network, Internet links may be heavily loaded but are not saturated.  This, combined with relatively high capacities, means typical delay variations are fractions of a millisecond and packet loss is negligible, i.e. best effort is good enough even for low latency applications like voice telephony. Except during major failures, the effect of QoS in the Internet backbone is negligible.

Consumers aren't going to pay a premium for a performance improvements they can't see! They might be induced to pay for a brand (after all people pay premium prices for branded water), but as yet no such brand has emerged.  And if one does, consumers will be paying for that brand, not for QoS technology per se.

Broadband Access Links

There is one place in the public Internet where limited, highly specific QoS measures make sense and are being deployed – at the consumer end of an asymmetric broadband access link.  Typical residential DSL connections offer a few megabits per second (or less) to the home, but only a few hundred Kbps from the home to the Internet.  Unlike links between core routers, traffic on such access links is highly bursty and bursts can saturate the link.  If there are no active peer-to-peer applications then, most of the time, little or no traffic flows on most residential broadband connections.  But suddenly someone sends an email with an attached file or photo.  This saturates the outgoing link for many seconds while several megabytes of data go out at perhaps 250 kilobits (~31 kilobytes) per second. 

Typical residential access links are narrow pipes where the cost of purchasing more capacity is prohibitive, if it's possible at all.  On the other hand, we can control the routing policy for what goes out on the link by deploying an appropriate residential router at our end of the broadband access link.  As a first approximation, one would like to give priority to VoIP packets (and gamers may want to give priority to specific multi-player games).  Simple priority is a good first step, but may not be enough on slow links.

Slow links have an added problem due to large packet serialization delay.  VoIP packets are typically less that 150 bytes, but a web page or an email is delivered in 1528 byte chunks.  At 250 Kbps, a 1528 byte packet takes ~50 milliseconds to pass over the link.  If a VoIP packet arrives just after a 1528 byte packet has started, it doesn't matter that the VoIP packet is the highest priority to be sent next, it will have to wait for the current packet to complete.  Intermittent 50 ms delays are handled by a jitter buffer at the other end of the VoIP connection, but only at the expense of an additional 50 ms of delay.  If the uplink is slower than 250 Kbps, serialization delays are longer.

Actual situations can be worse than this.  I made a series of measurements with Comcast cable modem service from my house in Massachusetts to MIT's web server (in May 2005 when I still had Comcast service).  Comcast New England peers with MIT, so a traceroute is very short (8 hops) and entirely within Massachusetts.  With nothing active in my house, ping times to MIT average 15ms with a peak of 33ms.  However, while sending a large email (one which took more than 100 seconds), ping times were fairly evenly distributed between 13ms and 349 ms – hardly what wants for a VoIP connection.

Luckily this is one place where priorities work and can be imposed.  Indeed most consumer VoIP devices incorporate at least simple priority and some include the ability to fragment large packets (so as to reduce serialization delays).  And, because it's so useful for VoIP and for gaming, this functionality is showing up in popular residential routers, e.g. from Linksys, Netgear etc.

Branding might command a premium, but Internet QoS never will

So, while individuals can benefit from simple priority queuing at their end of a broadband access link, they are not going to pay for benefits they can't see, i.e. QoS technology.  Operators interested in premium services should focus on branding and perhaps on facilitating simple priority queuing in the access network.

Comments

[Anonymous]

See also David Isenberg's original article about: "The rise of the stupid network".

[brough]

Actually, while David Isenberg's 1997 paper, The Rise of the Stupid Network at http://www.rageboy.com/stupidnet.html, was seminal, it did assume that some QoS would be built into the Internet. I quote

"... in the Stupid Network, because the data is the boss, it can tell the network, in real time, what kind of service it needs. And the Stupid Network would have a small repertoire of idiot-savant behaviors to treat different data types appropriately. If the data identified itself as financial data, the Stupid Network would deliver it accurately, no matter how many milliseconds of delay the error checking would take. If the data were two-way voice or video, the Stupid Network would provide low delay, even at the price of an occasional flipped bit."

David is a very sharp person, but in 1997 the only published concept I'm aware of that might have applied to QoS was the original end-to-end paper of J.H. Saltzer, D.P. Reed and D.D. Clark, End-to-End Arguments in System Design at http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf . This paper focused on reliable transimssion, but presented a general concept that suggests why multi-cast, QoS and other complexity in the network is probably a mistake.

[Chat]

theme site lince looks nice