This morning's mail included a query from a reader who'd been looking at my April 2009 post on Google's Peering and Caching Strategy. His question: Are ISPs allowed to transparently cache content? Here's my answer:The simple answer is yes. Caching is not only allowed, it is an essential part of some Internet functions (for example DNS, see below). For a more complete answer, we must ask "allowed" by whom? At a minimum, there are technical, legal and commercial answers.
1. From a technical perspective, there are three reasons for caching. It reduces latency, it reduces traffic on upstream connections and it distributes load thus reducing load on originating servers. Today, latency reduction primarily benefits individuals browsing the web, while traffic reduction reduces upstream costs - an advantage for the ISP that may or may not be passed on to the consumer. But reducing latency and traffic have been important goals from the beginning of the Internet and both considerations have influenced protocol design along the way. For example, the Domain Name System (RFCs 882/883 superceded by 1034/1035) makes extensive use of caching to improve performance and reduce load on the root servers. And many compression protocols, like ROHC (RFC 3095), work by maintaining local state at the receiving end - effectively a mini-cache of earlier header fields.
However, I assume you are focused on web caching. In the specific case of web caching, some relevant technical documents are RFCs 2186 and 2187.
2. From a legal perspective, some copyright proponents initially argued that caching violates copyrights. I am not aware of anywhere where such arguments have held up. In the US, the 1998 Digital Copyright Millennium Act specifically exempts ISPs from copyright liability for caching web content.
3. From a commercial perspective, consumers of Internet connectivity (individuals, enterprises, government agencies, etc.) typically seek minimum latency access to any Internet endpoint using any legitimate protocol. In a competitive market, i.e. one with many ISPs, "legitimate" protocol means just about anything built on IPv4. In the future, that market definition may evolve to include just about anything built on IPv6. Of course, many people use only a subset of full Internet connectivity and thus choose an Internet access service that gives little more than http and email but costs less.
Since latency and cost are key in any competitive market, web caching is widely deployed by ISPs. What's more, specialized content distribution services like Akamai have emerged to further reduce latency for specific kinds of content that might not be well handled by conventional caches. The interfaces to a service like Akamai's are based on Internet standards, but the internal workings are Akamai-specific. These sorts of services are not only "allowed," they have become a competitive requirement for many distributors of Internet content.