Communications

eComm agenda is on-line ―Check out the speakers!

There's a fascinating agenda building for eComm in Amsterdam in October.  Here's the latest list of talks, but what's best is the set of speakers who are giving these talks.  I personally know quite a few and know of many more (who I look forward to meeting).  The group includes a preponderance of innovators -- new views on what's happening, new ideas for how to drive change.  If you don't read through this list of talks, look at the speakers list here.  Then come to Amsterdam and meet these people.

21st Century Economics: Lessons for Telcos - Umair Haque (Havas Media Lab)

Advances In Spectrum Transparency, Software Defined Radio/Cognitive Radio - Darrin M Mylet (Spectru-Station)

Almost all Marketing & Product Management of Telco Services is Wrong - Rudolf van der Berg (Logica)

Current Trends in Community Wireless Networks and Beyond - Aaron Kaplan (FunkFeuer)

Death of the Handset: Evolving from Mobile Devices to the Mobile Digital Life - Mark Rolston (frog Design)

Edge As Value - Value As Edge - Graham Brierton (Voicesage)

Enslaving Humans using Communications Technology for Fun and Profit - Thomas McCarthy-Howe (Jaduka)

Entrepreneurial Advantages with New Open-Source Technologies - Jay Phillips (Voxeo)

European Telecoms 2015: Silent Death or Generative Bazaar? - Julien Salanave (IDATE Telecoms)

Finding Disruption - Michael Jackson (Mangrove Capital)

Goodbye Minutes, Hello Moments - Martin Geddes (BT)

Google Wave - David Wang (Google)

How the "Internet of Things" will Change the Way we Connect - Alexandra Deschamps-Sonsino (Tinker.it)

How to get More Value out of Customer Interactions by Blending Online with Voice - Stefan Hopmann (Swisscom)

Humans as a Service: Abstracting Communications to Reach New Applications - Todd Landry (NEC Sphere)

Is LTE being Held Hostage by Ordinary Voice Telephony? - Dean Bubley (Disruptive Analysis)

Lifestyle Segmentation from Carrier Location and Call Data - Greg Skibiski (Sense Networks)

Lifestyle Segmentation from Carrier Location and Call Data - Tony Jebara (Sense Networks)

LTE - Long Term Employment or Less Than Expected? - Moray Rumney (Agilent)

Open Access Makes Economic Sense - Benoît Felten (Yankee Group)

Open Screen Project: Next Generation Contextual Applications - Andrew Shorten (Adobe)

Opening of the Terahertz Region - Robert Horvitz (Open Spectrum Foundation)

Opportunities in Post-Telecom Connectivity - Bob Frankston (Frankston Innovating)

Post Financial Trauma - How is the Telecom Value Chain Now Positioned? - James Enck (mCapital)

Redefining Gifts in the Digital Space - Katie Lips (Little World Gifts)

Slowing Down Communication: Designs Inspired by Quality, Intimacy, and Humanity - Stefan Agamanolis (Distance Lab)

Spectrum 2.0 – What's Really Happening? - Brough Turner (Ashtonbrooke)

Stealth Approaches to Legislating Open Spectrum - Brough Turner (Ashtonbrooke)

Technology and Biological Evolution: What This Means for Media and Communications Technologies - Tomas Rawlings (University of the West of England)

Telemedia Futures - Gerd Leonhard (MediaFuturist.com)

The Emerging Telecology of Social Networks and the Status Update - Stuart Henshall (Phweet)

The Future of Interconnection - Rudolf van der Berg (Logica)

The Global Battle for Communications Justice: An Open Spectrum Manifesto - Sascha Meinrath (New America Foundation)

The Next Wave of Communications Applications - Cullen Jennings (Cisco)

The Practical Edge of Speech Technology - Moshe Yudkowsky (Disaggregate)

Ubiquitous Voice over Broadband - is There a Future Role for the Smart Pipe? - Martin Taylor (MetaSwitch)

Unlicensed Spectrum: Future Regulation - Prof. William Webb (Ofcom)

Video Killed the Telephone Czar? - Jan Linden (Global IP Solutions)

When Will HD Voice Become a Reality? - Martyn Davies (Dialogic)

What Would Telephony be Like if we Designed it Today? - Matt Ranney (RebelVox)

If you've read this far, there's a 10% discount via this registration link - register here.  And for more information, visit http://eComm.ec.  This one conference is at the top of my list for:  interesting, productive and an excellent set of contacts.

Education due for radical disruption

The Internet has disrupted music publishing, encyclopedias, classified advertising, news and is poised to disrupt both television and the movies.  What else?

Education is overdue, but most of today's academics still don't get it.

For decades, even centuries, we've seen increased productivity in agriculture, in industry and most recently in services.  Productivity gains show up more useful output per person employed.  One place where this has not happened is education, as the graphs below illustrate.  Indeed, in education, schools compete on the basis of lower student faculty ratios, i.e. lower faculty productivity.

Individuals vary in how they best learn things but, in most cases, direct interaction with a teacher is a significant benefit.  So if the Internet is just about better access to information or remote video interactions with teachers, it may be wonderful, but it's hardly disruptive. What's needed is some way to multiply the number of teachers, dramatically.

A suggestion of how this might come about, i.e of how the Internet might disrupt the traditional academic process, comes from italki.com.  To quote Gang Lu,

italki is a language social network providing free language learning content. Unlike the traditional one-on-one education approach, italki is building an online community where any user can play two roles, tutor and learner. In italki, users can find language partners, post their foreign language questions which can be answered by other users, and join groups for language learning. The new version introduced a new feature called knowledge which is basically a Wiki service. Now users can not only share the language learning materials they uploaded but also collaborate on creating free language learning textbooks by their own. So italki is building probably the largest Wiki for language learning content!

This is not some academics opening their university course materials to the world (although I'm sure that's useful).  These are individuals contributing course materials and, more importantly, tutoring each other.  This approach tackles the hard problem, the student teacher ratio.

I'm sure there are other examples that I haven't stumbled on as yet.  In any event, it will be interesting to see how this sector evolves.  italki.com started in Shanghai and is already dealing with 16 languages and has global participation.

SIP revolution, massively delayed — but there's hope

The SIP Center asked for an article which I finally wrote the weekend before last.  My article was actually rather negative, but they published it anyway.  Now I'm feeling a little guilty as there is an optimistic note I could have used as my conclusion.  So let me try again...

First let me summarize my problem.  When SIP emerged in 1996, it's support for direct connections from one user to another was extremely compelling.  This was the VoIP protocol which would lead to a complete revolution in communications.  Yes, you might refer to a directory service, but you wouldn't need an operator to make a phone call.  You could do it yourself, directly.  Unfortunately, that revolution never happened.

So far, no revolution

The biggest change in telecommunications in the past 12 years has been the global deployment of three billion mobile phones, all based on conventional circuit-switching and Intelligent Network technology — nothing to do with SIP. And arguably, the most interesting telephony service enhancement, after mobility, came from Skype with its seamless integration of presence, instant messaging, wideband audio and video. But Skype is based on proprietary protocols, not SIP. Finally, VoIP technology has helped drive down the cost of international calling, but using MGCP, H.248 &/or H.323 protocols much more than SIP, at least so far.

SIP has been adopted by PBX manufacturers in recent years, but this doesn’t seem to have changed business practices at all. The IT department still buys the PBX and the telephone sets from a single vendor and then contracts with a service provider to handle calls outside the enterprise.

And then there's IMS

SIP has been adopted for use in the IP Multimedia Subsystem (IMS), but this completely warps the original SIP vision.  IMS is a centralized system — a next generation network for mobile and fixed operators.  It's the complete opposite of the original vision for SIP.

Why have things gone so far astray?

SIP assumed an end-to-end Internet

SIP assumes it's possible to make end-to-end connections over the Internet and therefore a SIP session can know about and use globally valid IP addresses.  That was a naive assumption, even in 1996-1999 when SIP was being defined.  The real Internet contains firewalls, network address translators (NATs) and other "middle boxes."  They are not going away, it's only getting worse over time.  Today, applications must be aware of and able to work around middle boxes and other network problems. 

Many middlebox issues can be overcome with the help of client software and central servers implementing Interactive Connectivity Establishment (ICE), a recently completed IETF proposed standard that in turn relies on STUN, TURN and/or RSIP.  A continuing obstacle for direct user-to-user connections is the need for central servers for STUN, etc..

So it there no chance for the original SIP vision of direct user-to-user communication?

P2PSIP — a reason for optimism

Actually, there is some reason for optimism.  The advent and widespread adoption of Skype showed what was possible and suggested how one might distribute central services among peers, potentially avoiding the need for an explicit service provider.  The past few years have seen rising interest in peer-to-peer SIP which has resulted in an IETF working group under the name p2psip.  Their goal is "to leverage the distributed nature of P2P to allow for distributed resource discovery in a SIP network, eliminating (or at least reducing) the need for centralized servers."

Assuming this is completed (during 2008 & 2009), we'll have the elements with which one could make a SIP-based open peer-to-peer communications system.  It will be interesting to see actual software implementing the ideas of the p2psip group.  We may yet see a revolution!

How tiered Internet pricing could actually facilitate P2P

Time Warner Cable's planned experiment with tiered charging for Internet access has generated a flurry of coverage in the blogsphere, but no new insights (at least that I've seen).

The primary problem ISP's complain about is that 5% of their customers use 90% of the available bandwidth and when they examine this traffic, it's mostly peer-to-peer file sharing.  A reasonable question is how to allow as much of this traffic as possible without increasing an ISP's variable costs or slowing down their other users.

This may not be as difficult as it appears.   Indeed if Internet access was as competitive as mobile telephony, we might already have seen what I'm about to propose — a combination of bundled pricing equivalent to mobile's "free nights and weekends" and "free on-net calls" with a way to facilitate P2P traffic that leverages exactly these "free" periods.

An ISP's costs

ISPs have some costs which are relatively fixed and others that are tied to usage.  A network is a relatively fixed cost and when it's not full, the incremental cost of adding traffic is zero!  This is the reason mobile operators give away free nights and weekend.  They've built their mobile network for the peak daytime traffic, so it costs them nothing to run promotions that add incremental traffic at off hours.  Peak hours and off hours may be different for an ISP, but the concept is the same. When a data pipe is lightly loaded the ISP's cost of adding incremental traffic is zero.

On the other hand, some ISP costs are usage based, for example "IP Transit" or more properly, Internet Transit.  This is the ISP's upstream cost to send and receive traffic to/from the rest of the Internet.  However, even here, usage-based costs occur at heavy usage.  Light usage periods don't save money.  To understand what's happening, it's worth a digression on Internet Transit.

Internet Transit

Internet access is monopoly or duopoly or a heavily regulated industry.  The middle mile connections from the local network to the Internet backbone may or may not be competitive depending on where you are.  But the Internet backbone itself is extremely competitive.  If you can get to a major Internet Exchange Point in the US or Europe, there are many providers offering extremely competitive rates for Internet Transit.  Typically these services are priced on a megabit per second per month basis (Mbit/s/Month) with lower rates for higher volume commitments.  The other key idea is that charges are based on the 95th percentile of all the five minute data rate samples taken during the month.  So an ISP can have a few bursts above their typical rate, as long as they represent less than 5% of the sampled intervals.

But this also means there is no extra cost to run at or near the typical rate at all times.

Local traffic

Even more important, if file sharing is done with other computers on the same ISP's network, then there is no need to pay for Internet Transit at all.  The question is how to figure out which potential peers are "on-net" and which are "off-net."

Sending signals to P2P software

Most P2P file sharing software has relatively little knowledge of locality.  Some P2P software practices "prefix awareness," for example, Joost gives preference to peers in the same /24 IP address block when they are available.  But if a major operator provided an automatic way for P2P client software to determine whether a prospective peer's IP address was currently reachable "for free", it seems likely the file sharing community would leap on it, and if there's money to be saved, active file sharers would download the new clients immediately.

A standard way to present such information might be via an extension to the XML-based response codes in one of the whois information exchange proposals, e.g. from ICANN or from APNIC.  Also, while what I'm proposing might start as a pricing plan rather like a mobile operator's "free nights and weekends" and "free on-net calling," it's not hard to see extensions where an ISP could offer dynamic access to underused capacity to those programs that were prepared regularly interrogate an ISP's server and use just the advertised off-hours capacity.

In closing

People liked fixed price deals.  Unlimited is great, but there's plenty of experience with bundles of minutes and the idea of data bundles has already showed up in 3G mobile data plans.  The combination of several tiered data bundle prices with the availability of "free" connectivity for "on-net" peers and during off peak intervals is likely to appeal to file sharers and produce better results for both the sponsoring ISPs and file sharers alike.

Joost entering world's most advanced P2P TV market, i.e. China

Gang Lu reports rather specific rumors that Joost is going to enter the China market on or around the Chinese New Year (Feb 7th) by partnering with the Chinese portal Tom.com.  That's not surprising as Skype partnered with Tom.com when they entered the Chinese market.  What's different is Skype was innovative everywhere in the world. 

Joost will have to play catch up in China, as the Chinese are the world leaders in P2P TV and P2P streaming media.  Well established Chinese firms like PPLive and PPStream pre-date Joost by nearly two years (see my earlier comments).  And today, in China, the P2P market is clogged with many more players like UUSee, Vakaka and Vatata.

Early entrants like PPLive focused on live TV in 4:3 ratio with simple user controls, perhaps for those familiar with TV but not with TIVO, however this is changing rapidly.  What's more, friends report performance has been excellent, even in early 2005.

Joost has not worked so well within China.  From one friend and from Google translations of Chinese reviews, it appears there are many places in China where the bandwidth requirements of Joost cause performance problems, even while PPLive works well.  Presumably this will be cured by local Joost support within China. In other words, I suspect a lack of local peers within China means, for now, too much Joost content must flow over clogged international links.  A local presence should cure that.

Meanwhile, Gang Lu describes a hybrid system (streaming servers and P2P bandwidth sharing) from Vatata which:

"... supports most of the video formats, including Microsoft, Real, Flash, Apple, MPEG1/2/4, OGG/MKV etc and H.264. Vatata system consists of two sub-system: Vata, the back-end streaming platform and Tata the front-end player. Tata is absolutely fascinating. It supports On Screen Display (OSD) and allows plugins, which means you can run multiple modules (e.g. instant-messenger, channel list, etc) on top of the video screen, which just sounds like what Joost does."

So it's clear Joost is moving into a very advanced market.  It will be interesting to see the resulting cross fertilization.

Emerging Communications Conference 2008

I'll in California quite a bit in March and April, but the highlight is my first week, when I'll be speaking at a new conference, eComm 2008, March 12-14.  While the conference in new, the community is established and fascinating.  eComm 2008 being put together by Lee Dryburgh, who was on the program committee for O'Reilly's eTel conferences.  When O'Reilly cancelled eTel 2008, Lee took the initiative to keep that incredible community alive.  He was soon joined by many others.

Click through the logo at the left for conference info.  Right now there's a board of advisors, an incredible list of speakers with more on the way, a wiki and a Facebook group with 170 friends!

The first thing I look for in a conference is interesting people, then new ideas.  eComm promises an abundance of each.  The focus is next generation personal communications and the schedule is set up for rapid fire delivery inlcuding many 5 minute and 15 minute sessions.  As far as new ideas goes, this will be a fire hose!

*** Correction: 12/21 ***

The conference is being held in the Computer History Museum in Mountain View.  This easily beats the typical conference facility, but it means there are only 300 paid admissions available.  Registration has opened, here.  If you register before the end of 2007, the $1495 registration fee is marked down to $1195.

I look forward to seeing you there.

Managed Storage Futures

Recently I wrote about differences in the exponential growth rates of computing and networking and promised to say more about how these differences cause substantial shifts in the technology landscape.  Managed storage is one example.  The relevant doubling rates (from that earlier post) are:

Doubling Rates

Technology Measure	Months
Computing performance	18
Storage capacity	12
Networking performance	15
Access connectivity	20-26

The increase in storage capacity per dollar has been phenomenal and is one of the reasons that Google can offer Gbytes of free storage for email and that Amazon can offer their Simple Storage Service (Amazon S3) at extremely low rates.

But it's also caused headaches for IT directors, as installed equipment becomes obsolete long before it's fully depreciated, and employees and department heads grip about inflated internal billing rates for storage.  Pity the IT staffer who sends a broadcast message justifying corporate email storage limits because "it costs the company X cents per megabyte per month."  I've seen such messages, and the employee ridicule they engender.

This sounds like a perfect opportunity for a managed service — provide an interface that looks a storage area network or network attached storage, using multiple (for reliability and arbitrage) Internet-based storage services to provide the actual storage.  But now differential growth rates become a factor.

Storage costs decline a bit more rapidly than the cost of Internet transit.  So, already it's the case that network-based storage is extremely low cost for backup but less affordable for transactions.

But the real problem is the cost of access connectivity.  If you're selling managed services to IT departments, you need to provide services at their premises.  Local connectivity is not fast, cheap or reliable, and the pace at which it improves is glacial in comparison with storage or Internet transit.

Has this prevented the emergence of managed storage solutions?  Of course not.  But most existing solutions focus on remotely managing equipment that's physically on the enterprise premises.

Is there opportunity for network-based managed services.  Also, yes.  But you will need considerable focus on local connectivity, both for the numbers you use in your business plan and for the specifics of how you implement the service.  Some thoughts:  interface your managed service via a remotely managed on-premise box that includes caching?  use a dedicated access link to guarantee QoS?  ???

In any event, three years from now, you can count of disk storage being ~8X more affordable, Internet transit being perhaps 4x more affordable, but local connectivity only 2x or 2.5x.  Don't give up your great  business idea, but plan accordingly.

Availability -- more than presence and a nice implementation to boot

I’ve never liked the term presence or the way the function is implemented in instant messaging systems.  I want to indicate my availability — something that, at any given moment, may be different for my wife, my co-workers or my friends in the blogsphere.  And, if I check my PC for messages at 6am, just before walking the dog, that doesn’t mean I’m planning to respond to those messages or accept calls or chats at that moment — my dog is desparate and she’s letting me know it!

Now there’s a new kid on the block, EnThinnai, that’s launched the beta of an information sharing site featuring privacy and control.  They also include a concept of availability that looks very much as I desire.

In addition, they’ve done a peer-to-peer implementation with a choice of query (you only ask when you’re interested in knowing my availability) or subscribe (you want to be notified when I transition to a specific state).  This makes a lot more sense to me than a central server farm monitoring everything I do and continuously broadcasting it to people who only contact me once or twice a year.

Aswath Rao has more info at the EnThinnai blog.

 

Architecturally induced denial of service susceptability

Fundamental design decisions have a big impact on how a network behaves under stress.  I just noticed two otherwise unrelated posts both touching on network design issues that result in (or protect against) network collapse under stress.

Yesterday, I wrote about August's Skype outage where the system took over 36 hours to recover from a collapse.  The issue was too many clients attempting to reconnect at once.  Instead of clients “backing off” when supernodes weren’t immediately available, they kept hammering away, trying to log in.

Last week I wrote about the ways the Internet and the Telecoms networks respond to congestion in the backbone.  In December 2006 an earthquake off the coast of Taiwan broke multiple fiber cables causing a congestion collapse on the Internet backbone in significant parts of Asia.  The issue was too many computers attempting to reestablish TCP sessions with the result there was no capacity left for any session to actually send data.

In both cases, there were externally induced problems but, rather than recovering, there was the equivalent of a denial of service attack, self inflicted!

Both of these failures have a fairly simple solution, at least architecturally.  Under conditions of severe overload, the system must be able to restrict new attempts (new TCP sessions, new Skype logins, etc.) to some small percentage of the available capacity.  This allows the rest of the capacity to serve the logins, sessions or calls that do get through, with the result that what capacity remains is put to good use.

As I commented last week, this is one place where the telecoms industry has the correct architecture.  When disaster strikes subjecting some part of the network to overload, it's easy to restrict new call attempts on trunks into the congested area, for example by call gapping. This limits the amount of new traffic to that which the network can handle.  Thus, if only 30% capacity is available, at least the network handles 30% of the calls, not 3% or zero.

Here's one place where network architects can learn from established telecoms practice.

Skype outage: post mortem evolves

Since my last summary, some new information has emerged.  Gerry Blackwell interviewed Skype's Director of Operations, Michael Jackson, got some comment from Martin Geddes, and wrote an interesting article.

I've also had an email exchange with Julian Cain, which I reprint below, and a brief exchange with Philippe Biondi of SecDev.org who referred my to an interesting blog post (in French) by his colleague Cédric Blancher at  EADS France, Innovation Works (Suresnes).

New information from Michael Jackson (in Gerry Blackwell's article) includes the idea of five supernodes per cell (of 300 users):

Each supernode handles about 300 nearby users. Skype configures five in each cell for redundancy. So with upwards of nine million users online, it takes something like 150,000 supernodes to make Skype work.

The triggering event is still attributed to massive computer reboots after Microsoft's Patch Tuesday.  Everything else in the article is consistent with the best earlier explanations, including Julian's which I summarized here, i.e. as Blackwell puts it:

... the real culprit, Skype now says—was a resource allocation algorithm in the client software that could not adapt to such a set of circumstances. Instead of clients “backing off” on their attempts to validate on the network when supernodes weren’t immediately available and waiting for the ship to right itself, they kept hammering away, trying to log in.

And the solution, having clients back off when supernodes aren't immediately responsive, is obvious.  What's left to understand?

1.  I still haven't seen a plausible explanation of why Microsoft's "Patch Tuesday" resulted in problems on Thursday morning, only a lot of questions.  If the problem was induced by massive reboots, why didn't it happen on Wednesday morning?

2.  I still haven't seen a reasonable discussion of scaling.  As I wrote back in August,

I wonder, apart from the login server cluster as a single point of failure, is there also a scaling issue?  FastTrack's breakthrough was the use of supernodes to make the system more scalable.  But was that just one layer of scalability?  If so, what happens when there are 300 million on-line users and one million supernodes?  Perhaps Julian (or another P2P expert) could comment...

Indeed I emailed Julian about scaling and also about Joost.  This was his reply.
On 09/05/2007 05:06 PM, Julian Cain wrote:

Brough,

Skype and Joost are utterly different however Skype is more like Fasttrack(Kazaa). Joosts' Network architecture is mainly "Centralized", they have their own server farm of Supernodes as well as Authentication and Jabber servers. The nature of Joost is less dependent on peer to peer routing as it's basis is tuned towards QoS. Joost peers route traffic and relay UDP based payload as media data streams as well as keep a small cache of what they have recently viewed, however currently every Joost peer is directly connected back to the Joost home servers unlike Skype and Kazaa where once authentication occurs it's "out of our hands".

I agree on the extent of Skype scalability being very limited because of the nature of the Supernodes. At any one time the Supernodes hold ~300-500 child nodes and maintain an "Overlay" network which consists of another several hundred Supernode to Supernode connections. Ie* The Supernode network is very dense in order to provide for best means routing of least cost, however the flaw in this architecture is where the "Overlay" network reaches a capacity and is unable to reliably route traffic. I do not currently have any statistics on how the "Overlay" layer is not scalable but as more Supernodes arise the management of the "Decentralized Data Store" becomes a very hard task as well as keeping this "Overlay" in one single "in sync" network. This was proven with the Skype outage as the network was "trying to heal" it had to start from many 10s of thousands of "Overlay" networks which very slowly were able to sync again as a "single" network however is still an issue today with presence.

For the current Skype "Overlay" network to scale indefinitely while maintaining  a "Single" network infrastructure it needs in place an organizational hierarchy of Supernodes and a level of Service for each of these Supernodes. *Ie. If Skype Supernodes worked in a way such as in Fasttrack then when the network reached 100 million users it would began to crawl. This is due to the dense nature of the upper "Overlay". I can only assume that Skype has thought of this and that when the Supernode ratio is beginning to "bottle neck" then there would be some ordered Hierarchy as to what role each Supernode was playing, otherwise the more Supernodes the more dense the "Overlay" the more the data is relayed back and forth before considering the "Supernode Overlay" into it's own Denial of Service attack.

I hope this helps to some degree, let me know if you have any other questions.

~Julian

So to the extent I have time to look into P2P technology further, I plan to explore what's been written about hierarchy in P2P networks.  Here are some references (which I've found but have not read as yet):

RFC 4981 on Survey of Research towards Robust Peer-to-Peer Networks: Search Methods

Hierarchical Peer-to-peer Systems by L. Garces-Erice, E.W. Biersack, P.A. Felber, K.W. Ross, and G. Urvoy-Keller.

An efficient peer-to-peer file sharing exploiting hierarchy and asymmetry, by G. Kwon and K. D. Ryu in the Proceedings of the 2003 Symposium on Applications and the Internet, 27-31 Jan. 2003 Page(s): 226 - 233.
< unfortunately only available on an IEEE pay-for site >

Best Skype outage post mortem so far

It's been more than ten days since the global Skype outage – time to reconsider what actually happened.  The most credible analysis is not from Skype, but from Julian Cain in a series of comments (here, here and here) that he made to a Gigom article about the outage (or see the single file in "References" below).  Julian is lead architect at Pando and, earlier, was head of Mac development for Kazaa at Sharmen Networks.  So he knows a lot about peer-to-peer networks and his work at Sharmen put him in a position to know quite a bit about the P2P technology that's also used by Skype (and likely by Joost).

Background

Skype's P2P technology was evolved from FastTrack, originally developed for Kazaa.  Their P2P network consists of clients and supernodes.  Skype distributes client software which includes all necessary supernode software, so any client that has appropriate capacity and connectivity can be promoted to become a supernode.  Supernodes dynamically link to other supernodes to support a distributed database and distributed index (called the distributed hash table or DHT).  For Skype, the DHT layer is responsible for maintaining client presence info, contacts and icons/avatars, and handling call routing.

Root Cause

But as I pointed out in several posts during the outage, there's also a centralized component to the Skype network.  That's the login servers.  Julian refers to them at the "authentication servers" and/or "login/connectivity servers."  They are implemented as one cluster of about 50 machines.  As for the root cause of the outage, he asserts:

Skype employees introduced code into the "login/connectivity" server farm that was not compatible with current Skype clients.

Other Issues

While that was the root cause, it was helped along by other network characteristics, notably that each client connects to only one supernode at a time.  According to Julian, there are 300+ clients per supernode and if a supernode goes off line, the 300 or so clients connected to it must reenter their "connecting" sequence, i.e., find and connect to another supernode.

A network with 8 million on-line users implies ~27K supernodes, a figure that's consistent with the ~20K supernodes estimated by Desclaux and Kortchinsky in 2005-2006 (see their June 2006 Recon presentation, PDF here).  The other point from measurements by Desclaux and Kortchinsky is that each supernode attempts to maintain a list of all other supernodes which means there is a substantial amount of traffic between supernodes.  This clearly contributed to the slow recovery, during which Julian commented:

Right now there are approximately 10,000 Skype networks instead of one single "in sync" network.

Scaling?

So I wonder, apart from the login server cluster as a single point of failure, is there also a scaling issue?  FastTrack's breakthrough was the use of supernodes to make the system more scalable.  But was that just one layer of scalability?  If so, what happens when there are 300 million on-line users and one million supernodes?  Perhaps Julian (or another P2P expert) could comment...

References:

I've extracted and assembled a complete copy of Julian's relevant comments.

Vanilla Skype part 1 and part 2, by Fabrice Desclaux and Kostya Kortchinsky, Recon, June 17th, 2006.

Skype traffic during the week of the outage, captured by Phil Wolff of Skype Journal.

Skype's centralized control of P2P network parameters

The blogsphere is abuzz with reaction to Skype's second attempt to explain what caused the recent crash of their entire "peer-to-peer" network, but I haven't seen any comment on the one thing that struck me (in their 4th paragraph):

Once we found the algorithmic fix to ensure continued operation in the face of high numbers of client reboots, the efforts focused squarely on stabilising the P2P core. The fix means that we’ve tuned Skype’s P2P core so that it can cope with simultaneous P2P network load and core size changes similar to those that occurred on August 16.

As I commented earlier, we know from presentations by Desclaux & Kortchinsky at Blackhat Europe (PDF) in March 2006 and at Recon in June 2006 (PDF in 2 files: one and two), that there is substantial traffic between the (3rd-party-owned, distributed, P2P) supernodes that form the core of the Skype P2P network and Skype's (centralized) login servers.

If Skype's explanation is correct, it's clear Skype also has a way of distributing parameters to supernodes that tune their behavior.  I'm not surprised.  It's a logical to design in both measurement and tuning capabilities. 

But such centralized capabilities also represent a potential venerability.  What would happen if a black hat got access to those tuning capabilities...

Understanding the Skype Outage

Skype's official explanation.  Phil Wolff has a good set of interpolated comments on the official explanation.  There are two things to add.

1.  As the Register points out, last Tuesday was Microsoft’s monthly patch day and those patches required a re-boot.  If we believe Skype that their problem started with excessive login attempts, this is the only plausible explanation on the table.

2.  There was no patch for the Skype client (i.e. this was routine and hasn't been widely adopted) so either:

• we face another problem next time Microsoft issues patches requiring a reboot, or
• they fixed something on their servers. 

I suggest the latter.  As I pointed out during the outage, Skype generates a lot of traffic between the login servers and supernodes (see slide 16 in  DESCLAUX and KORTCHINSKY's presentation.  I suggest Skype has patched something on the login servers.  It's well known (e.g. Desclaux & Kortchinsky) that Skype login is a centralized function.

Meanwhile, it will be interesting to see if any additional comments or new client releases appear from Skype in the coming days.  I suspect not, as their approach to security has always in included both encryption and obfuscation.

Skype crash probably not DOS

Skype says it's a software issue that they are working on.  For me, it's six hours now and I haven't had more than a few minutes of "connectivity" in the past hour.  And, when connected, the on-line count was at a new low:

Another glance through DESCLAUX and KORTCHINSKY part 2 suggests, in addition to traffic from clients to login servers, Skype generates a lot of traffic between the login servers and supernodes (see slide 16 for example).  This makes the login servers a doubly critical piece of Skype infrastructure.  Of course, if they merely introduced a bug in their login servers, it shouldn't take 12-24 hours to do a roll back.

Skype Network Crashes

As others are reporting, Skype clients have been disconnecting and reconnecting around the world.  Here is Boston, I've been off line and back again at least four times in the past hour.  And when I've reconnect, an amazing small number of others are seen as on-line:

In recent weeks, I've been seeing over 9 million users on line at this time of day, so 615K suggests very little of the global Skype network is accessibile to me, if they are on-line at all.  A few minutes ago, Jan was seeing  773K other users from his site in Malaysia, so this really is global.

We've known, at least since 2004, that Skype's peer-to-peer network wasn't strictly P2P.  The vast majority of traffic (control and media) is P2P, but everytime a client comes on line (well at least at startup and each login), it interrogates the Skype Login Server at skype.com.  We also know that a Skype client must establish a connection to a super node to successfully login.  But I don't know if there is anything about supernodes that cause them to crash if they can't reach a centralized or semi-centralized Skype server.

It will be interesting to see how this develops.  Hopefully Skype will be forthcoming, but if not, I'm sure third parties will piece together an answer.

More Joost invitations available

The last time I posted this message, I had two and later five invitations to give out. I got dozens of requests before I posted a note that they were all gone.  Luca Filigheddu has seen this effect even more than I.

I appear to have received 999 invitations sometime yesterday!  I don't know how long this will last, but I've already sent out more than two dozen invitations to some of the people who responded to my earlier post. 

If you are interested, send me an email at rbt@nmss.com, listing your first name, last name and email address (that's what the current Joost invitiation form requires).  Assuming my 973 remaining invitations don't suddenly disappear, I will try and get you invited within a day or so of your email.

Real-world P2P: Joost Network Architecture

Thanks to Colm MacCarthaigh, Network Architect at Joost, for this infomative presentation given at the UK Network Operators' Forum Meeting in Manchester on April 3rd.  I found the link in a comment Colm made to Jeremy Penston's post, Joost: analysis of a bandwidth hog.

An exciting thing about P2P is it's potential for minimizing video distribution costs by sharing data close to where it's needed, ideally with peers that can be reached at minimum cost.  Of course monetary costs aren't exposed in the Internet routing tables, so p2p applications must approximate.  The question is how and that is at least partly addressed in Colm's presentation and his comment.  Here's Colm's comment:

We have some efforts in the code to save transit costs, there is very very basic prefix awareness, and we're adding AS-level awareness using live BGP data. I have looked at adding AS adjacency information, ie prefer AS-adjacent peers, but it's a lot of work and the US internet is relatively poorly mapped, so I don't think this will come soon.

Prefix awareness refers to IP address prefixes used routing packets.  Joost gives preference to peers in the same /24 address range if possible and, if not, to those in the same /16.  AS refers to autonomous systems which are the separately managed networks that, together, compose the Internet.  In most cases, it should cost less to access a peer on the same network, be that an ISP or a corporate network, than it would cost to reach a peer elsewhere on the Internet via IP Transit services.

Other interesting facts:

1. Joost uses around 700 kbps (320 MB per hour) from the internet, i.e. download bandwidth.
2. Joost sends about 220 kbps (105 MB per hour) upstream.

The presentation also includes substantial detail on the server farms that seed the P2P sharing and host less popular content.  All-in-all, it's a good overview of what's actually needed to make a large P2P network.

To my specific interest, it appears the routing inefficiencies, e.g. the "packets flying around all over the world in order to reach me" that Jeremy mentions, are a result of the small number of peers on-line during the closed beta.  Hopefully Colm will publish additional information as usage grows.

Additional invitations for Joost beta

I have another two invitations to join the Joost (formerly The Venice Project) peer-to-peer TV beta program.  Send me an email (rbt@nmss.com) if you'd like one.  Of course there are plenty of other P2P TV projects in development and several in China that have been in operation for more than two years but I haven't the time to really investigate the field.  [ I have enjoyed the Rocky & Bullwinkle cartoons on Joost.  :-) ]

Peer to Peer TV

I've mentioned peer-to-peer TV in China, several times in the past, but I've never pursued the subject in detail, as I don't speak Chinese and I don't watch much TV anyway.  However, I was interested in The Venice Project as it's was started by the same team that brought us Kazaa and Skype, i.e.  Niklas Zennstrom and Janus Friis.

Last week I got an invitation for the beta courtesy of James Enck (Thank you!).  The initial installation failed, but after a brief exchange with the project team, I was able to get up and running this weekend.  On my Verizon Fios connection (currently ~15 Mbps / ~2 Mbps) audio was perfect and the video was very good but occasionally had brief pauses. I did a brief test at NMS HQ (6 Mbps / 6 Mbps) this morning between 7:30am & 7:50am.  Everything seemed to work very well until I stopped being able to connect to new channels at all.  I haven't pursue it further as I have some real work to do.  :-)

Most likely it will be next weekend before I have time to make actual measurements, on bandwidth consumption, etc.  But meanwhile, if you are interested, I just got two invitations to participate in the beta which I'd be happy to give to the first interested parties who contact me.

Why not call "new presence" availability?

Alec Saunders has just published an excellent article on the future of presence.  If you follow VoIP, this is the Saunders who brought us the Voice 2.0 Manifesto (October 2005).  I highly recommend Alec's current essay on the "new presence," but I have some thoughts, of course :-)

Can't we change the name?  With my mobile phone on my person I am always "present" but I may not be available.  Alec's essay covers these concepts, but why not suggest a new name?  something around the word "availability" would be good.

I want a communications interface that helps me capture my preferences.  When I receive a call on my mobile, there's a one-click way to capture the caller ID into a phone book entry.  I need comparable (and better) help in creating and maintaining profiles.  If, in a particular circumstance, I decline a call from a specific caller, the phone should recognize this behavior and ask me "Do you always want to send this caller to voice mail when your phone is in vibrate mode?"  and so on.

And there's one place where I disagree with Alec.  I'm not worried about the proliferation of standards and semi-closed networks.  Just as Skype came on the scene with a proprietary system that "just worked" I expect someone to solve the "availability" problem without waiting for standards or heterogeneous networks.  I already run four instant messaging clients on my laptop.  A single client would be nice, but it's not that important.  Once we finally learn how availability should work from an existing player like Skype or from an entirely new overlay network (as Skype was a few years ago), then we can worry about consolidation.

P2P TV: in China today; new service from Skype founders shortly

Among their various pursuits, Skype founders Janus Friis and Niklas Zennstrom have a new company, called The Venice Project, focused on advertising-supported TV and video over the Internet, based on peer-to-peer technology. As they get closer to a product launch, they're getting increasing coverage, culminating in an interview and resulting article (subscription required) in today's Financial Times.

As usual the western press appears to have no knowledge of what's been going on in China.  I've commented on the problem before, in the context of instant messaging.  In the case of peer-to-peer TV, there are at least two Chinese services, PPLive and PPStream, that have been running for nearly two years.  PPLive grew out of work done at HuaZhong University of Science and Technology in WuHan in 2004.  They launched services as PPLive in January 2005 and I stumbled on them in mid-2005 when they got some press coverage in Hong Kong.  According to their website, their software was downloaded 20 million times in their first 12 months of operation.

Of the two services, PPLive is easier for a westerner to view as they have English on their website and an English option for their software's GUI.  They've also acquired fans in the west, some of whom provide further information for those not proficient in Chinese.  There are also English Wikipedia entries for PPLive and PPStream.

None of this is to disparage the Venice Project.  If the Skype founders' track record is maintained, the Venice Project should be an extremely interesting service, and it should "just work."  I've certainly signed up to participate in their beta.

Chinese clone YouTube

James Seng points out 6rooms, a Chinese clone of YouTube.  Surfing further on the subject, the best information I could find was this post from Brian Curtis, especially his further discussion in the comments area.

Actually, www.yoqoo.com was recently brought to my attention, as was www.56.com. According to Alexa, they rank:

1) 56.com
2) 6rooms.com
3) yoqoo.com
4) toodou.com (dead last)

Drives home the point that being first to market and well funded is not good enough…

Each time I’ve checked, YouTube was accesible in China. However, the censors can run hot and cold on certain sites. I am not in China at the moment to verify current status.

So a Chinese video market is emerging.  6rooms appears to have a bit over 100K videos up as of today - clearly a tiny fraction of YouTube's inventory.  On the other hand, the Chinese market is large and they are actually ahead on some video applications, like peer-to-peer TV.  So it will be interesting to see how this plays out over the next 12-18 months.

Scale free networks. long tails and big bucks - or maybe not

Some of my blog posts serve to organize a set of references on a topic that's important to me.  This is such an occasion -- hopefully one or two others may share this interest.

Back in 2002, I really enjoyed Albert-Laszlo Barabasi's book on networks, Linked: How Everything Is Connected to Everything Else and What It Means, but haven't paid much attention to scale-free networks since then.  Yesterday I stumbled on this interesting presentation by Chris Anderson at "SciFoo" last week, which prompted me to revisit the subject.  The notes in Anderson's presentation include two great references (Limpert et al., Mitzenmacher) for the mathematically inclined, and the current Wikipedia article on Scale-free networks has excellent content and a lot more references.

New to me, are the diverse models that generate networks which are scale-free but differ in other meaningful characteristics.  Barabasi discussed preferential attachment, a model which generates scale-free networks with highly connected nodes near the core.  These networks are robust even if a large fraction of the nodes are removed.  However the Internet, while a scale-free network, has it's highly connected nodes closer to the periphery, so the preferential attachment model doesn't apply.  And as Wikipedia currently puts it,

Indeed, many of the results about scale-free networks have been claimed to apply to the Internet, but are disputed by Internet researchers and engineers.

The "big bucks" part of my headline was prompted by Chris Anderson's The Long Tail: Why the Future of Business Is Selling Less of More.  Anderson's thesis, originally published in Wired magazine, is that the popularity of items in certain markets, like media, follows a power law -- the same law that applies to the connectivity of nodes in a scale-free network!  In such a market, a few products have enormous sales but there is a very long tail of products that have only a few sales each.  This long tail goes way out, so taken together, sales on the long tail represent big bucks.

Since Anderson's book was published there's been considerable argument about the long tail.  Nicholas Carr has good summary of the argument between Wall Street Journal columnist Lee Gomer and Chris Anderson.  But Andrew Odlyzko (on a private economist list, quoted here) has the best overall answer to the questions Lee Gomer raised.

As with the Internet, the key question is: does the phenomenon follow a power law (as scale free networks do)?  If however, it follows a log-normal distribution, there's a lot less money to be made selling products out on the long tail as illustrated by this slide in Anderson's presentation:

As Anderson puts it:

But here’s the problem. I’ve been assuming that the powerlaw is the “natural” shape of all these markets. But there are other distributions that start off straight on a log-log chart, then fall off the line not necessarily because of a bottleneck effect but because they’re simply not powerlaws. The best example of this is the “lognormal” distribution, which often stays straight for several orders of magnitude before sloping downward. How to tell whether a market that falls off the line is a natural powerlaw shape distorted by a removable bottleneck or a natural lognormal market that will look like that no matter what you do? In markets such as film or television archives, this could be a billion-dollar question.

Indeed!

eBay productively driving Skype features

The eBay developers conference is going on right now in Las Vegas and, while I'm not attending, eBay is posting presentations on their daily agenda pages, for example, yesterdays' are here. I did a quick read on Sunday and an update early this morning.

Reading thru the slides for Using Skype to Improve Buyer/Seller Communications by Gunjan Bhow of Actiontec Electronics one gets a good sense of just how useful Skype video and the new Skypecast feature is for eBay sellers, e.g. "Pitch your product like Home Shopping Network."

There's also a slide showing PayPal cash transfers built into the Skype User interface.  This potential feature was first discussed at eBay's May 4th financial analyst conference.

Still no word on when this feature will be released - it's not in the current Skype 2.5 beta.

Then there's the idea of deliver new services using Skype, e.g. fortune telling, Windows trouble shooting or French lessons from a native Parisian.

Interesting Concepts from the Peer-to-Peer SIP Session

I'm listening to the Peer-to-Peer SIP session at VON Europe and find myself more interested by the general concepts being discussed than by the specifics of SIP for peer-to-peer networks.  The interesting discussion is around identity, community, security and legal intercept.

Identity comes down to public key encrypted unique identifiers that you then choose to make available to various communities, perhaps including the entire Internet.  But whenever you make any identifier public, you invite people to contact you.  If you make an identifier widely available, you may need a screening function.  This can be on your client or you can delegate the function to someone else.  But this is true whether the communication means is peer-to-peer or centrally controlled. I publish my Skype user name "brough" quite widely. I only publish Skype presence to people I know and I chose what kinds of interactions I want to have with those I don't recognize.  If I did choose a 3rd party to screen contacts for me, it could be a peer proxy or a 3rd party server, but that is independent of whether the systems is peer-to-peer or not.

The security discussion exposed the fact that, whether you have peer-to-peer or not, no firewall can protect against a rogue client within the trusted environment.  This of course is why Skype works anywhere.  It just nice to hear a public discussion of this simple fact.  Not everyone gets it.

Legal intercept in the Internet can be done via a cooperative client (e.g. in an enterprise call center that does logging) or at an aggregation router.  At the router, one can only expect to get address pairs, flow statistics and copies of the packets.  If the packets are encrypted, you won't get the contents (easily).  That's it.  With encryption at the end points, you'll have to defeat the encryption to read the content of the flows.  That's independent of whether you are using peer-to-peer or a centrally controlled service.  It's a function of encryption at the end points.

All and all, it was an interesting session.

Community applications in Asia

I'm continually surprised how insular people can be, even those of us who think we're globally connected because we use the Internet. Recently I’ve written on various social networking subjects and a few weeks ago I spoke on the subject at VON, so I’ve been getting feedback and having discussions with people all over the world.

If I'm talking to another American, it's a safe bet they've heard of MySpace and, if they follow social networking applications, Facebook, Friendster or LinkedIn.  But few if any have heard of CyWorld, the largest social network in Korea, where over 30% of all Koreans (90% of teenagers) have a “mini-hompy” or homepage.

When the discussion turns to instant messaging, Americans and Europeans have heard of AOL Instant Messenger, MSN, Yahoo and Skype.  But few have heard of QQ from Tencent.  QQ is the largest instant messenger in China and, with nearly 500 million registered users appears to be the largest instant messaging community on the planet.

The Internet is global, it’s wonderful and it has vastly increased the flow of information, but we’re far from overcoming our islands of language and culture.

 

My presentation at VON

Here is the slide deck for my "Industry Perspective" presentation at VON yesterday, Mutual Disruption:  IM/ Presence Meets the Mobile Phone.

Download mutual_disruption_im-presence_meets_the_mobile_phone_v3.ppt

When Carl Ford interviewed me for a podcast the week before VON, he was surprised to see I wasn't talking about Layer Zero Competition, my topic at the last two VON conferences. So, before diving into my discussion of communities, contextually-aware communications and mobile phones, I briefly mentioned three of my passions:

1. Layer zero competition
2. The next 4-5 billion subscribers
3. VoIP disruption of telephony

and I didn't even mention video...

VoIP disruption is not a surprising topic. It's been a recurring theme for me.  The first time I spoke at a VON conference was in 1996 and my topic was wideband audio.  I had organized an IP-PBX panel and used my position as moderator to advocate wideband audio rather than "toll quality speech," i.e. don't struggle to duplicate the PSTN, use IP communications to do better!

Some of the points in this presentation are:

• With a few exceptions, most VoIP service providers are still thinking like telecom engineers, not focusing what the user is trying to do, i.e. connect with another person in a fashion appropriate to what the user is trying to accomplish and what the other party is available for.
• Presence is the wrong word.  What we mean is availability, with contextual qualifications.  My mobile phone is always with me, always on, but I may not take your call !
• Mobile phones are more important than desk sets.  Indeed, mobile has done a better job of addressing user interests than VoIP has so far. (And, mobile has been far more widely adopted than VoIP).  Even more significant, in developing countries mobiles far exceed PCs with Internet connections. Finally, Asia leads in mobile innovation, so US & EU folks need to pay attention to Asia.
• Person-to-person communications is the biggest opportunity (bigger than content sales and most other "data applications"), but it's centered around communities. Individuals participate in multiple communities, so there is no way a walled garden operator can serve all their customers' communities.
• Communities in Asia are equally available on Web & WAP.  I have some views of my WEB and WAP home pages on 12wap.net (in Singapore).
• Recently the GSM Association announced ubiquitous interoperability for presence on mobile phones. If they succeed with this initiative, it could represent a significant threat to AIM, Yahoo and MSN Messenger.

I closed by suggesting there was an enormous opportunity, in the next 18 months, to create the best combination of communities, availability, Web 2.0 and mobile phones.

Reverse engineering Skype

In the Peer-to-Peer SIP session at the VON conference on Wednesday afternoon, Henning Schulzrinne mentioned the existence of this really interesting presentation that was given by Philippe BIONDI & Fabrice DESCLAUX at the recent Blackhat Europe conference in Amsterdam March 2nd & 3rd.

It's 115 slides with tons of techie detail.  Read the summary first.

Multimedia creation easier on mobiles than text

I just stumbled on a short presentation that VeerChand Bothra gave at BarCampDelhi on March 4th.  It's really great! - in three slides it touches on several topics of recurring interest to me.  :-)

I follow his Mobile Pundit blog for it's coverage of telephony news in India, but this was the first (well maybe the second) time I noticed how much our thoughts on telecom and cultural issues are in alignment. My thoughts, triggered by his presentation:

First, about user-created content.  Mobiles are ubiquitous and as VeerChand points out, with mobiles, multimedia creation is easier than text creation.  We're already seeing explosive growth in Internet-based picture sites, Flickr and the like.  According to this article, most of the referrals to photo-hosting sites already come from blogs and moblogs.  With 3G, user-created video clips are about to soar.  Part of my interest in our recent acquisition of Openera Technologies is it gives us a peer-to-peer video sharing capability. As our "community-based services" kiosk at 3GSM suggested, capturing P2P shared video on behalf of the creator preserves an option for its later use in a moblog, blog or community homepage (think Myspace, Cyworld, or 12wap).

Second, the importance of communications technology to the advancement of civilization.  This was the subject of my immediately previous post.  VeerChand's comment was:

Language, the alphabet,cities, the printing press did not eliminate poverty or injustice.  But they did make it possible for groups of people to create cooperative enterprises such as science and democracy that increased the wealth, welfare and liberty of many.

Third, communities dominate brands. For decades I've put more store in public relations than in advertising. With the advent of the Internet, it's now clear that chat rooms, blogs, etc., i.e. unfettered community discussions are more important than traditional PR.  Organizations that candidly talk with their communities of interest get more useful input -- and more external credibility (leading to good PR) than any advertising or PR campaign could ever generate.

Communities more important than social networking software

In the past 48 hours, I've had three very different conversations about social networking, touching on US and Asian variants like Myspace and Cyworld (in Korean or about Cyworld in English).  In each case, my perspective was a little different than that of the people I was talking with, so maybe it's worth setting out.

A social networking environment (software, etc.) is necessary but not sufficient for success. Success depends upon communities.  That's the reason Myspace is doing so well and Friendster has languished.  As Anick Jesdanun points out,

...when its founders noticed heavy usage among musicians and fans, MySpace embraced that community with custom features...  ultimately music is what made MySpace special.

Once communities had formed around bands, musicians and their fans, Myspace was off and running.

Of course the underlying social networking software must have competitive features, but I already run four instant messenger clients because the different communities I participate in happen to run on different platforms.  It's the communities, not the platform that counts.

I've looked at (and joined) Myspace and Facebook, but they are both Internet centric, i.e. no mobile support. Not reading Korean, Chinese or Japanese, I haven't been able to join Cyworld. Luckily, at 3GSM, I ran into long time friends Kevin Chia and Michele Chan of Singapore who are operating an interesting Internet/ mobile community under the name 12wap (pronounced "want to wap").  I've just started exploring 12wap which lets you set up both WAP and Internet homepages.  I haven't set up my home page, but you can see my photo on your mobile at wap address 16172850433.12wap.net :-)

End-to-end overtakes Multicast

A few weeks ago, Jeff Pulver wrote a lament for IP Multicast and the Mbone (an experiment in multimedia broadcasting over the Internet).  One problem with IP Multicast (and the Mbone) is it depends upon extra complexity in backbone routers, i.e. in lower layer protocols, in the network.  Jeff quotes a friend,

It appears that most ISPs are afraid of the multiplication effect that can be achieved by allowing their customers to send multicast -- and the difficulties this creates for capacity planning and traffic engineering.

One could attribute lack of widespread multicast deployment to its technical complexity, economics or evil intentions, but it's probably just the end-to-end principal, as articulated by Saltzer, Reed and Clark in 1981, at work.  In other words, building a broadcast mechanism into the lower layers of the Internet backbone makes less sense than providing the equivalent functionality at the ends -- in this case at many, many ends as we're talking about broadcasting.

The first end-to-end approach to distributing multimedia content was BitTorrent.  While BitTorrent is not real time -- it doesn't handle broadcasting as we know it -- BitTorrent did show how to distribute the load of one-to-many distribution among many end points.  And as TiVo demonstrates, real time broadcasting isn't always required..

But various groups are discussing real time versions of peer-to-peer TV and sites in China, such as PPLive and PPSteam, are actually providing live (Chinese) TV over the Internet today.

The end-to-end principal is wins again.