Communications

Fiber problems cause six hour outage

On Friday evening, netBlazr had it's first major outage, unfortunately lasting six hours. In the end, the problem was with one of the fibers in Cogent's riser cable inside the John Hancock Tower. It was not a simple cut. That would have been obvious. Instead we had marginal light levels, perhaps due to a nick, a overly-tight bend or a poorly assembled connector.  

For those that are interested, I wrote up my Friday evening adventures in summary form, followed with more detail for those who are really, really interested. Then this morning I wrote my followup list, again in the netBlazr blog for those who are really interested in the day-to-day issues of being an ISP.

No spectrum shortage, just an allocation problem

As a new study from Citi Investment Research & Analysis make clear, the US does not have a spectrum shortage. We've just allowed a relatively small number of carriers to control the spectrum. Quoting the study's summary:

Today, US carriers have 538MHz of spectrum. And, additional 300MHz of additional spectrum waiting in the wings. But, only 192MHz is in use today.

Perhaps if we had an effective "use it or lose it" policy in place, or a heavy tax on unused spectrum a more vibrant market for this spectrum would emerge.  But today, the problem is not a shortage of spectrum but the fact that what's out there is not being utilized.

Obviously things vary by geography, but Citi's summary is completely justified. Their methodology is thorough both as to who owns what and what is deployed county-by-county for 3100 separate counties. Here's the summary of what's in use:

and here are the details on what's currently owned by US carriers:

So why would we repack the TV broadcasters and auction off that spectrum when we've just finished putting in place unlicensed access to TV white spaces?  Unlicensed spectrum will be heavily utilized while more exclusively owned spectrum will just add to the pool of under utilized resources.

White spaces above 3 GHz

Yesterday, I gave a pitch for more access, on a secondary use basis, to the vast swaths of spectrum above 3 GHz - spectrum that is largely unused in most locations most of the time.  Then I gave an update on netBlazr and how we are using the existing license-exempt spectrum in the 5 GHz band (over 500 MHz in the US).

I'm sorry there's no audio, but here's the deck I used:

White spaces above 3 GHz and an application

View more presentations from Brough Turner

Wireless conferences in Austin Texas this week

While the big event is Internet Telephony Expo, there are two other conferences and a startup pitch fest going on in Austin this week.  I'm leaving in the early morning and will be speaking twice tomorrow afternoon and then pitching netBlazr on Wednesday afternoon.

Tomorrow at 2:30pm I'm appearing on a panel at 4G Wireless Evolution. The panel is entitled "Wireless Fixes Access with Redundancy." The other panelists are from Sprint and Wave2Wave. Sprint is using point-to-point wireless for backhaul from their cell sites since they can't get competitive access to the fiber backhaul that AT&T and Verizon have. Wave2Wave is wireless Internet Service Procider. As netBlazr's approach is rather different, in both our business model and our use of fixed wireless technology, this should be an interesting panel.

Then at 3:30pm I'm the sole speaker for a session in the Super WiFi Summit just down the hall. The description is:

White Spaces: The Radio Evolution

Tuesday - 09/13/11 • 3:30-4:15pm

Brough Turner , Founder , netBlazr.com

Smart antennas and smart radios, Cognitive Radio and Beam Forming are on the verge of being incorporated into product. As we head toward these technologies, the opportunities exist for new models of service sharing and interconnection to deliver broadband solutions.

My talk will be a mix of what's possible today, what will become possible with next generation silicon and how netBlazr is using today's and tomorrow's technology to do an end run around the existing broadband market.  Of particular interest is where additional white spaces would be useful - hint it's not just in the TV bands.

Finally, there's Startup Camp on Wednesday afternoon and early evening.  netBlazr is one of five new companies choosen to present and I'll be doing the honors.  Bob Metcalfe is the keynote speaker and there will be a panel of judges to provide critiques on the startups' pitches.

Freemium applied to wireless networks

Recently, netBlazr got an inquiry from someone who had seen this video.

It's the talk I gave at the International Summit for Community Wireless Networks (#IS4CWN) in Vienna last August. I put my slides up on Slideshare even before I returned from Vienna but I didn't realize the conference organizers had put the videos up (apparently this happened in March).

Comparative broadband speeds - US loses again

Recently Google Labs added the Ookla Speedtest data set to their wonderful Public Data Explorer so I just had to try it out.

Here's my first graph:

These are not bogus statistics. These graphs show the average of all the millions of actual speedtests run in the respective countries over the past 4 months.

The apologists for the US's poor showing usually start by pointing out that the US has much lower population density than South Korea, Sweden or Bulgaria. Of course that doesn't hold water if you compare Seoul, Stockholm and Sofia to New York, Los Angeles or Chicago. In Seoul, Stockholm and Sofia consumers can get 100/100 Mbps Internet access at affordable prices (e.g. $13/month in Stockholm) while such service isn't even offered to consumers in New York, Los Angeles or Chicago.

 

Please vote for netBlazr at MassChallenge

netBlazr benefitted from last year’s MassChallenge startup contest even thought we were only a few weeks old when we entered.  We’ve come a long way in the past 11 months.  If you haven't been following, we're trying to change the face of broadband access in US cities (more info here).

netBlazr has entered the MassChallenge 2011 Startup Contest and we’d like to solicite your endorsement of our entry.

It’s easy to vote:

1.  Register at https://masschallenge.org/user/register You need to provide an email addresss and pick a “role.” But if you specify your role as “Observer/Supporter” you can avoid filing out any other forms.

2.  You’ll receive an email from MassChallenge.  Click on the link in this email to validate your registration.

3.  Go to http://masschallenge.org/profile/team/netblazr and check out our profile.

4.  Float your cursor over the “Votes” area in the upper right, then click on five stars.

That’s all there is to it.  Thank you!

Paid Peering: Issues and misunderstandings

Recently I was asked my opinion of Google paying France Telecom (FT) to deliver traffic to into FT's network, i.e. Google paying to peer with FT. I wasn't aware Google pays FT. I don't even know if it's true. But I do know this is a topic fraught with misunderstandings. Also, if there is a "problem" here, the problem is one of competition (or lack thereof) in portions of the French broadband access market. It is not a problem that can be or should be fixed by "network neutrality" regulations or legislation.

To understand what's happening, you need to understand the Internet's business model and you need to know something about peering as it works in the real world.

The most suscinct definition of the Internet I'm aware of is: 
Internet (n.) A voluntary agreement among network operators to exchange traffic for their mutual benefit.

This is very different from the telephone networks (fixed or mobile) where relations between operators are governed by laws and regulations, change at a regulatory pace, i.e. decades, and are continuously gamed by political processes.

Each Internet operator needs to be able to reach all possible Internet addresses, so interconnection is essential. If you are a small operator, like netBlazr, you purchase "Internet transit" services from a larger operator. Rates are volume dependent and arrangements are similar to those of any other business or individual purchasing an Internet access service.  If you are a large operator, a large content provider or a very large business, then interconnection costs and arrangements become negotiable. Two operators in Holland will likely agree to exchange that portion of their respective traffic that is destined for each other's networks at the AMS-IX which is local, rather than pay for Internet transit services that would likely carry that traffic to London or the US where a higher tier operator would handle the exchange.  Thus the two Dutch operators engage in peering for that portion of their traffic for which it makes sense.

This pattern is repeated throughout the Internet as each operator evaluates where, how, and with whom it makes sense to exchange portions of their traffic so as to minimize costs, increase performance or achieve other goals important to that operator. Thus there are vastly more interconnection contracts, formed under more diverse terms, for the Internet than ever was the case for the telephone network. Further, this network of agreements is in constant flux as individual operators find better deals.

It's instructive to follow the growth of YouTube from their founding through their acquisition by Google. When YouTube began, they had to pay standard rates for Internet access. As they grew to be a significant content source, their cost of Internet access fell as more and more operators asked to peer with them, as I discussed two years ago.

So between Google and France Telecom, who has negotiating leverage? and why?

France Telecom already has peering agreements with multiple other Internet backbones, so their customers have several paths to get to Google content.  Why would either party want to connect directly?

First, it might cut costs. Whatever pre-agreement path the traffic takes involves some cost. Perhaps that's money paid to other operators or it may just be the cost of maintaining the facilities needed for the pre-agreement traffic (e.g. to go via London).  I can't guess the specifics for Google or for France Telecom, but each has a possible interest here.

Second, it will likely improve performance. Even shaving 50 milliseconds off of round trip times (RTT) can improve user experience as accessing a single web page can require the exchange of dozens of messages, thus 50 ms less RTT can shave more than a second off the page load time. In a competitive market, France Telecom might want to advertise their network as the "fastest," but local access competition is not that great in France (or in many markets, like the US).  France Telecom has nearly 50% market share.  They are losing ground to Free, but apparently it's not enough to justify a marketing campaign based on the "fastest" network.  Google on the other hand, has a strong reason to want better user experiences. In Google experiments with search latency, longer latencies reduce the number of searches a user does per day. Given the number of Google search users on the France Telecom network, Google can directly compute their monetary return from cutting search latency by X ms. So Google's negotiators have hard numbers in their back pocket when they go to talk with France Telecom.

Is this good or bad? Well, if France Telecom doesn't have to compete on the performance of their service, that suggests a failure of competition policy. But then a solution should address competition in the French Internet access market, not government regulation of peering or the imposition of "network neutrality" regulations on operators that don't have a monopoly position.

For myself, I look forward to continued growth of new providers in France, like Free, as I discussed in the January issue of NGN Magazine.

 

Paul Baran and open spectrum

Paul Baran, an engineer who helped create the technical underpinnings for the Arpanet (the precursor to today’s Internet), died Saturday in Palo Alto, California.  While everyone else celebrates his early contributions to packet networking, I will remember him for his open spectrum advocacy.

I never had the privilege of hearing Paul Baran speak live, but as I became interested in wireless spectrum, one of the earliest references I encountered was this transcript of Paul Baran's Keynote Address at the 8th Annual Conference on Next Generation Networks Washington, DC, November 9, 1994.

His title says it all:

Visions of the 21st Century Communications: Is the Shortage of Radio Spectrum for Broadband Networks of the Future a Self Made Problem?

And here's an excerpt:

   The key point at issue that we will question is the widespread belief
   that we don't have enough radio spectrum to go around. This is a
   common, fundamental belief. Since we live in a world of scarcity or
   natural resources it is almost automatic that we believe that there is
   a shortage of frequencies. This particular resource is somewhat
   different.
   
   This morning, let's start by reviewing this presumption of a permanent
   shortage. Let's consider how, with an application of already known
   technology, we could create even a surplus of frequencies. What may be
   going on is an inadvertent shortage created by a regulatory structure
   which has yet to appreciate the potential capabilities of the new
   digital signal processing technology as applied to communications.

Paul Baran is hardly the originator of open spectrum ideas, but the complete transcript shows he once again led the industry.

 

Mobile oligopoly tightens

I remember the heady days of the late 90s (while the PCS build out was happening).  Depending on how and when you counted, we had more than 6 mobile operators in the Boston area.  Everyone was competing for customers, to the great benefit of the consumer, as mobile phones rapidly transitioned from expensive car phones to relatively low cost, widely adopted personal handheld devices.

In recent years we've seen that competition shrink to just four operators (Verizon, AT&T, Sprint and T-Mobile).  At one time, I was hopeful that Clearwire would be one more operator, but then they became effectively joined to Sprint.  More recently I've been hopeful about Lightsquared, although they are a long way from providing service.

Now we get bad news.  AT&T is buying T-Mobile.  Consolidation!  Less competition not more.  Ugh!

My Column has moved to NGN Magazine

"Next Wave Redux," which has appeared in Internet Telephony magazine for many years has moved.  I'm now writing for NGN Magazine.

With my attention focused on netBlazr, I wasn't in a position to do justice to Internet telephony topics.  I thought of abandonning the column entirely, but the folks at TMCnet urged me to move to a different magazine, more in line with my current focus.  Of course, the term NGN carries a lot of baggage which I address in my first article, NGN: ITU Misses the Boat, written last fall for the November 2010 issue.

If you followed by columns in Internet Telephony, you may want to sign up for electronic delivery of NGN magazine (i.e. via email), as the issues typically don't appear on the TMC website until a couple of months later.

Super Wi-Fi Summit, 4G Wireless Evolution and IT Expo

I’m in the midst of multiple co-located conferences, Super Wi-Fi Summit, 4G Wireless Evolution, IT Expo, all conveniently located in Miami, although I barely got here on Tuesday just ahead of a major storm in Boston.

At the Super Wi-Fi Summit, I gave a presentation about netBlazr.  This conference is focused on technologies for use in the newly available TV White Spaces spectrum, particularly Wi-Fi (802.11af). I told Carl Ford (the conference organizer) that netBlazr was using 5 GHz 802.11n, not TVWS, but he still wanted me to present the netBlazr story.

Disruptive broadband

View more presentations from Brough Turner.

Then, in the 4G Wireless Evolution Conference, Carl had asked me to give some perspective on when Over-the-Top applications will conquer all.  With all my attention focused on netBlazr, that was a little off topic, but I have views and opinions which I put together in this presentation.

Why Over-the-Top is the Future of Communications

View more presentations from Brough Turner.

Aside from my two presentations, I’ve met a lot of old friends and met several useful contacts for the future.  I’ve also met a number of wireless ISPs, both through the Super Wi-Fi Summit and through the WISPA booth on the show floor.  All in all, well worth the trip!

 

Cable trounces the telcos

Yesterday, Netflix posted graphs of how well various ISPs deal with Netflix video streams.  The results are striking.

All the cable companies easily beat all the phone companies with the exception of Verizon where we're seeing a mix of DSL and FiOS results.

And just in case you believed the hype about 4G, notice that Clearwire's Clear service comes in below DSL.  There is no reason to believe LTE networks will do any better once they have customers, i.e. once they are carrying commercially viable levels of traffic.

 

netBlazr, the Movement!

A few weeks ago, Gordon Cook, publisher of the Cook Report on Internet and moderator of a well respected private email discussion entitled the Economics of IP Networks, asked me to explain to his list members what netBlazr was up to.  I got a bit impassioned and ended up included the background and motivations as well as our vision of how we will change the world.  

If you are interested, I've posted it on the netBlazr blog.

Broadband Plan panel discussion at eComm

At the Emerging Communications Conference in San Francisco last spring, I organized a panel to discuss the US National Broadband Plan which had been published about five weeks earlier.  Videos of that discussion are now on line.

or watch it full screen in HD here (Flash).   iPad/iPhone users can watch it full screen in HD here (Quicktime). 

Besides yours truly as Moderator, the panelist are:

• Paul Brigner, Verizon
• Susan Estrada, FirstMile.US
• Tracy Rosenberg, Media Alliance
• Richard Bennett, ITIF

Of course, organizing this panel forced me to read the FCC's National Broadband Plan cover-to-cover. It's a mile wide but only a few inches deep.  There are no major policy changes, just battling between existing vested interests.

So the release of this plan only increased my resolve to find a commercial path that could fix US Internet connectivity mess - thus the launch of netBlazr.

An opening to rationalize Internet regulation

Late yesterday, a relatively diverse, and hopefully significant, group of advocates for the Open Internet filed reply comments in the FCC's proceeding "Further inquiry into two under-developed issues in the Open Internet Proceeding."

I joined in this filing, for reasons I've described in the netBlazr blog.

David Reed has a more elegant description of his reasoning here.

Beamforming for Wi-Fi, WiMAX and LTE

Concentrating a wireless signal in a tight beam has tremendous advantages. You need less power for equivalent distance, you create less interference to others and, if your receiving antenna is similarly directional, you avoid a lot of interference from others. Of course people have been using directional antennas to achieve narrow wireless beams for years. But a directional antenna has to be aimed and then it's fixed.

That's great for a point-to-point link but useless for someone on the move.

Beamforming creates a narrow wireless beam that can be steered in software. Suddenly the advantages of narrow beams can be applied to point-to-multi-point systems and in systems where one or both ends are in motion. Of course the process is quite sophisticated and, until recently, was very expensive. Luckily that is changing, rapidly!

This slide shows how, with beamforming, one Wi-Fi access point can communicate, in rapid succession, with three different clients.  And this can happen despite another access point using the same channel to talk with one of it's clients. Indeed, it all works despite the existence of a legacy Wi-Fi device that is broadcasting omni-directionally on the same channel.  Pretty powerful!

Think what something like this could do for tenants in a large apartment block. It no longer matters that 34 neighbors have nearby Wi-Fi access points.

Proprietary or "standard" ??

Based on questions I get, there appears to be quite a bit of confusion. The answer of course is we see both proprietary and standard implementations within the Wi-Fi market.

Beamforming is an optional part of the 802.11n standard and silicon support for this option is emerging (more on that in a moment). Beamforming is also part of the most recent revisions of WiMAX and LTE standards. Confusion arises because there are several different things which are legitimately called beamforming.  

The simplest is a switched beamformer in which there are multiple directional antenna elements and the radio or radios are connected to the appropriate elements as needed.  This is what Ruckus Wireless does today. It's what Vivato Networks did with 802.11g back in 2003. The Ruckus products have 12 or more fixed elements and, on a frame-by-frame basis (e.g. every 10 ms), they decide which two of those antenna elements to connect to the two terminals on an Atheros (2x2 MIMO) Wi-Fi chip, based which combination works best for the specific device the access point is communicating with.  Many people have patents here, but the ideas are very old, so the patents may not be very valuable.

Next are phased array beamformers.  Here multiple simple antenna elements, equally spaced, form an array.  Each element sends (or receives) the same signal but phase delays are introduced so, via constructive and destructive interference, the resulting wireless signal forms a narrow beam.  This beam can then be steered by varying the phase delays. Phased array beamformers are also well established with examples dating back to 1905 and widespread use in radars since World War II.

Finally, in MIMO systems, precoding is a mathematical generalization of beamforming that leverages antennas elements that may not be equi-spaced and deals with the fact that the other end has multiple receiving antennas. And at the receiver end, maximal-ratio-combining (MRC) optimizes decoding of the transmitted signals. Very roughly you can think of these MIMO computations as putting the maximum beam lobe as close as possible to the desired target while placing nulls (of the antenna pattern) as close as possible to the primary interferers.

Today's widely deployed Wi-Fi chips have two radio chains (2x2 MIMO) which are typically used for horizontal and vertical polarization, but 3x3 and 4x4 MIMO chips have come to market.  With 4x4 MIMO Wi-Fi chips we see transmit beamforming via precoding and receive beamforming via MRC.  Indeed, two silicon startups, Quantenna Communications in California and Celeno Wireless in Israel, have announced Wi-Fi chips that support 4x4 MIMO with transmit beamforming and both vendors' chips have shown up in consumer products, e.g., the Quantenna chip in the Netgear WNHDB3004.

The 802.11n standard specifies how the needed information is passed, so the computations that Quantenna and Celeno do (and others in the future) can be carried out at either end even though the devices are from different vendors.

 My interest

Beyond mentions in my speaking engagements and wireless tutorials, beamforming will be directly useful to netBlazr, our new (and radically different) wireless ISP.  netBlazr depends upon participants deploying network elements.  Today, these require aiming but, with beamforming, aiming won't matter and, if the unit is bumped later on, beamforming will compensate on-the-fly.

 

Singapore's fiber infrastructure beginning to pay off

It's still early days, but Singapore's approach to fiber deployment is beginning to pay off.  In December 2007, Singapore announced a major program to get fiber deployed throughout their city state (see my blog posts here and here).  A critical advantage of their approach was government mandated structural separation between the dark fiber layer deployed in the public right-of-way (a natural monopoly) and higher layer services (where competition is possible and highly desirable.  To ensure that the dark fiber is lite, they have also contracted with one company to provide wholesale layer 2 services over the dark fiber, but that company is independent of the dark fiber company and cannot prevent any interested 3rd party from directly obtaining dark fiber.

Now three years later, the first stages of the new network are coming on line (it's scheduled to be 60% complete by December 2010).  So far there are only(!) five retail ISPs offerring services over the new network.  Examples in other cities like Stockholm suggest the availability of dark fiber will result in dozens of service providers (>50 in Stockholm) and very low prices (less than $10/month for 100/100 Mbps service in Stockholm).

This article has detail on the initial service offers in Singapore.  The first offer is SG$50 (US $38) per month for 100/100 Mbps Internet access service.

As I've commented in the past, structural separation is critical to obtaining vibrant, but the layer at which the separation occurs is also important.  This chart:

provides a clear illustration why the logical layer at which to provide open access (and above which one seeks competition) is dark fiber!

 

netBlazr at MassTLC's Innovation Unconference

I was at the Mass Tech Leadership Council's Innovation Unconference yesterday.  As in the past, this is a great event, but this year it gave me a chance to update various friends and mentors on our progress at netBlazr.  

I ran a session on Broadband Disruption for which I had prepared some slides although I didn't use them in the end - we just talked.

Another highlight was the "Freemium" discussion but again, no slides or notes, just a good discussion among an excellent set of people.

US Broadband - a personal rant

With netBlazr up and running, it's time for me to write some posts on the netBlazr blog.  I've started off with a personal rant about broadband in the US and why I'm putting all my time into our attempt to do an end run around the whole mess — the duopoly (phone company & cable company), the regulatorium (FCC, etc.) and Congress.

Of course this mention on my personal blog is not only to induce you to look at the netBlazr blog, but to attract some links pointing to the netBlazr website.  Right now, when Google sees "netBlazr," it thinks you've mistyped "netblazer." Please help us change that!  Write something that points to somewhere on the netBlazr website.  Thanks!

Wi-Fi: the Real 4G

Yesterday I gave a keynote address at the 4G Wireless Evolution conference.  Since I assume I get invited to these things because I'm controversial, I picked the title "Wi-Fi: The Real 4G" and used my 45 minutes to hit several points:

1.  All key 4G technologies have been and are being pioneered by Wi-Fi (3-5 year lead over WiMAX or LTE!).

2. Wi-Fi will be the dominant solution for mobile data offload.

3.  4G technologies represent a wireless tipping point with the result they will revolutionize backhaul and eventually the first mile (via wireless ISPs).  And, of course, Wi-Fi products lead here as well.

Finally, I closed with two slides on our new wireless ISP, netBlazr.  Here's the deck.

Wi-Fi is the new 4G

View more presentations from Brough Turner.

 

4G Wireless Evolution conference next week

On Monday, I'll be in Los Angeles for the 4G Wireless Evolution conference.

I'm giving a keynote address on Monday afternoon at 3:15pm and I'll be on a TV White Spaces panel at 11:15 on Wednesday morning.  I'll of course post my keynote presentation to my Slideshare account after the conference.

And for those who are curious, my keynote will connect the ideas of mobile data, 4G, Wi-Fi and mention how it relates to my new company, netBlazr.

If you will be at 4GWE or at IT Expo, please stop by and say hello.

 

Network Neutrality is the wrong fight!

Winning would mean giving up much more important rights – historical rights that were in place in the US as recently as 1995 and remain in place in most of Europe even today. 

We shouldn’t settle for network neutrality. It’s a poor substitute for what we had and much less than what we need. Let me explain. There are two topics to discuss. The first is “common carriage,” a centuries old legal concept that applied to the US telecom industry throughout most of the 20th century. The second involves communications protocols. Both topics are complex, so I will cover only what’s needed to understand why we shouldn’t accept network neutrality and why, at a minimum, we should fight for enforcement of existing common carriage rules. 

Network neutrality is about allowing any Internet application to run over an Internet connection, i.e. over a connection that uses Internet Protocol (IP). But under common carriage as it applied prior to the late 1990s, we had a more powerful right – the right to run any kind of network protocol, IP or otherwise, over a lower, simpler service which today we call a “bit stream*.” Why does this matter? Because real innovation is also possible at these lower layers and that innovation continues to be important. But today, such lower layer innovation is restricted to inside one building or one campus. Yes, we can tunnel some lower level innovations over IP, but not all of them and only at a cost. 

IP telephony (VoIP) is one place where problems arise. Most enterprises use IP PBXs internally, yet calls between enterprises use the PSTN. Many companies have attempted to address this gap, but progress is slow and expensive. Within an enterprise, IP telephony packets are given priority, but that priority is not supported on Internet access links and network neutrality doesn’t help. As a result, to interconnect VoIP calls, enterprises must lease separate dedicated access circuits – circuits usually based on bit stream access – to support “SIP trunks.” Up until the late 1990s, these circuits were regulated under common carriage. Today they are an unregulated monopoly, with prices derived from the cost of voice circuits 15-20 years ago, i.e. abnormally expensive for today. 

 Common carriage is the legal concept that, in exchange for government granted monopoly access to rights-of-way, the monopolist must carry anyone’s traffic over the resulting infrastructure, at regulated rates. For centuries this has applied, to canals, to roads, to railroads, to telegraph lines and, until nearly the end of the 20th century, to telecommunications lines. But during the legal battles after the Telecom Act of 1996, the FCC basically gave up on common carriage.

If we accept Network Neutrality instead of common carriage, we guarantee future innovations happen only above the IP layer. Innovation at lower layers will be restricted to enterprise or campus applications. That’s too bad as it was the existence of common carriage that allowed the Internet to develop in the first place. Do we want to eliminate that kind of innovation in the future? 

If anything, we should be fighting to extend the ideas of common carriage to lower layers, e.g. dark fiber. Installing dark fiber is expensive and requires access to rights-of-way that are limited. The installed fiber is capital expensive infrastructure that lasts for decades. Such conditions justify granting monopoly access, in exchange for common carriage and regulated rates of return. But when you light up a dark fiber, you use (relatively) low cost gear with a short life (even if it can survive for ten years, Moore’s Law renders it functionally obsolete within 2-3 years). What's more, there’s rapid innovation in opto-electronics gear. Just look at the order of magnitude difference in cost between enterprise and carrier fiber-optic gear. 

Today, the US is loosing leadership in all things Internet. Network Neutrality will just put a nail in our coffin. To stop our decline, fight for restoration of the common carriage principals that existed through most of the 20th century and still exist in law. To regain world leadership, fight to extent those principals to include access to dark fiber at regulated rates.

---------------------------------------------------------------------------------------

* Bit stream access.   In the 20th century two regulated services provided what the 21st century calls bit stream access. These were voice telephony and T1 circuits. T1 circuits directly carry a stream of digital bits. Modems allowed voice connections to carry digital bits, for example, for bulletin board services and other purposes long before the Internet became popular.

Ben West's coverage of ISCWN

While at the ISCWN in Vienna, I met Ben West of WasabiNet in St. Louis. He’s provided excellent coverage of the sessions he attended in his day-by-day posts here:

Day 1;  

Day 2;  

Day 3.

My notes from the Community Wireless Networks Summit in Vienna

As with most conferences I attend, I took most of my notes using Twitter.  Since I can't always depend on accessing old tweets, here is the entire tweet stream from the conference in one blog post.  I don't know if this is of use to anyone else, but at least I know I'll have a record of my notes that I can refer back to.

The full name of the conference was The International Summit for Community Wireless Networks (ISCWN) and it was held in Vienna on August 12-15, 2010.  See the conference website.

International Summit for Community Wireless Networks http://bit.ly/9Lp41p
Good conference Wi-Fi http://bit.ly/dkwUh2
OK, I'm finally set up to Tweet &/or blog the International Summit for Community Wireless Networks http://wirelesssummit.org
While there's plenty of broadband behind the Wi-Fi, there are very few power outlets, at least as yet.
Community Wireless conf finally getting underway
Sascha Meinwrath opening: 1/2 the participants are missing, even some who were here an hour ago, so expect stragglers, but we start.
Sascha: 80% of cash raised goes to travel allowances; 10% everything else; plus donations.
Aaron Kaplan, Funkfeuer, on TechGate facility science park estbsh'd 2001 by City; conf venue; also supports Funkfeuer
Aaron there is swimming in the old Danube - it's clean now! Also boat rental nearby!
Aaron: Funkfeuer hotspots since 2003, now reaching almost to Bratislava. Mixed mesh and P2P backbone
Sascha on previous feedback - social time is key
Jim Baller: America at the Crossroads: Greatness or Mediocrity. 15+ yrs fighting ILECs on behalf of muni's - mostly successful!
Baller: credits Broadband Coalition http://bb4us.net/  Also as key to Congress requiring FCC to do a BB Plan
Baller on the US national BB plan: We need big goals, the BB plan didn't deliver! It's goals for 2020 happening elsewhere today.
Baller: US BB Plan focuses on inches, not yards and miles.
Baller: US BB Stimulus: Awards coming quickly (9/2010 deadline): middle mile, rural first mile
Baller on Google gigabit initiative yielded 1100 muni applications and showed pent up demand
Baller: Ugly side - incumbents bigger, stronger & nastier than five yrs ago. Cable industry even nastier. Now Google is in question.
Baller: But btwn BB Plan (it's a plan), Google fiber init., $7B Stimulus goes to new parties, community BB efforts
Baller responding to Q about wireless: I'm a fiber guy, then disappointing comments on FCC being pro-wireless. Misses FCC focus :(
Baller responding to Q about Australia admits that most in US don't know what's happening in the rest of the world.  US always has excuses - "we're different."
Sascha introducing Ramon Roca, President, guifi.net Foundation, a fantastic alternative that's happening in Spain. 
Guifi runs a very large network across rural (and urban) Catalonia in Spain. http://guifi.net/
Ramon Roca: Vision for the Future. Will talk about: Why scale? and their sustainable economic model!
Ramon: Guifi has 10K+ nodes and 15 Km of networks, but still tiny compared to telco. Scale as a goal. Need growth to be sustainable.
Ramon: need sustainable model - critical mass; org issues; legal; partners. Non-profit core w/biz around.
Ramon: Demand is there; also incumbent not interested in low 40%.
Ramon on mgmt: horizontal, bi-directional & collaborative - examples from their website. Website appears to be key for communit.
Ramon on public/private: public requirements must be written: contracts, license, P2P agreements. Must be clear, and legal.
Ramon on public/private - like OS licenses, the network is considered private by govnmt. Need documents just as open src does.
Ramon facilitates integration and cooperation between diverse groups, e.g. need churches for their steeples! need public ROW access.
Ramon: Guifi license is required if you want to participate! Like GNU license, Guifi license even deals with revisions.
Ramon: Guifi license defines everything
Ramon: Guifi is also a ""public"" network in the legal sense in Spain because they had to in order to deploy some fiber.
Ramon: Guifi ROI for many: supply chain dlrs/shops, prof svcs and svc provdrs who mediate wholesale mkts.
Ramon: Catalonia is a bit ahead of Spain but w/o Guifi Osona (rural Catalonia) way behind. Now Osona ahead of rest of Spain & even the UK.
Ramon: Guifi network now 15% of BB links in Osona; and yet DSL subscription haven't dropped.
An English version of the Guifi Wireless commons license is here: http://guifi.net/WCL_EN
Ramon futures: working on fiber from farms (FFtF) cheap & easy & Gbps; always a combination.
Ramon has had fights with incumbent about ROW (even when on private property) and about access to poles. Ramon - do it! then fight.
Ramon - their main focus is to stay on private ROW, but as a public carrier they can now go after public ROW.
Ramon: stories about fights over public fiber & public ROW - everything you expect, except Guifi seems to win, espc. in rural area.
Ramon: Guifi participants getting access to public fibers laid for cameras after threaten authorities - tractor might hit camera...
Ramon concludes by emphasizing need to enlist service businesses and computer shops, e.g. 3rd party for profit businesses.
Lightning talks where individuals describe their networks.
Austin (TX) Wireless - hotspot service; wanted free Wi-Fi; customers incl coffee shops that compete with Starbucks.
Austin Wireless - other things to help venues, e.g. splash pages
Austin Wireless: splash pages useless if not totally relevant for the customer: Weather, sports, local info, Facebook links, venue.
Austin Wireless: Now making $ by providing marketing services to their venues, restaurants, coffee shops, etc.
Austin Wireless: Also does Chimpit which brings venue portal even if you're on another network.
Chambana.net joint project of IMC and Acorn IT collaborative - ~6 active people and 9 servers. Walk-in computers, media prod. lab.
Chambana.net host websites, many other IT services, but also some wireless! Small network, switching from NetBSD to OpenWRT.
Chambana.net: It sounds like their wireless net is secondary to what they do, and is currently down and being reworked.
Chambana.net: Wireless network is completely open, no splash page. Runs on donated hardware
Chambana.net runs on donations, but entire uplink is one Comcast business service. But part of a stimulus grant, so near future good
Tribal Digital Village: 19 tribes HQ in San Diego CA, only way to get Internet to reservations. sovereign nations within US, but...
Tribal Digital Village: 350 miles of P2P and P2MP links mostly license exempt; 18 bldgs
Tribal: Paid NW in parallel reaches 200 individual homes. 2/2 Mbps for $34.95 per month. Hope to reach 2000 of 2700 homes.
Tribal: 1K devices connected to net. Fiber at headend feeds an arc of P2P links. Now many gamers accessing via local centers.
Tribal: Too small and separate to think about running their own radio regulatory regimes. But not too worried about conforming :)
Djursland http://www.diirwb.net/ Rural area in Denmark; 95% can get DSL (fr 1600 exchgs). Last 5% not svc'd.
Djursland P2MP design
Djursland: NW built by volunteers
Djursland gets favorable Interent transit because the farmers drove (& control) the fiber backbone deployment.
Funkfeuer: mesh using OSLR; got help from jaap@scii.nl & friends in Berlin.
Funkfeuer: 240 roofs; financial sustainability based on hosting center revenues; Gbps uplink
Funkfeuer: slightly below mkt for hosting; TV streaming experiments; running Wi-Fi for events.
Funkfeuer: Fiber splicing is easy, if you have the expensive machine (~6000 Euros). Basically welding glass.
Funkfeuer is a closed user group. This is important for VoIP and other regulatory issues, including streaming TV to you members!
Funkfeuer Graz: like NW in Vienna; Technical Univ in Graz helped them grow fast; slower growth >2007 as 3G faster.
Funkfeuer Graz: now working to link with Bratislava (Slovokia) which could be the first international community network in EU.
Funkfeuer Graz: longest link today is 30 km which gets them 1/2 way to Bratislava (Ubiquiti radios)
Malcolm Matson knows of two networks connected (quietly) btwn Solovakia and Hungary.
Belgrade Wireless: up to 20 km links (> 50 Mbps) using corner antennas as shown in previous conf. NW now extends over 100 Km.
Belgrade: 3D corner antennas invented by Prof in Belgrade. Big focus on community events.
Belgrade: 3D Corner Antennas http://su.pr/AjXm1p - feeder for dish reflector - mixed polarization
More 3D corner antenna info here  http://su.pr/7t9990 Need to follow up on whether this remains relevant w/ MIMO using polarization.
Updates from the OLSR-NG Project - Henning Rogge (FKIE) & Aaron Kaplan (Funkfeuer.at) - history, today & futures.
OLSR-NG: 1 of 2 major mesh stds (other AODV). RFC 3626. Tonnesen PhD, Lopatic LQ & Fisheye extensions
OLSR-NG working on OLSR. Guys in Berlin starting over (BATMAN); HSLS hazy sighted link state (CUWin).
OLSR-NG session: CUWin HSLS didn't get beyond simulations
OLSR-NG: OLSR is link state - every node knows whole graph (100K entries = 4.8 MB); but MPR now off
OLSR-NG: MPRs only matter with really dense NWs, but with only 2-3 links per node, they don't pay.
OLSR-NG: ETX link quality metric used instead of basic hop count, i.e. sum of ETXs not sum of hops. But heavy compute load!
OLSR-NG bringing down compute load (now linear not exponential). 100-to-1 benefit with 400 nodes. Still Dijkstra, but optimize data.
OLSR-NG: malloc() thrashing fixed
OLSR-NG futures: soft refresh (CSN), better metrics (ETT, MIC), multipath routing (experimental), Q of layer 2 capabilities.
OLSR-NG: Henning comments on how few multi-path routing projects have made any progress... Problem: must choose whole path. very hrd
OLSR Henning: OLSRd 0.6.0 is current. Clean rewrite of routing code; smarter gateways to reduce thrashing btwn GWs.
Henning: 0.6.0 has very few (& only site-specific) bugs - very stable!
Henning on future plans: telnet/http server (done); config mgmt (stability, flexiblty.
Henning also thinking about better metrics but limited by packet format prior to OSLR v2. Many metrics in discussion in academia.
Henning responding to Q: negative about dual protocol mode to support migrations. Thinks it wold be very hard.
Henning - active developers= ~1.5 people
Henning complaining about academics who've made patches w/o consulting (thus doing stupid things) and without plans to give back.
Henning & Aaron on how plugin system makes it easy to try new things in a clean fashion.
Freifunk Berlin started 2002; then OSLR in 2003-04; 28 devices @ 2004 OS conf; PC to openWRT for embedded.
Freifunk took off in 2005 despite labeling website ""OSLR experiment"" but users wanted reliability
Freifunk can't even switch to B.A.T.M.A.N. because OSLR widely deployed
Freifunk net is now shrinking, as people who came only for bandwidth are getting DSL and 3G mobile.
Freifunk has issue of switching gateways which doesn't affect Frunkfeuer with their fixed gateways and public IPs.
guifi.net is a bunch of communities, not all interconnected. 10,300 nodes using same software, same tools and same license.
guifi.net is showing off an impressive set of tools for examining nodes, plus there's extra data that the node owner can access.
guifi.net Using MRTG http://oss.oetiker.ch/mrtg/
guifi.net is a large wireless LAN. Must search for & connect to services, like Internet access. Libraries & other offer Inet-GWs.
Île sans fil network in Montreal  http://www.ilesansfil.org/  Now 200 hotspots installed, free Internet thru portal page.
Île sans fil limits users to 7 GB/wk. 150K users registered. Each hotspot has own page. Projects: Authpuppy, WiFiDog
Wireless Toronto - captive Wi-Fi portals, started with Wifidog from ◊le sans fil.
Wireless Toronto has struggled compared to Montreal. All volunteers. Hotspots, but now adding mesh, e.g. in parks.
Wireless Toronto - no spt fr Government who started and sold a parallel NW. Hard to find location-based content for portal pages.
Wireless Toronto survives on annual fees from businesses with portals; organized as a club, not a nonprft
Wireless Toronto new mesh started with BATMAN but found OSLR more reliable. Using Open-mesh but moving to Authpuppy.
Wireless Toronto - the less you talk about Wi-Fi and the more you talk about mktg, the better you do selling business hotspots.
Wireless Toronto using AutoAP in part to just monitor what's going on, wirelessly, in their neighborhoods.
Open Wireless Networks http://consume.net in London
Consume fell out of use ~2003
OWN: nodes all clustered in London near Greenwich Park. Not 400 nodes
Good line-of-sight planning tool:  http://www.heywhatsthat.com/
Village Telco http://www.villagetelco.org/about/mobiles have brought telecom to Africa, but not Internet, yet...
Village Telco: it didn't take off
Village Telco: Built ""mesh potato"" - solar mesh device w/analog phone adapter; production units next month.
Village Telco: Mozilla is filming the project.
Village Telco: Retail cost $119
Wlan Ljubljana: http://wlan-lj.net Can't beat widespread fiber at $14/month, but each has excess capacity and willing to share.
Wlan Lj now up to 50 nodes & adding rural areas - now wlanslovenija! http://wlan-si.net entirely volunteers
Wlan Lj is clearly a group of hackers having fun, but it's not clear to me if they are really serving a need. Sustainability???
Wlan Lj has done solar nodes - 24 hr reliability but froze during the winter and wasn't restored until spring (cold on roof!).
Athen Wireless Metropolitan Network (AWMN) strtd 2002 because no DSL; open experimentat WLAN.
AWMN has some people who offer Internet access, but it's not the primary goal. Participants tend to be Univ types - young, educated.
AWMN mostly at 5.4 GHz with Linux and MikroTik routers. Islands of OSLR connected by BGP; 2505 nodes; 1100 backbone nodes.
AWMN speeds vary 11 Mbps to 150 Mbps. 730 access points. Organized by an association; events; community; no grants.
AWMN Recently, large deployments of 11n
AWMN As a local LAN, they have mirrors of many Internet services, also transliterated version of Google (Woogle), Yahoo, etc.
Richard MacKinnon of Austin Wireless follows me in the Freemium session. AW strt'd as all free, but wasn't sustainable.
MacKinnon: Austin Wireless used automation to replace 50 volunteers with 2 fulltime staff. Merchants pay to provide free access.
MacKinnon: Pull together local news for hotspot portal pages
MacKinnon: Used to charge $5/mo
MacKinnon: Integrate Facebook into portals, incent patron's to talk about the business they're visiting on their FB page. Biz value!
MacKinnon: installations were fun at 1st
MacKinnon: $55/mo buys support. Restaurants hate Wi-Fi but have to have it to be competitive. So support is key - ""power cycle box!
MacKinnon: deals with cable company; POS crdit crd installers.
MacKinnon: Ads didn't work but local ads may be coming back.
Nemanja Topovic, Belgrade Wireless, Serbia started as all volunteers. Had problems with Government (spectrum laws).
Topovic low cost svc didn't work (1 Euro/mo) as people expect full svc. Discusses many paths they've tried, unsuccessfully...
Topovic - network grew rapidly 2004-2006 but growth has stopped. Looking for a program that could restart their network.
Topovic: BGWireless not a mesh, uses high speed P2P and P2MP. Closed network.
MacKinnon suggests his biz mode for Topovic. Key is offering free svc to avoid spt issues and then find a premium svc to cover costs
MacKinnon - important to offer new paid service as something new, not as a price increase on old service.
MacKinnon: 200 customers today (and 1 & 2 yrs ago) but different group & paying more & more loyal. Free customers were least loyal.
MacKinnon user community divided: some just want Internet
Paul from NFP? talking about SW defined GNU radio work going on in the building. Available for discussions later...
Robin Chase, Meadow Networks (previously founder of Zipcar) will be evening keynote spkr - next up.
Robin Chase on tie-in of transportation and networks: financing (fuel tax moving to road tax eventually - per km!).
Chase Auto density in cities (congestion) but expense of rural rds. Moving to congestion pricing. Will need more technology...
Chase on transport costs not reflecting true costs (fuel/environment, etc.)
Chase Public-Private Partnership discussions are missing the individual. Transport tech has decided they need their own stuff.
Chase: transportation guy gets his spectrum; EMS/medical types need their own stuff- it's just comms
Chase: Transport problems: Lumpy density, congestion, financing, right pricing same problems as in comms infrastructure -> dist. NW
Chase: future for transport and comms is distributed networks
Chase focus on distributed and collaborative inputs as a path to innovation.
Chase: words to use when talking about Gov. spending: open data; open standards; open source
Chase talking about Comuto (ride sharing in France)  http://www.comuto.fr/ now has more unique users than ZipCar (& just in France).
Chase: talking about CouchSurfing: 7 yrs old; 200 countries, 71K cities; more ""beds"" than major hotel chains.
Chase on Chatroulette - built it in 3 days; but in 6 months it now gets 30M unique visitors - mindboggling!
Chase: Andrey could do ChatRoulette because platform (Pcs, Internet) was there and there was excess capacity available.
Chase on all the crazy ideas that are now iPhone apps and yet the excess capacity has fostered some incredible innovations.
Chase: People & Platforms -> Speed & Scale
Chase wants open mesh device in every car that will have to be paying congestion fees, getting traffic data
Chase wants an open platform in cars so we foster new apps we can't can't even envision now.
Chase: what if all nodes (smart grid, smart cars, smart infrastructure) were peers? and open platforms!
Chase: www.networkmusings.blogspot.com & @rmchase & rchase@alum.mit.edu
Vic Hayes of TU Delft University: Spectrum Assessment for Wi-Fi. History of FCC & license exempt spectrum and Wi-Fi market.
Hayes: FCC landmark decision 1985- license exempt; use more spectrum than required; spread spectrum tech.
Hayes: Standard CDMA history including Hedy Lamar's patent, but I learned something new - she was born in Vienna!
Hayes has some good slides to explain CDMA. I'll probably stick with my standard slides http://su.pr/1LMCYl
Hayes: FCC wanted to allow spread spectrum over wider bands but got objections so they settled on ISM because no one cared.
Hayes on attempts to get similar rules thru CEPT - succeeded in 1991 for 2.4 GHz band only and with slightly different rules.
Hayes: CEPT said only -10 dBW ERP (100 mw) and 10 mw per MHz
Hayes: because of cost of electronics, 900 MHz took off 1st (in 1989)
Hayes on how Wi-Fi 11 Mbps beat HomeRF even though the FCC permitted wideband hoppers (1998-2000). Wi-Fi faster
Hayes: 2002 FCC permits intelligent hoppers (reduce blutooth intf) & power spectral density rule (opens the way for OFDM, i.e. 11g).
Hayes: Now 5 GHz - FCC NII proceeding - Apple, Lucent, etc. release 1997. Adds 5 GHz spectrum.
Hayes: meanwhile CEPT yielded to Satellite industry and reduces pwr at 5 GHz, but add more spectrum for HIPERLANS (455 MHz).
Hayes: CEPT decides to go to WRC 2003 to make 5 GHz primary and global. US problem because NTIA refused. Took until Jan2003 to win.
Hayes: In June03 WRC 2003 allocates 455 MHz co-primary in 5 GHz band. Accepted in US & EU - still in flux in many countries.
Hayes: Once spectrum is allocated, it still must be defended (find reasons in style with current political agenda).
Hayes promoting his upcoming book: The Innovation Journey of Wi-Fi, Edited by Vic Hayes et al. to be published by Dec2010.
Clarification of 1st unlicensed spectrum was 1937 for baby monitors, etc.
Hayes: Questions about Ad Hoc mode
Rabi Karmacharya & Basanta Shrestha of OLE Nepal using wireless to deliver Internet connectivity to schools in Nepal
OLE Nepal: Initial focus (2006) was OLPC, but if you got PCs, there was still no Internet
OLE Nepal: Don't have to teach kids how to use computers - that comes naturally - but need to create content in local language.
OLE Nepal: Teachers have to be retrained, and need to realize the kids may know more about the PCs that they do.
OLE Nepal: Need network infrastructure, in schools and between schools and the Internet.
OLE Nepal: Open source, open content
OLE Nepal: 28 people on staff
OLE Nepal: First priority is a network to and btwn schools as Internet upstream is very expensive in Nepal.
OLE Nepal: One highway with fiber runs length of country in the southern lowlands (near India border). Northern areas to 8K meters!
OLE Nepal: fiber line is connected to India so Internet transit pricing has come down substantially, but still very expensive.
OLE Nepal: approaching 38 schools in six districts & 4K students connected. P2P WLAN w/ typ. 5-10 Km distances
OLE Nepal: ADSL where available, used with VLAN.
OLE Nepal: On the plains, need to find a hill to act as relay point. Sometimes >20 km links needed (example 26km).
OLE Nepal: In hilly areas, may need 3-4 relay points at remote sites (4 hr walk)
OLE Nepal: School NW have: radio; switch; & 2-3 Wi-Fi Aps; all w/UPS. Select equip. for low power! Linksys WRT54GL w/DDWRT.
OLE Nepal: MikroTik 433 AH & 411 CPE - have used this up to 26 km. Also EnGenius radios, but not reliable.
OLE Nepal: SW tools: Google Earth, Radio Mobile, mirror tst'g. Use trees as towers (cut away foliage).
Ben West of Wasabi Networks (St. Louis) has posted his Community Wireless Day 1 notes here:http://su.pr/3FG39a
Ben West's Day 2 notes include notes from the session I spoke in: http://su.pr/1DHUFi
There's a big EU network that's not here. It's Czech Freenet: http://su.pr/2qMyeE 
Whoops - and this Czech Freenet: http://su.pr/24yYU8
Future for Community Wireless Networks session lead by Aaron Kaplan and Vic Hayes. EU wired net getting better and CWN shrinking...
Future of CWN: ideas fr. grp: underserved rural areas; hacker grps want to stay that way.
CWN futures: my categories: hacker comm, social comm & svc provider. All NWs have some of each, but one tends to be primary.
CWN futures: several people object to categorizations (in general?)
CWN futures: Movement to integrate node databases fr. many diff. networks. Exchange knowledge on making node db's. Mtg in <6 mo.
CWN futures: A world node database would create a larger vision of the movement, espc. for members in individual networks.
CWN futures: some desire for turnkey node solution prompts objection from Berlin hacker community that people would stop learning.
CWN futures: Ramon (guifi) sees evolution in the discussions over past 4 yrs: more consensus, meta focus (e.g. world db).
Networking rural areas: Rantanen - Tribal NW mostly done with grant money.
Networking rural areas: Rantanen offerring his cast off equipment to Krusevac Open
Networking rural areas: Krusevac network is technically illegal under Serbian law (ISPs must be licensed w/many problems & cost).
Telecom for disaster relief - Mark Summer of Inveneo talking about wireless in Haiti after the earthquake -http://www.inveneo.org/
Mark Summer: Mountain range around Port au Prince was key
Mark Summer: so many Sat Phones came into Haiti that satellite capacity was saturated.
Mark Summer: got Ubiquiti to get stuff from distributors (in 3 days)
Summer: Kitted everything over a weekend; VSAT vendor committed 5 Mbps link if dish fixed.
Summer: Google got very high res imagery on-line within 48 hrs and updated it every few days. Open Street Maps for former streets.
Summer: got VSAT up in <24hrs
Summer: Got 3-4 radios in per day
Summer: Linked 18 NGOs at 23 locations in 3 wks; NGOs didn't want it turned off, as better than pre-disaster.
Closing keynotes just starting. Tomorrow: open spectrum alliance
Aaron Kaplan thanking many, many. Node DB SIG being set up for this coming winter.
Sascha Meinrath asking for ideas we should ponder over next yr. Answers: 1. OS physical layer (GNU radio?) - seeking FPGA designers.
Answers 2) Allison Powell (@postdocal) learned we're still not good at telling stories. She's seeking people to interview.
Answers - Rabi Karmacharya (OLE Nepal) very struck by the discussions of biz models and how NW can help in disaster recovery.
Audience comments: worried that several major networks are loosing nodes
Audience comments: Session on splicing fiber was exciting. Community radio + community fiber (+ community satellite?) !
Audience comments: Credit to Matt Rentenen for offerring to pass on older equipment. Sascha suggests an email to CWN list.
Audience comments reinforcing idea of using Wikipedia as the master list of Comm Wireless NWs + spectrum laws http://su.pr/2C3YFz
At OSA Mtg http://su.pr/2AtH5O 1st formal mtg took 1.5 hrs as bylaws are in German & English. Hopefully we'll talk spectrum soon.
OSA Mtg: @postdocal asks OSA view on net neut & suggests links with other advocacy grps. Open a rathole? Sent for email discussion.
Now in GNU radio discussion. Ah techies, > interesting than policy talk.. Getting into to USRP http://www.ettus.com/ which I already know of.
Paul Fuxjaeger on GNU radio: OFDM for 11a, 11g required substantial mods to GNU radio blocks. Also hard meet IEEE timing req.
Fuxjaeger, GNU: For 2x2 MIMO, externally sync 2 USRP2s (not w/Ettus cable!) so as to maintain 2 Gig data streams, vs 1 w/Ettus cable
Fuxjaeger: http://www.oz9aec.net has interesting GNU radio stuff, also an update on Gumstix & GNU radio! More shortly...
Fuxjaeger: http://su.pr/A899kx has the pointers for Gumstix and GNU radio
Fuxjaeger: http://su.pr/1TZZeQ is the PR for Gumstix's product Stagecoach which packages TI OMAP processors that may be used w/Ettus
Kaplan wraps up the GNU radio session w/demo pasting macro blks & receiving signal from a Ham radio in 70cm band, i.e. 420-450 MHz.

Good conference Wi-Fi

Thanks ISWCN!

 

That's especially nice as it let me upload my photos from yesterday afternoon's stroll around Vienna -- all completed in a couple of minutes while waiting for the conference registration desk to get set up.

 

International Summit for Community Wireless Networks

I'm off to Vienna to attend a conference.  Conference info is here.  On Friday afternoon, I'll be leading a session in which I hope to get useful feedback on the "freemium" business model we're using for netBlazr.

Europe has a number of community wireless networks that have been very successful.  The networks in Vienna, Berlin, Athens and Catalonia stand out in my mind.  Of course, in the US we've had active legislation that's made it hard for governments to create community network.  But the EU networks mentioned above are member-based.  They may have benefitted from government funds at some point, but they formed independent of any government effort (at least as I currently understand things).  The other issue I've observed in a number of US community networks is a reliance on grants that aren't renewed and/or on enthusiasts who eventually leave to go to grad school or otherwise move on.  Something is different in the EU networks mentioned above.  Hopefully I'll better understand this by Sunday.  :)

As is my habit, I'll be taking notes in real time using Twitter (@brough).  After the conference, I'll gather those notes into a single page for a blog post.  This is primarily for my own benefit, as it's been an extremely useful way for me to take and save notes (assuming it's an event like the ISCWN where I want to keep notes!).

AT&T Wireless still struggling with the basics

Has AT&T gotten better since I last wrote about their network performance at some length?   Maybe not ...

 

Comcast: Use your bandwidth and we'll throttle you

Recently a friend decided to use an on-line service to backup his 750G raid array. He selected CrashPlan which, among other things, generates a projected completion time. With his existing DSL service it was going to take months, so he signed up for Comcast Internet with a plan that offered "up to" 10 Mbps down and 2 Mbps up. As he put it:

... when I was signing up for Comcast I specifically asked about high-volume on-line backup, and in particular whether it was Ok for me to buy the 2x10 service and run a heavy duty upstream job 24x7 at 1 mbit, and I was reassured that this was OK.

To see what happened, look at the "wan-out" statistics in gray above. Comcast came on-line in late March and CrashPlan was configured to use 1 Mbps of the "2 Mbps upstream" bandwidth. This worked for less than two weeks, then Comcast throttled the upstream to less than 600 Kbps.  Over the next six weeks, he was further squeezed to ~400 Kbps. Nothing in his house changed during this period, so the data above reflects Comcast's performance. As my friend puts it:

I guess what they meant was "... it's OK to try, since we can rate limit you ... to anything we want, you won't bother us by trying.  You won't get your bandwidth either, but that's not our problem. "

Comcast:  "Up to X Mbps," but only for occasional use.

Commercial use of TV White Spaces in 2 years?

One of the interesting discussions at the Wireless ISP Association meeting in St. Louis last week was around TV White Spaces.  Alex Goldman has a good summary from the WISPA point of view.  We also heard from Julius Knapp of the FCC during lunch on Thursday.

What's clear:

The FCC is going to act, in the 3rd quarter (before October 1st), on the 17 petitions for reconsideration that are pending.  While the details of their decisions will have to wait until their announcement, it's likely the final rules will allow viable commercial markets to develop.

At the same 3Q meeting they are going to adopt policies that allow multiple independent database managers to compete.  (There will be a mandatory coordination function).

My conclusions:

Once the final rules are determined, we'll see both Wi-Fi and WiMAX equipment release - in both cases by re-banding existing equipment and adapting existing antenna technology.  Over time, other products should emerge, but rebanding existing standards will happen first.  We should see products that WISPs can deploy within 2 years.

My notes from the WISPA conference in St Louis

The following is based on my Tweet stream (@brough) during the conference plus some other notes I took off-line during the Tower session (when there was no Wi-Fi).  

Brief summary:  The conference was a great success, certainly for me, but seemingly for all.

Notes from the Wireless ISP Association (WISPA) meeting
St. Louis, July 21-22, 2010

WISPA meeting scheduled to start @8:45am (20 min) but room near empty. Sleeping late in St. Louis? Conf Agenda, see http://su.pr/24Zlii near bottom.

Twitter's location service keeps telling me I'm in Berkeley CA when I'm in St. Louis. Having to reset it makes the svc less than helpful.

The 40 Mbps wireless link set up for this meeting died in last night's lightening storm. Hotel has DSL? ugh. Hope radio get fixed!

Rick Harnish speaking: WISPA has signed up ~50 new members just because people wanted to come to this meeting.

Nice to finally meet Matt Larsen after many years of seeing him on email lists. Matt comments that WISPA membership now over 400.

Steve Coran, Doug Karl, Dewayne Hendricks, Patrick Leary visionaries panel is up, but 1st the ""Did you know, shift happens"" video...

Dewayne H started as a ham (age 12, 1961), then 56kbps packet radio in 1986; went commercial 1990 - wireless Internet, mesh, early!

Doug Karl started in IT as OSU, needed Internet for off-campus folks and started P2MP wireless net.

Patrick Leary started in fiber but focus is broadband Internet and that led him into wireless (initially in Atlanta ex-urbs.

Doug Karl became advocate for WaveLAN (from NCR) radios and their use for P2MP, then an advocate for early WISPs.

Dewayne Hendricks - the gating issue is regulatory. Original NOI discussed open spectrum, no power limits! Today rules are tightening.

Doug Karl commenting on having your town put extra requirements on cable company when up for renewal.

Dewayne's mission impossible plan for spectrum - demo alternate spectrum approaches in other countries (e.g. Tonga); existence proof!

Dewayne: reinvent, repurpose mass-market stuff - Wi-Fi silicon, DOCSIS modems (use transverters!).

Dewayne is pessimistic about starting a new WISPA today. I have to have an argument with him!

Doug Karl: remember you are in the communications biz - wireless may be only part of that (don't be wireless only).

Now it’s hard to pick which of three tracks... Whitespace policy; 4G; or VoIPoW. Love policy issues, know 4G & VoIP but wish I had all PPTs.

Raja Gopal - Alvarion - pushing WiMAX. $5K for a 16e basestation today but admits in 2-4 years LTE will win. But deploy WiMAX now.

VoIPoW discussion justified by how much money can be made with voice, but need QoS and high level of service - must be dependable.

TV White space discussion: politics! politics! chg sensing requirements; still large antenna issue.

Marlon Schafer asks about eventual pollution of TVWS by home LAN devices. Jack Unger on current interference mitigation discussions.

another 3 tracks decision: Mktg; FCC's 3rd way regulation; or Fiber deployment. But the only Wi-Fi access in the Mktg room. :(

Mktg Elizabeth Bowles loves on-line but (on behalf of Forbes Mercy) pushing radio (PR & Ads) for rural WISPAs; also yard signs!

Mktg: Ken Janc, Lorex Inc, on targeted direct mail and door-to-door - expensive but can be cost effective (cost per acquisition).

Mktg: Ken Janc - at least have your installers do door hangers on next door houses when doing an install.

Elizabeth Bowles of Aristotle.Net on SEO, pay for advice if you are not found via ""broadband in <your location>"", get prof advice.

Elizabeth Bowles big proponent of video on your website - dramatically boosts response rates.

Elizabeth Bowles: mobile phone enable your website. Remember smart phones with Wi-Fi access. Unlimited mobile data is dead, so people using WiFi.

Elizabeth Bowles: need a Facebook Fan page! Twitter account; LinkedIn; YouTube, Flickr, Yelp, etc. Track and respond.

Elizabeth Bowles: Optimized press releases (select key words to get coverage) is to get on-line coverage, not local press.

Martha Huizenga of DC Access a WISP in small DC neighborhoods: pick targets; then: postcards, signs <cars, yards>, local newspapers.

Martha Huizenga postcards are the only printed media; leaves them at dry cleaners & other local stores. Rest are PDFs by email/online.

Elizabeth Bowles responding to a Q: Market Wire costs $400-$700 per release; so search optimization your release first!

Elizabeth Bowles answering Q: need 3-4 touches to get a response. Link sign, postcard, website landing page to drive touches.

Michael Calabrese of the New America Foundation is luncheon keynote - talk is focused on open spectrum and especially TVWS issues.

Michael Calabrese: big problem - Genachowski's focus on auctioning large parts of the TVWS.

Calabrese: FCC, Congress & WhiteHouse intent on auctioning everything, including 20 TV channels (after paying off TV broadcasters).

Calabrese: If 20 TV channels went, after repacking broadcasters, there wouldn't be enough TVWS for an equipment market to emerge.

Calabrese: Spectrum sharing still sailing under DC radar; he's pushing that TVWS database extend to additional bands (e.g. military?).

Calabrese: we should use the same database to allow temporary access to ""warehoused spectrum"" i.e. until spectrum holder uses it.

Calabrese: Why shouldn't WISPs have temporary access to any spectrum band that is not yet in use in their geography?

Calabrese: FCC is not nearly as aware of the WISP community as it should be. Pls visit them; write to congress; file ltr to NOI.

Network Management: Matt Larsen, Vistabeam with Butch Evans; Brian Vargyas, Baltic; Alex Phillips, HSLink; Cameron Crum, Wispmon.

Larsen favors open source. Issues: integrate related items, respect business (may have to fire specific customers), leverage tech.

Larsen on processes: Billing, Provisioning, Dispatch, NW Mgmt (core & customer), Support, CRM, CustSat, Mktg, Employee accountability.

Larsen: Vistabeam has 1700 miles of wireless backbone over 600 miles. Documentation is critical before someone goes out in the field.

Larsen: Billing Freeside, Powercode, Platypus, ... He uses Freeside.

Larsen: NMS - What's Up Gold, Nagios, Xymon; proactively call people whose service is down. Also RT for trouble tickets.

Larsen on phone system - uses Asterisk but also familiar with Tribox and others. Msg when there's an outage cuts support calls!

Larsen on documentation - uses Network View, Visio, Dude and a wiki to capture other things (e.g. details of every tower).

Larsen reevaluate your performance metrics regularly. Ditto for biz processes.

Brian Vargyas on NM Mgmt: Be proactive, not reactive; BW usage; link history - Dude, Cacti, Nagios. New Dude 4.0 (beta) on SQL DB.

Vargyas - Dude works with any SNMP enabled device, not just MikroTik devices. Any MT board can be an agent to offload central box.

Vargyas - Cacti is open src PHP-based graphing system with web GUI but no notifications. Looks at anything with SNMP OID.

Cameron Crum of Wispmon - started a WISP, needed an IT infrastructure and a way to monitor his network. Visual kind of guy.

Crum is focused on demo'ing the Wispmon product.

Butch Evans speaking on QoS and prioritizing traffic to improve customers' performance - all on MikroTik. Others use Imagestream.

Butch Evans QoS script runs at an aggregation point. QoS won't fix massive overload problems; will help schedule during congestion.

Matt Larsen uses remote power control by Digital Loggers - auto power cycle if no ping on time.

Butch Evans QoS script for MikroTik is $175. Uses IP tables; categorizes most traffic and assigns it priorities; updated as needed.

Butch Evans was trying to emulate NetEqualizer when he started on his MikroTik QoS scripts.

Off to a Tower Technology session but there is no wireless in that room, so no Tweets.

WISPA Tower tech panel - Walked after they had started so I missed the introductions - not sure who is who…

Seems everyone is using 11b or 11g as they regard selecting polarization as critical.

Jack Unger (and at least one panelist) believes narrower channels (10 MHz or 5 MHz) cause the signal to go farther.   Statement that narrower channels have less background noise to contend with, thus the signal goes farther.  Given OFDM, this doesn't make sense to me.  Need to investigate what’s really happening.

One panelist argues for regulated power as the power supplies in the MikroTik's are cheap.

One approach to power and UPS is to use 24v PoE with a central 24v DC supply and two 12v deep cycle batteries.

Everyone advocates UPS.  One guy has 8 hours for every site.

At least one guy tells his customers to buy APCs for the client site.  Tornado zone, take laptop to basement and track tornado in real time.

Weatherproofing for coaxial connections:

1. Courtesy wrap (white tape) protects connector from butyl layer
2. Butyl layer (a mastic like putty)
3. 3M Super 33+ electrical tape (3 layers, shingled down on last layer) to extend at least an inch past the butyl layer.

The Andrew website has good tutorial on weatherproofing antenna connectors.

Use DC4 dielectric grease to protect Ethernet connectors.  Note this is not clear silicon! (even though it looks similar).

Inspect everything at least once per year.

Just posted some pictures from the conference here: http://su.pr/2XQuqp  More to come (will be added to this set!).

Conf has 253 registrations and 40 vendors. Mtg will breakeven or be slightly profitable. 43 new members joined in past one(?) month.

Board Mtg.: I should look into the Ambassador's program... They're looking for members to represent WISPA at regional events.

CALEA update - WISPA spec needs 3 updates to meet new rules; complex negotiation btwn FBI & WISPA. Next issue up is IPv6.

FCC Committee efforts: TVWS (not auctions); TDWR Database (stay 30 MHz away) - problems in Boston & PR); 3650 MHz (4 rule changes).

FCC Comm spent $5K in June, slightly lower than average; but advocacy and meetings are the bulk of what WISPA spends $ on.

Discussion of FCC-specific, or issue-specific, fund raising + volunteers to help engr'g (versus raising dues). Also partnering!

Steve Coran pitch at Marlon's request. (Rini-Coran is WISPA's law firm for DC politics).

Several members suggest that FCC Comm provide boilerplate that members can use to send letters to their representatives.

An aside in my twitter stream, not directly related to the WISPA meeting:
Where do telecom lobbyists come from?  There's a fantastic graph about half way down this page: http://su.pr/1tjZnZ    Hint: they used to work for congress!

Peter Stanforth, CTO, Spectrum Bridges up next on TVWS Trials. First focus on 2ndary mkts - time of day, location, freq. >10yr effort.

Stanforth has nice graphics on what TV white spaces are and how they arise. Hope to get his slides, eventually.

TVWS database administrator. Stanforth suggests we can have competition even here, and still accomplish desired results

Stanforth Summary of trials' field results in 04-186 FCC proceedings.  http://su.pr/2F9dOx

Stanfort: All TVWS trials have worked; getting 3x-5x coverage vs. 2.4 GHz systems; no interference complaints!

Stanforth: sensing unnecessary; drop antenna height limits; use fixed power limits & masks. Make DB higher fidelity than orig req'd.

Stanforth running TVWS BB antenna next to TV antenna in trials. Don't need separation! Must relax power limits and masks!

Stanforth notes that many TV stations are spraying out-of-band energy (a license violation!) but they haven't cared (so far).

Stanforth: if there are multiple database admins, they will coordinate btwn each other once each night, at least as proposed so far.

Rob Kubik of Motorola talking about FAA radars (TDWR) and DFS as basis for renewed outdoor use of 255 MHz of 5 GHz spectrum.

Kubik: experiments show 30 MHz away is ok, even in close (<35 km) to radar; combined w/DB approach should solve problem, but FCC..??

Jack Unger showing TDWR database which WISPA & Spectrum Bridge did jointly - going on-line next week. Meanwhile  http://su.pr/1Xk1SC

Stanforth showing prototype of TDWR database that will be up next week. Currently at http://su.pr/2iHAdO although that will change.

Panel on Buying and Selling WISPs. Prices seem to be from 1x to 2x revenues but many factors. Based on buyer estimate of cash flow expected after the transaction.

Across the board, panel is really cautious about any deal that involves stock or deferred cash. :)

Shiraz Moosajee's advice to buyers: plan on 20% reserve to fix problems and be sure you have 20% beyond that just in case.

Buy/Sell panel: 90-180 day clawback or true up clauses are common, with 20-25% in escrow.

Buy/Sell panel: Even if you don't plan to sell anytime soon, while growing your biz, keep revenue history, per tower.

Buy/Sell panel: Even though everyone prefers cash, very few deals are all cash (and they were fire sales).

Buy/Sell panel: Re buying small neighboring WISPS (100 subs) - frequently done with seller financing over 2-3 years. Estimate growth.

Buy/Sell panel: The smaller the network the less info you are liable to get!

Buy/Sell panel: Need transferability in all your lease agreements as buyers want assets (sellers prefer stock sale for Cap gains).

Buy/Sell panel: Sellers frequently have personal guarantees associated with corporate accts - got to know how to unravel these.

Julius Knapp  http://su.pr/9BXRJs  FCC, lunch keynote: Update on spectrum activities, but starts by hyping BB Plan.

Knapp on spectrum: ack's spectrum that's not yet built, but still pushing FCC/mobile operator's line: need more (licensed) spectrum.

Knapp promoting: spectrum dashboard; incentive auctions; more spectrum for mobile BB & for Backhaul - but this w.b. licensed. Ugh!

Knapp - only items that resonate with me: more unlicensed (Rec 5.11) and opportunistic use of spectrum NPRM (Rec 5.15) - Q3-2010.

Knapp & Ruth Milkman co-chair FCC Spectrum Task Force. Good, but he doesn't mention that FCC only controls part of the spectrum.

Knapp - Pres memo is forcing FCC & NTIA to work jointly on 500 MHz of new spectrum & Pres used words "licensed and unlicensed.

Knapp on TVWS 17 petitions for reconsideration. Expects multiple database mgs to be approved as part of this action.

Knapp acknowledges that 3650 MHz lite licensing has been a significant success, even as there is work afoot to improve things.

Knapp - TDWR protection w.b.: avoid freqs & 30 MHz guardband within 35 km. Make sure this works, so further licensing isn't required.

Knapp: the proceeding on backhaul scheduled for August 5th commissioners' meeting.

Knapp Q&A on Presidential memo suggests that ""licensed & unlicensed"" means some mix. How that's divided is to be argued...

Knapp says Aug 5th mtg will result in increased flexibility in use of spectrum for backhaul. But he wouldn't give details.

Knapp defends existing build-out reqmts. Charles Wu asks about Pt 101 licensing. A: spectrum transparency point is to find squatters.

Ray Savich of Moto describing software tools they have to help WISPs plan their businesses. These are biz tools, not just RF stuff.

Savich on ""Leased line payback calculator"" another tool to help sell WISP services.

Jay Lawrence of Gigabeam showing wireless carrier in Las Vegas who's built a wireless carrier Ethernet network using >50 1Gbps links.

xG Technology is here. Have dropped their original magical claims http://su.pr/1fBB9d for some completely conventional RF technology.

xG now has a proprietary mobile voice solution that runs on unlicensed spectrum. Not interoperable, but more a community cordless solution.

Cameron Crum of Wispmon demo'g their WISP-in-a-box IT solution including qualifying prospects against towers & terrain map. Very cute!

Crum emphasizes taking & preserving pictures of every install indoor incldg where power is connected (tell cust. how to power cycle).

Robert Olive of Wispmon is now pushing their new iPhone apps that work with Wispmon.

Nathan Stooke of WisperISP started in 2003; took a yr to breakeven; 3 yrs to make a salary; today, $2.4M revenue & 20 employees.

Stooke focus on best network and best customer service; has since taken over 15 competitors (many evolved from dial-up days).

Stooke: competitors were dial-up ISPs who entered wireless reluctantly. Diff was he wanted to build a wireless NW and he focused on svc.

Stooke: struggle to find good people; hirer slowly, carefully; some start as contractors, become employees later. He doesn’t provide benefits...

Stooke: employee training espc customer svc - one full day a month devoted to on-going training.

Stooke: run it as if you were setting up to sell it, i.e. keep it clean, fix problems, don't let them linger. Make it a showcase.

Stooke: Has a fire truck with 100' extension ladder that he got for $1K (but spent $26K getting fixed & certified) = temp tower.

Jim Murphy of NetSapiens describing their VoIP solution for WISPs - scales up, also down to system for just 10 simultaneous calls.

Murphy positioning against Asterisk (lower opex, expertise) and hosting (facilities based, more control; potentially better QoS).

Murphy also offering local virtual switch where realtime elements are local but NetSapiens handles everything else.

Now I have to leave to catch a plane, so I'll miss the final prize drawings. :( But this has definitely been worth it!! 

Thanks WISPA!!

WISPA - Wireless ISP Association meeting in St. Louis

I've just landed in St. Louis for a WISPA regional meeting that sure looks national to me.  Of course, I'm from Boston but the first person I talked to was Jim Murphy, COO of NetSapiens who had just flown in from La Jolla California. 

I'm new to the wireless ISP business but it was nice to see several familiar faces in addition to Jim (who I know from my VoIP days in the late 90s), for example, Alex Goldman, formerly of ISPCon and now writing for WISPA and Donny Smith of Jaguar Communications who I had met at FiberFete in April.

During the conference I'm be taking notes via twitter:  @brough

Word use as a key to US Broadband Plan

The inestimatable Frank Coluccio has finally started to read the US National Broadband Plan but, before starting, he did a very interesting document analysis.  He counted the number of times various words and phrases are used.  Here's a suggestive subset I picked out from Frank's more detailed document.

Commons - 1
Symmetric (not "symmetrical") - 1
Backbone - 1
Line of sight - 1
Gigabit Ethernet - 4
Dark Fiber - 9
Wi-Fi  - 10
FTTH/B/C/P (combined) - 20
Cellular - 31
Backhaul - 39
DSL (all types) - 40

TV - 122
Video - 201
Rural - 266
Wireless - 377
Mobile - 397
Internet (all contexts) - 453
Government - 669
Spectrum - 715
FCC - 1439

Stealth no more! Also, I need your vote

A few of you have commented on my various profiles that say I'm associated with Ashtonbrooke Corporation or with BigBroadband.Net.  Of course neither website provides much information...

Well our service launch is still weeks away and we haven't found a name we like yet, but we have entered the MassChallenge and that means we've revealed what we're up to. MassChallenge is running a competition for startups and they're giving out grants.  Since we are entirely self-funded so far, a grant would be sweet.

To see our entry you must register at the MassChallenge website, http://www.masschallenge.org/user/register?role=Supporter and then go to Competition, Team Pitches, search for "BigBroadband.Net" and click Details.

Once you are there, you can not only get an idea of what we're up to, but you can vote for us!  Ideally before midnight tonight.  :)

Please Vote for BigBroadband.Net

As a registered supporter of MassChallenge you can rank any team's entry.  Just click on the right most of the five stars under "Ranking" on the right of our details page to give us a 5 Star/Awesome rating, ideally by midnight tonight.

Thank you.

Longer term..., we're seeking a new name

Once we roll out our pilot network (in Boston later this summer), I'll have a lot more to say about how we expect to pull a wireless end run around the duopoly, at least in dense urban areas in the US.

Meanwhile, we are seeking a better name. WirelessEndRun (our internal name) is either brazen or obtuse, or both.  BigBroadband.Net says something about what we're doing but has a lot of negatives.

All suggestions gratefully welcomed.

Audiocast tomorrow: Satellite phones, spectrum games and more

Tomorrow, Tuesday the 15th at 11am eastern time, I'll be live on an audiocast organized by Carl Ford of 4GWE and TMCNet.  

The title is "4G is in the Stars: Satellite to Terrestrial" and speakers include Barlow Keener, Lyman Chapin and yours truly.

The prize is 68 MHz of spectrum at 1.4 GHz.  In the US and Canada, this spectrum has been allocated for Mobile Satellite systems but it can also be used for Ancillary Terrestrial Services - that is services that are ancillary to the satellite services! This spectrum would be good cellular spectrum if you could get it, being above the 800 MHz original cellular bands but below the 1.9 GHz PCS bands. Technically, the satellite signals go up and down while the ancillary signals go mostly horizontally, so they are relatively independent. The big question is: Can anyone afford to launch satellites and run satellite services just so they can also use this spectrum for LTE services at ground level?

Gigaom has been skeptical, but when you stack up the estimated $5B to build a network against the possible value of this spectrum at auction, i.e. 68 MHz at US$1.2 per MHz-POP (the price paid in the US 700 MHz auctions) and 335 million POPs in the US & Canada you get roughly $27B. That math could work...

Join us tomorrow at 11am eastern by registering here.

Top cities ranked by Broadband Speedtest results

 

For those who are US-centric, note that MD is not Maryland, it's Moldova!

These results are aggregates of measurements made at Speedtest.net.  These and other results have been published by Ookla (the company that created Speedtest.net and Pingtest.net).  See Net Index for more results.

It's interesting that many of these cities are in eastern Europe, e.g. Latvia, Moldova, Romania, Bulgaria and the Ukraine.  Indeed, three of the top 15 cities are in Latvia.  I don't have current data on Latvia but I wonder if it benefits from the same relatively free access to rights-of-way that is present in Bucharest and several other parts of the world.

Africa moves from satellite to fiber links

2009-2010 marks an exciting transition for the Internet in Africa. 

Historically, most countries had only satellite connections or a connection to a single submarine fiber optic cable (SAT-3) controlled by an incumbent monopolist. Not surprisingly, prices for SAT-3 bandwidth in the African countries it serves were high (US$4,500–12,000 per Mbit/s per month, vastly more than than bandwidth prices in the US or EU).

But finally, things are changing, radically.  Beginning in 2009, the Seacom cable connected eastern and southern Africa with Asia and the TEAMs cable reached Kenya. Now four more cables (Glo One, Main One, EASSy and LION) are coming on line in 2010, with WACS and ACE expected in 2011-12. Steve Song has an excellent map and more details here.

Impact on Satellite services

Balancing Act Africa has a new market study out which projects African bandwidth markets through 2014. Among other conclusions summarized in recent letter from Russell Southwood:

The overall balance of satellite vs fibre use in Africa’s top Sub-Saharan markets 20 markets goes from 45.6% vs 54.4% in 2008 to 11.9% to 88.1% in 2014. Of these top 20 markets, the top 5 markets (South Africa, Nigeria, Kenya, Angola and Sudan) make up the majority of bandwidth demand across the continent and are the countries that will experience the fastest bandwidth growth over the period. So satellite operators and resellers will not only be selling into a smaller share of the overall bandwidth market but will become niche players in many countries. There will be growth but most of it will be on fibre.

After three and a half years since December 2005 in which only ten satellites were launched with coverage over Africa (four of which failed), at least 36 new satellites will be launched by the end of 2013, costing between US$4.4 billion and bringing the equivalent of 26,325 MHz of additional capacity over the region. By the time the second constellation of O3b satellites is launched, there will be almost as much Ka-band capacity over Africa as the total stock of C- and Ku-band capacity.

The satellite operators have invested huge amounts in new and replacement satellites. US$ 4.395 billion will be invested in new and replacement satellites against US£2.15 billion in seven international submarine cables, excluding ACE where the budget has not yet been announced. The satellite operators are investing at a time when the tide is turning against satellite use.

What are satellite operators thinking?

Broadband speeds - Boston and Bucharest

Recently I was conversing with a friend from Romania and we got to discussing consumer access to the Internet. Not surprisingly, connectivity in Bucharest is substantially faster than what I have here in the Boston area (even though I have Verizon FiOS service).  Anyway, rather than go by two isolated samples, I thought to look at the international records at speedtest.net to see averages of many, many measurements.

Download measurements

   

Notice that the first three measurements on the Boston side are business services, not things a consumer could afford. On the Bucharest side, Voxility is a wholesaler but #2 and #3 are selling to anyone.

Upload measurements 

   
 Yes, it's one rather specific set of measurements, but still...

City comparisons

The greater Boston metropolitan area has a population of ~4.5 million making it the largest city in New England and the tenth largest metropolitan area in the US.  Bucharest is the capital of Romania and the Bucharest metropolitan area has a population between 2.15 and 3 million. Bucharest is a bit older than Boston and today has a bit higher density, while Boston has somewhat more high rise buildings.  Still the cities are not so dissimilar that it would explain the wide divergence in speeds of Internet connectivity.

Structural Separation: Open for discussion, at last?

On April 1, 2003, I gave a keynote address at the Spring VON conference in San Jose.  I opened with some misdirection about a fiber project, implying it was near my home in the US.  The project, which delivered 100/100 Mbps fiber to the home for $24/month, in 1999-2000(!), was real enough.  But it was in Ulmea Sweden.  Even at that time, I had been talking for several years about the need for either: user control of first mile fiber or multiple competitors gaining open access to dark fiber in the first mile.

But whenever I used the term "structural separation" in the presence of US policy wonks, I was put down with comments like "Be serious. That could never happen in the US."  Indeed, while the idea is openly discussed in international circles, in the US, only fringe people mention structural separation.  The friendliest comment was from a communications lawyer acquaintance who advised me to avoid the words 'structural separation' if I wanted anyone to take me seriously.

So I downplayed those specific words but continued to advocate open access to dark fiber.

Good News!  The words 'structural separation' may no longer be verboten in the US.

While the US Broadband Plan avoids mention of any kind of structural reform, the topic is finally showing up in mainstream US discussions.  Harold Feld (Public Knowledge) directly mentions structural separation in his comments on the plan.  Esme Vos has slightly different views but is also "appalled that structural separation is not part of the Plan."  And when Paul Budde (admittedly an Australian) opened the topic, he prompted a vigorous discussion from Richard Bennett in his comments section.

A Long Way to Go

The US has a long way to go and the US Broadband Plan is not bold enough to get us there, but at least it is now possible to discuss a major issue in public in the US.  Let's be thankful for any step forward.

eComm Panel Discussion on the US Broadband Plan

At the Emerging Communications Conference in San Francisco next month, I'm organizing a panel discussion on the US Broadband Plan. Independent of my personal views, I've tried to get panelists with as diverse a set of views as possible.  Controversy always makes for a more interesting, and more informative, session.

Growing interest in TDD versions of LTE

A few weeks ago I put the provocative title "LTE and spectrum stupidity" on a blog post, prompting a flurry of objections in the comments.  (Thanks guys. I enjoy controversy).  My complaint was that, despite defining both FDD and TDD versions of LTE, the industry had only implemented the FDD version even though TDD can better match available capacity to data traffic demands that can vary widely between 10 ms LTE frames.

It's obvious from the comments that I didn't win over all objectors and I haven't had time to write an article on short term (10 ms) Internet traffic variations, but I did notice this article at dailywireless.org and, in particular, this article at Light Reading which lists a large group of companies, both operators and equipment providers, who have supported a request by Clearwire that the 3GPP specify how TD-LTE should work across the 2.6 GHz spectrum.  In addition to Clearwire, 

... there was broad industry support for the proposal from other companies, including: Sprint Nextel Corp. (NYSE: S), NII Holdings Inc. (Nasdaq: NIHD), China Mobile Communications Corp. , UK Broadband Ltd. , Motorola Inc. (NYSE: MOT), Huawei Technologies Co. Ltd. , TD Tech Ltd. , WiChorus Inc. , ZTE Corp. (Shenzhen: 000063; Hong Kong: 0763), Chinese Academy of Telecommunications Technology , Nokia Siemens Networks , Cisco Systems Inc. (Nasdaq: CSCO), Sequans Communications , Alcatel-Lucent (NYSE: ALU), Alcatel Shanghai Bell Co. Ltd., and Rohde & Schwarz GmbH & Co. KG.

Part of the 2.6GHz band is already specified for TDD, namely the 2570MHz-to-2620MHz band. The new work at the 3GPP will ensure that all of Clearwire's spectrum will have a definition for TD-LTE operation and will get the TD-LTE specs in line with Federal Communications Commission (FCC) requirements as well.

It will take a few years to play out, but we should get real data on real traffic using TD-LTE within five years.  Before then, I'll try and find time to gather the data and write the definitive article on millisecond-to-millisecond Internet traffic variation (but not in the next few weeks - sorry).

As the dailywireless.org article says, "In a data-driven world, symmetrical pipes can be a waste of space." And even when long term data averages are symmetric, short term variations in IP traffic are extreme (see the 3rd diagram here: http://su.pr/1Vmge3 ).

Community fiber ripoff - Exclusive marketing & bulk service agreements

Four years ago I read an article in Broadband Properties magazine with frightening implications.  This caused me to write a blog post, Community Fiber - Read the fine Print. Well, through to the wonder of Google, I've just been connected to a group of people who were caught in exactly the kind of trap I was worried about.

It works as follows:  The developer of a planned community or condominium enters into an exclusive marketing agreement or a bulk services agreement for communications services (Internet, TV, phone, etc.) for the community.  The contract is placed with a firm run by the developer's brother-in-law, or otherwise set up to benefit the developer.  The initial rates look good and prospects are sold on the idea that their new residence comes with state of the art communications services.

Later, after the developer has sold most units and the community is up and running, it turns out the communications contract is not such a great deal, but it's a solid contract and it runs for 25 years!

That's right, 25 years.

As chronicled by BanBulkBilling.com, this is now a problem for at least many dozen communities developed by many different developers.  Of course there are consumer protection laws in most states, but that's a long hard slog - good for the attorney's but not so great for home owners associations.

The Wi-Fi solution for mobile data offload

A recent Google alert pointed me to this article which I wrote in December 2009 for my monthly column in Internet Telephony magazine.  It was published in the February issue and has only now become available on line. None-the-less, it's points are timely. It opens with:

LTE versus WiMAX is a standard topic in the press and at conferences, as if something disruptive was happening or might happen. Wrong!  WiMAX and LTE are technical variations on the same business model providing similar services. If we’re looking for disruption, we need to catch up on what’s happening with Wi-Fi.

and concludes:

Expect to see both fixed and mobile carriers including free Wi-Fi access in their subscription bundles as Wi-Fi trumps femtocells. Conventional operators are not going away but, over the next decade, it’s Wi-Fi that will shake up business models and drive disruption.

Check it out.

LTE and spectrum stupidity

Mobile operators are counting on Long Term Evolution (LTE) technology to handle surging demand for mobile data access.  But LTE developers made some poor choices, cutting spectral efficiency and thus driving up operator costs.

LTE was envisioned as an all IP system, but the RF allocations follow the voice-centric approach of earlier generations.  While LTE standards allow for either Frequency Division Duplexing (FDD) or Time Division Duplexing (TDD), all initial LTE equipment uses FDD.  FDD requires two separate blocks of spectrum - one for each direction.  FDD makes perfect sense for bi-directional voice traffic. It makes no sense for data. With the exception of peer-to-peer file sharing (which most mobile operators block), data traffic is very asymmetric. Sending data via FDD means one block of spectrum is fully utilized and the other, equal sized block, is dramatically under utilized.  Result: the operator pays for almost twice the spectrum they actually use.

Verizon is deploying LTE in the 700 MHz C block which means they are using 746 MHz to 756 MHz (a 10 MHz channel) for their downlink (to the mobile device) and wasting most of 777 MHz to 787 MHz (another 10 MHz channel) for the uplink. If Verizon could deploy TDD (as used by WiMAX and as defined for LTE but not implemented), they could fully utilize both 10 MHz blocks for data tranfers, almost doubling their data capacity.

I don't know the actual capacity Verizon will realize on average with their first generation LTE infrastructure. But suppose Peter Rysavy is correct (as implied by Gigaom) that Verizon will initially average 15 Mbps per 10 MHz channel. That's 15/15 Mbps, symmetric, even though average traffic is likely to be 15/2 Mbps. No single user is likely to see 15 Mbps; rather that 15 Mbps is shared among all users in that sector. With TDD (the default for WiMAX and an unimplemented option for LTE), the Verizon spectrum could support two channels of perhaps 13/2 Mbps each in that same sector. Again, no single user will see 13 Mbps, but all the users in the cell will be sharing 30 Mbps of capacity that can be dynamically divided between up and down - mostly like averaging 26/4 Mbps but able to allocate 15/15 or 28/2 as the traffic mix changes.

It's ironic the LTE implementors got this wrong when you consider their decision to use only IP in the rest of the LTE design, thereby dropping support for traditional voice or SMS services.  That's right, initial LTE deployments won't support voice telephony or SMS messages, only data services, and yet LTE spectrum assignments were made as if voice comes first.

That's ironic.

Broadband Wireless — the future is 5 GHz

and eventually, spectrum from 4 GHz to 10 GHz.  

Unfortunately, the mobile industry doesn't understand the implications of MIMO or beamforming, so they are asking for the wrong thing.  They are asking for more spectrum near their current bands (below 2.1 GHz) or lower, e.g. in former TV bands below 700 MHz.  This is all wrong.

Until recently you could say: lower frequencies "work better" meaning they go farther.  But this was a technology limitation, not something in the physics. From a physical point of view, 5 GHz photons pass through the atmosphere just as well as 700 MHz photons. Today, MIMO and beamforming are correcting some of these historic technology limitations and changing everything.

• MIMO not only delivers significant performance improvement, it makes 5 GHz spectrum as useful as lower frequency spectrum for all sorts of applications.
• Beamforming delivers the next performance increment (20x and up), and it's actually easier to implement at higher frequencies.

Addressing mobile data demand

There is no doubt data consumption is growing more rapidly than mobile operators' data capacity (see slides 10 and 11 of my presentation at 4GWE). Unfortunately, US cellular operators are pressing for more low frequency radio spectrum and they are being heard in Washington. In his keynote at this year's Consumer Electronics Show, FCC Chairman Julius Genachowski said an impending shortage of wireless spectrum in the U.S. will dampen future economic growth unless action is taken to fix the problem. 

However, the short term issue is investment dollars not spectrum and the long term solution is new technology at much higher frequencies (above 4 GHz) rather than more spectrum for existing technology near existing bands.

Why 5 GHz spectrum is more useful than TV frequencies

Everyone know TV signals go long distances but remember than TV broadcasters use from 100 Kilowatts to 5 Megawatts ERP.  That's a million to a hundred million times more signal than your mobile handset puts out.  No wonder it covers a lot of distance.

There's also an equation most wireless engineers use in one form or another to calculate free space path loss which says higher frequencies have more loss. But this equation encapsulates two factors:  the true path loss and the size of the antenna.  It assumes a 1/2 wavelength antenna. Higher frequencies have shorter wavelengths, so it assumes the antenna gets smaller as the frequency goes up. Smaller antenna, less signal. With comparable antenna apertures, path loss in the atmosphere is the same from below 50 MHz to nearly 10 GHz. Thus in open air, 5 GHz photons go just as far as 500 MHz (US channel 19) photons or indeed photons for Channel 2 or Channel 50.

Multi-path and MIMO

The reason people have had trouble with higher frequencies for the past 100+ years is "multi-path" propagation.  As a signal radiates from a source, some of it goes directly to the receiving antenna but some of it goes in other directions where it may be reflected or refracted by objects it encounters. When reflected signals also reach the receiving antenna, they are slightly delayed because they traveled a slightly longer distance.  In the days of over-the-air analog TV, we saw these delayed signals as "ghosts" or shadows around images on our TV screens.  For digital data transmission, multi-path contributes to the "noise" in the signal-to-noise ratio. The historic problem with higher frequencies is their shorter wavelengths made it easier for them to be reflected and refracted by man-made objects like buildings, window frames and even closely spaced double pane glass surfaces.

 

But with MIMO, all this changes. With MIMO's multiple antennas and multiple radio front ends, it's possible to separately decode and make use of the multi-path signals.  Now "multi-path" is not only removed as a source of "noise," it adds additional signal and helps carry more data.

Beamforming

A beamformer uses signal processing to control the phase and relative amplitude of the signal at each of a group of independent antenna elements.  Radiation from multiple antenna elements causes a pattern of constructive and destructive interference in the resulting wavefront.  This can produce a tight beam just like one from a highly directional antenna.  But with beamforming, the antenna beam can be steered in software on a packet-by-packet basis.

With eight antenna elements spaced 1/2 wavelength apart (total 3.5 wavelengths), you can create a beam like this:

 

Highly directional beams significantly extend the usable range of a wireless system. And since the beam is computed with digital signal processing software, it can be steered to different directions in microseconds. What's more, the benefits of beamforming apply both while transmitting and while receiving. Either way, the beamformer accentuates the signal in the desired direction while surpressing signals to/from other directions.

But what about wavelength?  To obtain this narrow beam, the outer antenna elements are 3.5 wavelengths apart.  At 5.8 GHz, that's less than 7.5" so the whole antenna array easily fits in a ceiling mounted access point just 8" x 3" by 2".  At 700 MHz, that degree of beamforming still requires 3.5 wavelengths, but wavelengths are longer so now we need 5 feet of separation — something that may fit on a cell tower, but is difficult for a microcell and impossible for a femtocell.  For beamforming, higher frequencies are an advantage.

More spectrum at higher frequencies

Finally, there's a lot more spectrum potentially available at higher frequencies and, today, in the 5GHz band, there is over 555 MHz of license-exempt spectrum already available for applications like Wi-Fi. That's more spectrum than Verizon Wireless, AT&T Wireless, Sprint PCS and T-Mobile USA have, combined!

Wi-Fi blazes the trail

The Wi-Fi market place is vastly more diverse than the 3G/4G mobile operator market place, which means many new technologies show up in Wi-Fi years before they are deployed in mobile networks.  That is certainly true of so-called 4G modulation (OFDM) which was deployed for Wi-Fi with 802.11a (1999) and 802.11g (2003), years ahead of WiMAX (2005) or LTE (2010).

The 802.11n specification includes MIMO and optional beamforming and silicon technology is appearing to support 4x4 MIMO with beamforming.  MIMO products (2x2) have been shipping since 2007 and 4x4 MIMO in consumer products expected in the next six months. Meanwhile, many players are scrambling to deliver 11n options, including beamforming. Early systems are already deployed.

Mobile operators, please pay attention

The technology benefits of MIMO and beamforming apply to the mobile phone industry, it will just take a few years for the deliberate pace of the industry to catch up.  Meanwhile, mobile phone operators should be tracking real world measurements of Wi-Fi performance to understand what spectrum they will really need five to eight years hence.

Network Neutrality, Common Carriage and Layers - an international perspective

I'm attending an FCC workshop on 'The Open Internet" which means Network Neutrality. I am very much in favor of open access to the Internet. However, by settling for network neutrality rules, we won't solve the real problem, which is a lack of competing ISPs.  If we don't address this, we'll insure the US falls further and further behind other countries. Consider:

 

In the US, we claim to be supporting facilities-based competition. This means the rights-of-way are open to multiple competitors and you expect to see competing networks built from the ground up.  I call this open access at layer zero.  Unfortunately, this approach hasn't worked in the US. Most locations have only one or two access networks (the phone company-cable company duopoly).  

The reason is fairly easy to understand:

 

Building a new set of facilities represented a large fixed cost.  This investment must be paid back and typically only larger players have enough revenue to be able to make the interest payments. If you only capture 10% market share, you can't payback the investment.  As a result, facilities-based competition usually results in 2-3 competitors, at most.

Note that in Korea, facilities-based competition is part of a fairly complex history of government and private network evolution.  Hong Kong also has facilities-based competition, most notably by Hong Kong Broadband Networks Ltd.  But that's one fiber over-builder in a market that has open access to copper (i.e. layer 1 access) and many, many DSL providers. Facilities-based competition is a great idea, but by itself it doesn't work.

The ideal solution, at least for metropolitan population densities should be clear from this diagram:

The horizontal axis shows years of useful life. Notice that dark fiber is expensive to install but has a useful life of decades. But the equipment you use to light up the fiber will be functionally obsolete in 30 months just due to Moore's Law.

So what happened in Stockholm?  In 1994, Stockholm chartered a company, Stokab AB, to lay dark fiber and offer it to all comers.  Throughout the 90s, Stokab only offered dark fiber.  The result is many, many ISPs and corporate customers who lease fiber for diverse purposes.  A bigger result, is one of the lowest costs for some of the best Internet access of any city in the world.

What the US really needs

We need open access to bare copper and dark fiber from a utility that is regulated and is structurally separate from companies who provide telephony, TV or ISP services.

Rather than talk about network neutrality, we should be talking about structural separation, or at a minimum, functional separation with common carriage requirements. That would be a big change and is probably politically impossible, but that's the only way to get ISP competition and ISP competition is the only approach that will get us world class broadband.

FCC workshop on Open Internet at MIT this Wed January 13th

I haven't seen much publicity for this, but an FCC staff "workshop" on "Innovation, Investment and the Open Internet" will be held in the Bartos Theater at MIT"s Media Labs on Wedensday January 13, 2010.  "Open Internet" is usually political speak for Network Neutrality, although presumably some of speakers will differentiate between the advantages of open access versus the specifics of network neutrality, common carriage, structural separation and other approaches to achieving open access.

Details were provided in a press release which was picked up by a few people, but not many that I follow so, if you are in the Boston area, here's the scoop:

JANUARY 13 WORKSHOP ON INNOVATION, INVESTMENT, AND THE OPEN INTERNET

The Federal Communications Commission will hold a staff workshop on innovation and investment as part of the Commission’s Open Internet proceeding. This workshop will examine how the Internet’s openness affects the ability of network operators, Internet content and application providers, and other Internet technology developers to innovate and to drive investment, job creation, and economic growth throughout the Internet ecosystem. These issues will be explored from the diverse perspectives of innovators and entrepreneurs, investors, network operators and equipment vendors, and experts in Internet innovation and investment. 

WHAT: Innovation, Investment, and the Open Internet
WHEN: Wednesday, January 13, 4:30 p.m.
WHERE: MIT Media Lab, Bartos Theater Cambridge, Massachusetts
ONLINE: www.openinternet.gov/workshops/ 

PANELISTS:
Ajay Agarwal, Bain Capital Ventures
Tim Berners-Lee, World Wide Web Consortium (W3C)
David Clark, MIT Computer Science & Artificial Intelligence Laboratory
Jeffrey Glueck, Skyfire
Shane Greenstein, Kellogg School of Management, Northwestern University
Nabeel Hyatt, Conduit Labs
Susie Kim Riley, Camiant, Inc.
Paul Sagan, Akamai
Lynn St. Amour, Internet Society
Amy Tykeson, BendBroadband
Barbara van Schewick, Stanford Law School
Marcus Weldon, Alcatel-Lucent
Christopher S. Yoo, University of Pennsylvania Law School 
Sharon Gillett, Wireline Competition Bureau (Moderator)
Paul de Sa, Office of Strategic Planning and Policy Analysis (Moderator) 

The workshop will be open to the public; however, admittance will be limited to the seating available. Audio/video coverage of the workshop will be streamed live with open captioning over the Web at www.openinternet.gov/workshops/.

Broadband capacity - sizing buffers to handle traffic bursts

This is part 4 in a sequence of posts examining how broadband services actually work.  Part 1 looked at ISP concentration ratios.  Part 2 examined the effect of  averaging traffic from many subscribers.  Part 3 considered the impact of congestion on user experience. Here we'll take a finer grained look at traffic in the access network.

How do we achieve "zero" packet loss? Most websites are "hosted" and connect to the backbone by Fast Ethernet or Gigabit Ethernet links. Individual websites may be mismanaged, but there is no excuse for packet loss here.

Internet backbone links and backbone-to-backbone hand-offs are extremely high capacity multiplexing the traffic of thousands to millions of users.  Traffic statistics are predictable and operators compete on the basis of service levels, so packet loss is typically zero.

If there are performance problems, they're usually in the access network. Of course, one bottleneck is the service I've signed up for, for example a DSL link at 6 Mbps /1 Mbps. By definition my traffic is shaped to the service level I've signed up for.  This shaping typically happens at the first aggregation point (CMTS, DSLAM or WISP P2MP radio) or, on an IP or MAC address basis, at a bandwidth management device further upstream.  If they arise, problems are in the aggregation points and/or links between me and the backbone.

Since packet loss significantly degrades user experience, we'd like to understand how much over-provisioning a broadband service provider needs in order to absolutely minimize packet loss in the access network.  To understand this we need to look at smaller levels of aggregation (dozens and hundreds of users) and shorter intervals (sub-second rather than 5 minute averages, as few routers buffer more than a few tens or hundreds of milliseconds of traffic).

But first lets look at some five minute averages from a WISP with ~300 customers, each with either 1 Mbps or 3 Mbps service.

If one took these graphs at face value, it seems 20 Mbps of backhaul would be more than adequate, as the traffic peaks at 17 Mbps.  But 5 minute averages don't speak to what's really happening.

I don't have sub-second averages for the specific WISP above, but Fergal Toomey, Chief Scientist at Corvil Ltd., has some very detailed measurements on a similar Internet access network providing services to users at 512 Kbps, 1 Mbps and 2 Mbps. These are available in a whitepaper you can request from Corvil's website. Fergal's measurements were made at the Internet POP looking at traffic headed towards individual users.  Also, his measurements were made in 2004 when most hosted servers had 100 Mbps connections to the backbone and the majority of computers had TCP stacks configured for a maximum TCP receive window of 32KB.  In this graph, green shows 5 minutes averages, while light blue shows 500 ms averages. Dark blue shows 5 ms averages and they are striking!

What's Happening?

Web browsers typically use persistent TCP connections to minimize connection overhead and to avoid repeating TCP's slow start.  As a result, servers will frequently respond to a new request with a full window's worth of data.  In 2004 that was 32 KBs of data or ~23 packets of up to 1500 bytes each. Today it could be much more. Absorbing one such burst should be no problem, but to what extent are such bursts coincident?

Beyond web browsing, detailed traffic analysis uncovers a variety of odd behavior and some less savory behavior. For example, one burst was a blast of many 41-byte packets sent by a web application that apparently does a series of 1-byte writes to a TCP socket with the NO_DELAY flag set.  Several thousand packets arrive in quick succession causing packet loss.  Worse yet, follow-on losses occur as retransmissions cause a decaying series of echo bursts.

On a more malicious note, the data includes bursts caused by port scans.  These consist of many, many minimum size probe packets that attempt to scan all of a subscriber's ports in one enormous burst.  Luckily these lost packets are not retransmitted.

Beyond these, Fergal Toomey's analysis uncovered a variety of application anomalies that I found unexpected, but which crop up in the real world.

What headroom is really required?

Clearly we could just increase the size of the buffers in the router and smooth out any of these bursts, but that would also increase the delay while the burst lasts and thus introduce significant jitter.  A better question is how much headroom is required to guarantee zero packet loss and no more than 20 ms of incremental delay?

Conveniently, Fergal Toomey has run that calculation on the data set above and the result looks like this:

In this graph, the average traffic (the "mean" shown in light green) peaks just shy of 5 Mbps.  To guarantee zero packet loss and no more than 20 ms incremental delay 100 % of the time requires 19 Mbps of capacity or nearly 4x.  However, to guarantee zero packet loss and no more than 20 ms of delay 99% of the time, requires only 8 Mbps, i.e. a little less than 2x.

Interestingly, while ISP practices tend to be trade secrets, the rule of thumb that 2x headroom provides good service crops up repeatedly.

Conclusion

As a practical matter, real Internet traffic is fairly complex and includes a wide variety of anomalous behaviors. Rules of thumb based on averaging the traffic of a few hundred or even a few thousand subscribers are likely to be very expensive (in terms of excess capacity).  A better approach is to continuously monitor packet loss and delay variation (jitter), adding capacity as needed to keep those measures below desired levels.  

Zero packet loss and less than 20 ms incremental jitter, more than 99 percent of the time sounds plausible objective to me.  

Now if only my broadband service provider was willing to share any information about their service level objectives or performance.  Ooops, I must be dreaming.

Broadband capacity - the impact of congestion

This is part 3 in a sequence of posts examining how broadband services actually work.  Part 1 looked at ISP concentration ratios and part 2 examined the impact of averaging many subscribers. Future posts will consider how to size backhaul links and how to configure buffers.

Two major reasons people purchase broadband (or sign up for more capacity) are to improve interactivity and to get access to new applications, for example streaming movies.  New apps is clear, but it's worth looking congestion and how congestion affects interactivity.

Congestion has two impacts. First you get increased delays, as packets are buffered in queues waiting for free capacity on the next link. Usually this adds tens or hundreds of milliseconds to round trip times. Then, if the average traffic exceeds link capacity, you get packet loss. If you do have congestion, it's these packet losses that cause TCP traffic flows to back off, thus avoiding congestion collapse (and even less throughput).

Here is a graph of packet loss on a service that most of us would consider unacceptable.  These measurements were taken by Tom Dunigan on an early cable modem service (February 2001) that was substantially upgraded about a year later.

 

The next graph (also from Tom Donigan) shows how this congestion impacts interactivity.  The graph shows the echo delay for typing into a remote program using telnet in character-at-a-time mode.  One test (red line) was run in the early morning and one test (green line) was run during the evening when packet loss was peaking.  Each test measured the echo response time for 100 successive individual characters. Notice the typical echo delay goes up from ~130 ms in the morning to roughly 200-300 ms in the evening, but with occasional delays of more than one second.

Relatively few programs operate character-by-character, so the extra 100 ms or so might not matter.  On the other hand, a once second delay is noticeable!

A more common activity is viewing content on websites.  Here, the dominant time in any interaction is the time spent waiting for web content to download to your browser.  Since web content is downloaded using TCP (and more generally, TCP is the dominant protocol in use today), it's worth looking at the impact of packet loss on TCP throughput.  

The TCP protocol includes a congestion avoidance algorithm which is triggered by packet loss. When a packet is lost, the TCP sender slows down. As a result, the data rate for a single TCP flow looks like this (thanks to Guido Appenzeller):

Of course this assumes the TCP flow lasts long enough to saturate the bottleneck link and that this is the only flow in the network!  

What happens in real networks is complex, but RFC 3155 has an approximate formula for how a single flow is affected by packet loss in real network.  Even better, Bill Gibson at Niwot Networks has produced a nice graphic based on that formula for his paper on TCP limitations on file transfer performance.  It looks like this:

 

This is dramatic!  The different lines reflect different end-to-end round trip times (RTT) - times that will vary depending on the site you are connecting with.  Also, they represent the maximum throughput you could achieve with long duration TCP flows and no other bottlenecks.  What's notable is the logarithmic throughput scale on the left and the fact that, at 1% packet loss (0.01 on the bottom scale), potential throughput drops by a factor of 100!

Again, there are many caveats.  A real network has a mix of short- and long-lived flows. The local operating system may not be optimized to take full advantage of broadband speeds (although MS Windows actually got better at this with Vista). None-the-less, even 1% or 2% packet loss is correlated with poor user experience.

In short, to actually obtain the instantaneous throughput you thought you were purchasing, you don't want packet loss in upstream portions of the network.

I'll discuss what this actually implies for routers and backhaul links in a subsequent post.

Broadband capacity - the impact of averaging many subscribers

The fundamental idea of packet networks like the Internet is to transport variable rate data flows over a shared infrastructure.  Individual end points may generate little or no traffic for minutes or hours on end and then suddenly want to send or receive a large data set as quickly as possible.  Packet networks efficiently share common transport facilities among multiple users but how does this sharing actually work?  Yesterday I posted some actual, relatively current data from three small ISPs.  Today, let me show you how averaging traffic from many individual end points produces a completely stable core network flow.

Each of the graphs below shows successive five minutes measurements during a 24 hour period, but the first graph reflects a few hundred subscribers while the final graph represents the average of many, many millions of subscribers.

In the first graph, the blue line reflects outbound traffic from a few hundred subscribers.  While the average is just over 1 Mbps, the peak (at ~1615 hrs) is 6.9 Mbps for a peak-to-average of nearly 7-to-1.  The green bars represent inbound traffic with an average of 5.6 Mbps and a peak of nearly 16.8 Mbps for a peak-to-average of 3-to-1.  Some of this is time-of-day dependency but a lot of it, particularly the 6.91 Mbps spike at 1615 hrs, is the result of averaging only a few hundred subscribers.

 

Here's a traffic measurement at AMS-IX, the largest Internet Exchange in the world.  With an average of nearly 1 Gbps, the peak-to-average is a modest 1.43-to-1 and it's almost entirely due to time of day.  The largest change from one 5 minute measurement to the next appears to be about 140 Mbps so the short term peak-to-average is perhaps 1.1-to-1.

 

Finally, here a measurement of all the traffic flowing through AMS-IX – over 822 Gbps peak.  Here the time-of-day variation remains but there are virtually no statistical fluctuations.  The short term peak-to-average is almost 1-to-1.

 

To put this in economic terms, for an Internet backbone link that runs at many Gbps your daily traffic profile is completely stable and you can guarantee zero packet loss merely by providing 10%-20% extra capacity above your daily peak.  

But if you are running a small ISP, both the capacity you need per subscriber and the extra "headroom" for unanticipated peaks must be substantially larger.  To get a handle on what is required we also need to look at shorter intervals (shorter than five minutes).  But more on that in a subsequent post.

ISP Contention Ratios and average traffic per user – some data points

Recently there’s been a flurry of discussions about bandwidth shortages, not to mention bandwidth hogs, bandwidth caps, network neutrality and what constitutes fair practices by ISPs. But there is very little public data and no transparency! 

So let me offer a (very) few actual data points, one from a few years ago and three fairly current. These come from friends who are operators and from emails shared on trade association mail lists. Since some of it was not intended for publication, I have withheld specific names. 

Notice contention ratios vary widely and yet each of these operators has enough extra capacity that they claim to avoid link saturation and its impacts, like packet loss and excessive ping times. It’s true these are all small operators (280-1200 customers) so this data lacks the statistical smoothness one would expect when aggregating traffic from thousands or millions of subscribers. 

I also have some data on traffic averages by time and by number of subscribers that I’ll share in a subsequent blog post. 

Mar 2006 – Rural ISP (Dialup, DSL & Wireless)
130 Motorola Canopy subscribers (with 512 Kbps service)
150 DSL subscribers (110 with 512 Kbps service; 40 at 1 Mbps)
100 dialup lines serving 800 customers (average 40 Kbps)
Daily traffic was saturating 4.5 Mbps (3 T1s) for an hour per day but not with 6 Mbps.
Contention ratio: 164Mbps / 6 Mbps -> 27:1
Traffic per BB user: 6 Mbps / 280 subscribers -> 21 Kbps 

Sep 2009 – Rural ISP (DSL & Wireless)
1200 customers, DSL subs at 1.5, 3 or 6 Mbps; wireless subs at 384 Kbps, 768 Kbps, 1 Mbps or 3 Mbps. I don’t have the mix, so I'm guessing the contention ratio.
Daily traffic peaks at around 35-40 Mbps; up to 150 Mbps Internet transit available.
Contention ratio: 2400 Mbps(?) / 150 Mbps -> 16:1
Traffic per BB user: 40 Mbps / 1200 subscribers -> 33 Kbps 

Sep 2009 – Community-run WISP
340 customers, mostly on 3-4 Mbps service
Daily traffic peaks around 17 Mbps; 45 Mbps of Internet transit available.
Contention ratio: 1190 Mbps / 45 Mbps -> 26:1
Traffic per BB user: 17 Mbps / 340 members -> 50 Kbps 

Sep 2009 – Rural ISP
370 customers: 300 Customers on wireless (1-3 Mbps) and 70 customers with FTTH (10/100 & up, but most have 10 Mbps peak). Monthly fees based on a 10 or 20 GB cap with extra charges for heavier use.
Daily traffic peaks at 13-14 Mbps; 20 Mbps of Internet transit available.
Contention ratio: 1200 Mbps (?) / 20 Mbps -> 60:1
Traffic per BB user: 14 Mbps / 370 subscribers -> 37 Kbps