Communications

Mobile subscriber growth forecasts are always wrong

I've been following forecasts of mobile phone adoption since the mid-1990s and there is one thing that's completely consistent – future subscriber growth is always underestimated.  This is true across every analyst, every firm and every agency I've encountered. 

Usually the underestimation takes 2-3 years to become obvious, but here's a graph from Wireless Intelligence (a joint venture of Ovum and the GSMA) which appears in a report Deloitte did for the GSMA entitled SIM Activation Tax and Mobile Telecommunications in Pakistan, which I got from Babar Bhatti (Thank you Babar!).

When I saw this I was immediately suspicious.  Why does the green dotted line slack off.  If anything the rate of mobile adoption in Pakistan has been accelerating.  So I took a quick check of the "Telecom Indicators" on the Pakistan Telecommunication Authority website.  The figures from Wireless Intelligence and from the PTA agree up through June 2006, but at that point Wireless Intelligence begins under estimating.

The Wireless Intelligence estimate for 2Q07 appears to be 32 or 33.  The actual figures for 2Q07 were 39.94.  The Wireless Intelligence estimate for 4Q07 is 37 or 38, but the actual figures for August 31, 2007 were already 42.65.

Why the consistent under estimates?

I don't have a answer but I have an idea, based on 4 or 5 discussions spread over the past decade.  Most people (including noted industry analysts) haven't really grokked Moore's law outside of PC speeds.  Or more generally, they don't get the power of new ideas and new methods to improve price/performance across the board.

In the specific case of Pakistan, I've heard at least two people (not from Wireless Intelligence) say that mobile phone adoption will slow down as, even with today's low rates, the poorest people will never be able to afford mobile phones.  What this misses is, every year the low end mobile phone costs less, and every year the cost per subscriber for new mobile infrastructure goes down.  Pakistan is a highly competitive mobile market.  When costs go down, subscribers see the benefit fairly rapidly.

Moore's law is rather specific.  The economist Paul Romer describes the more general concept:

Every generation has perceived the limits to growth that finite resources and undesirable side effects would pose if no new recipes or ideas were discovered. And every generation has underestimated the potential for finding new recipes and ideas. We consistently fail to grasp how many ideas remain to be discovered. Possibilities do not add up. They multiply.

Globally, it took 20 years to sign up the first billion mobile accounts, less than four years for the second billion and only two years for the third billion.  Yes, these are SIM cards not necessarily unique people but, on the other side of the coin, the world's poorest start by sharing a mobile phone.  That gets them on the bandwagon, helps them economically while improving their standard of living.  So there is another side to the coin as well, mobile phone driven economic growth also helps increase the number of people who can afford the ever less expensive mobile phones.

More on fat tails in stock prices

While it's a bit remote from communications, I've touched on mathematical models for financial markets several times (here, here & here), primarily as an outgrowth of my interest in long tails.  So I was very happy when Paul Kedrosky pointed to this an excellent survey by Dan diBartolomeo of Northfield Information Services.  The last eight slides are full of references – more than I'm likely to track down in a year...

There's a PDF version here.

New Zealand Joins UK in Operational Separation

New Zealand has ordered the country's dominant phone company Telecom, to split into three operating units.  Like British Telecom in the UK, this is functional or operational separation, not structural separation.

Structural separation is what happened to the Bell System in 1983-84.  It resulted in separate companies who went their own ways.  Note that they regulated monopolies never did compete and eventually remerged, but the less regulated entity (AT&T) did pursue a truly separate path for at least two decades and we did get real competition in long distance services as a result.

When BT (voluntarily) split their operations, there was quite a bit of discussion which confused structural and functional separation even among people who know better, or should know better.  Unlike the BT split, New Zealand's Telecom has fought the split, so it will be doubly interesting to see if it actually works out.

In both of the current cases, separation leaves the access group responsible for the physical ducts, the raw fiber or copper, and the electronics necessary to make it work.  I worry about the latter.  There doesn't appear to be an option for anyone else to get access to the ducts to put in their own fiber, for example a co-op or condominium or a municipality.  Instead, the separation relies on the now-separated access business as the sole entity to upgrade the access network.

Perhaps I've misunderstood, but I wonder what the 10-15 year impact of this approach will be – better and better DSL when we really wanted fiber?

Time will tell.

More articles on the productivity benefits of telecom investment

Over the past two years, I written on the Large Productivity Benefit of Telecom Investment, the Strong positive impact of telecom on economic growth in developing countries and Large Productivity Benefit of Telecom Investment in Emerging Markets.  Those articles and posts were the result of my finding papers by economists who'd done the hard work of finding data, building models and trying to understand what's happening.  Today, I came by another set of interesting papers, thanks to a pointer from Babar Bhatti who blogs at State of Telecom Industry in Pakistan.

The pointer came in a paper by Deloitte done for, and available from, the GSM Association here.  What's interesting is this on page 25-26:

Whilst not considered in this report, a further positive impact of the mobile industry is the additional value add created by the mobile communications industry across the economy. This is, where measured, captured by multiplier that captures expenditure in subsequent rounds. The following figure shows the values of multipliers that have been calculated in other studies.

Not all are easily available on the web, but here are the links for the first four studies:

http://www.o2.com/media_files/news_100504.pdf
http://www.dirsi.net/english/files/Ovum%20Bangladesh%20Main%20report1f.pdf
http://www.afom.fr/v4/TEMPLATES/homepage.php
http://www.ofcom.org.uk/research/radiocomms/reports/economic_spectrum_use/

The Spread of FTTH; Parallels with Electrification

Fiberevolution has a cute video from EPB, formerly the Electric Power Board, of Chattanooga Tennessee, i.e. the publicly owned electric company.  Check out the first 45 seconds where they graphically make the case for municipal fiber.  Too bad they are just at the beginning of what promises to be a long legal battle.

As with many other such entities, EPB started in the 1930s when local residents felt they weren't getting electric service as rapidly as they wanted, and were prepared to commit public funds to benefit their local community – the same argument for local electric projects then and local fiber projects today.

A non-profit agency of the City of Chattanooga, EPB was established in 1935 for the sole purpose of providing electric power to the people of the greater Chattanooga area.

EPB got into the telecom business in 2000 and started building to business premises in 2002.  Now, they have a plan for FTTH, to be supported by triple play services.  It's been approved by their board and is due for a City Council vote today.

In a replay of electrification politics of the 1930s, there are strong protests and lawsuits from local and regional Cable TV interests.  There's already legislation in place to at least partially block such efforts.  As Stacey Briggs, executive director of the Tennessee Cable Telecommunications Association, puts it:

In 1999, my organization, the Tennessee Cable Telecommunications Association, worked with Tennessee’s municipal electric distributors, including EPB, to craft the laws pertaining to municipal entry into the cable and Internet business. The laws that resulted are intended to protect the public and to ensure that public assets are not misused.

I.e. the incumbents got state laws enacted that slow down local FTTH, at a minimum, by providing grounds for lawsuits.

Some time ago, Susan Crawford pointed to parallels with electrification in the US in the 1930s, which led me to read other bits of that history.  Much as I hate to admit it, the electrification experience only partly supports my position, i.e. that local municipalities should be given free reign to do what they like.  In fact, what worked for rural electrification in the US was not just permission, but active Federal support in the form of the Rural Electrification Administration (REA) which loaned money to local cooperatives, provided advice and gave them cover w.r.t. lawsuits by investor-owned-utilities.

Today, in the US, we have state and national laws and regulations that slow municipal and/or cooperative bodies wanting to provide services or even access the right of way.  If electrification is any indicator, we're in need of major policy changes at every level...

KDKA, not quite the first commercial radio station

Here's another piece of engineering history I stumbled on during my longer than normal return from Europe, i.e., Paris - Dublin - Boston.  A long time ago, I was general manager of the MIT college radio station (now WMBR, then WTBS but that's another story).  The history of broadcast radio as I understood it then (and up until last week) was that Westinghouse started broadcasting in Pittsburgh in 1920 under the call sign KDKA.

But a history article in the June issue of the Proceedings of the IEEE points out that Westinghouse only started after they noticed the success their employee, Frank Conrad, was having broadcasting music using recordings loaned to him by a local merchant on the condition that the store be mentioned during the broadcasts.

So now that I'm back online, I've looked at the history of KDKA and find, not only did Frank Conrad's private efforts predate KDKA, but a Canadian station (initially XWA, then CFCF and CIQC) actually started in 1919 and went to a regular schedule six months before KDKA.  There's a more complete history of Frank Conrad & KDKA on the KDKA site here.  The Canadians are quiet, as their successor station "940 Montreal" doesn't even list their call letters on their website, let alone their history.

In short, as with most human endeavors, there were multiple similar efforts happening, in parallel.

Full cycle – Fodder for horses; Fodder for tractors

Among my "toys" is a Ford 9N tractor, originally manufactured in March 1940 and still in service today.

So when I saw a review of Harold Brock's account of his years working for Henry Ford Sr., including his work on the Ford 9N tractor, I had to buy the book.

On an unrelated note, I've been discussing methanol and biodiesel with Wendell Bishop, one of the original co-founders of NMS, now involved in other forms of high tech, including methanol fuel cycles for transportation.  So this paragraph in Brock's book was interesting:

Mr. Ford said that to be successful and cost effective to farmers, the tractor should not cost more than the combined cost of a team of horses or mules, a set of harness, and the ten acres of land required to grow food for the animals.  If the project was successful, this would eliminate the toilsome caring for animals after a hard day's work.  Likewise, farmers could raise foodstuff from the ten acres of land rather than raising fodder for animals.  If we could meet this objective, Mr. Ford was convinced he finally could accomplish for farmers what he did with the Model T car in providing a real value.

I read this aloud to my wife and she replied, "And now we raising fodder for the tractors?"  She was referring to the biodiesel discussion, so I did some rough calculations...

While extremely advanced when it was introduced in 1939, the 9N tractor is only 22 hp and even under load consumes only 1-2 gallons per hour.  During harvest and other peak times, a tractor might be in use 12-15 hours per day, but in off seasons it might not be used at all some days.  Let's say six hours (perhaps 10 gallons) per day and 250 days per year, or roughly 2500 gallons per year.

Biofuel yields per acre are all over the place:  70 gallons per acre for soybeans, 127 g/a for rapeseed, 230 g/a for corn and 600 g/a reported for Brazil's sugarcane.  So depending upon your location and what you use for fodder for your tractor, it appears you need between 4 acres and 35 acres of land for "fodder" – 11 acres if you are using corn.

Of course it's still less work caring for a tractor than caring for a team of horses after a hard day's work...

Vladimir Kotelnikov – Father of Sampling Theory

Actually one of several "fathers."  Caution: the following is for the DSP engineers among you.  :-)

When I learned communications theory it was attributed to Claude Shannon (his 1948 paper) and sampling theory to Nyquist.  I do remember my father once telling me that E. T. Whittaker had published the relevant mathematics long before Shannon, but I never looked up the history.  And I never asked my father to elaborate as he was pushing his copy of Whittaker and Watson at the time, together with the idea I should be a mathematician or a physicist – not an engineer!

But on the plane back from Paris I read the June issue of Communications Engineer from The Institution of Engineering and Technology which included an interesting article on V. A. Kotelnikov by Professor Chris Bissell. 

... in the late 1940s <Shannon> wrote that the sampling theorem was “common knowledge in the communication art, but in spite of its evident importance it seems not to have appeared explicitly in the literature of communication theory.”  But Shannon was only partly correct. Ideas about sampling were indeed common knowledge in the late 1940s, and the theorem in various forms had appeared in the mathematical literature. But the theorem had also been published in the ‘literature of communication theory’ as early as 1933. Trouble was, it was published in the proceedings of a conference in Stalinist Russia – and in Russian.

The article goes on to give Kotelnikov credit for putting the problem of sampling a continuous, band-limited signal into an engineering context. Now that I'm back home and on-line again, I've been able to track down more of the story and access an English translation of Kotelnikov's seminal 1933 paper, On the capacity of the 'ether' and cables in electrical communication.  Not surprisingly, Wikipedia has an even more complete historical background on sampling theory in communications.

Suffice it to say, "Whittaker-Kotelnikov-Raabe-Shannon-Someya sampling theorem" would be a better name, as many people reached similar conclusions with varying degrees of parallelism.  Kotelnikov suffered from writing in Russian under Stalin and before the west started tracking Russian science.

Even with the English translation of Kotelnikov now available, Shannon's 1948 paper remains the best and most complete source for me.  I think it's only available in hardcopy forms, but other URLs of possible interest are here.

Performance per watt – Intel talks the talk

One thing that struck me at last week's AdvancedTCA Summit in Paris was Intel's Keate Despain's focus on performance per watt during his keynote address.  What a change from just a few years ago!  And it's not just Keate Despain, or the AdvancedTCA audience.  CPU performance per watt has finally become a significant competitive issue.  A quick Google search on "performance per watt" returns more than 2 million results including two paid ads, one by Intel and one by AMD.

Also at the Euro-ATCA Summit I had lunch with Stefan Ludwig who was promoting P.A. Semi's PWRficient processors, i.e. power optimized processors based on the PowerPC architecture.

I love it.  For years, I've pushed our engineers to focus on performance per watt and to this day I have a running argument with Texas Instruments over their C6x series DSPs (which are optimized for performance rather than performance per watt). 

This has nothing to do with being "green."   Every system and subsystem has to fit within some power budget.  At one time, a plug-in board for an IBM PC was risky if it dissipated more than 15 watts.  Until recently, carrier hotels (for example) had cooling for 50-150 watts per square foot.  3000 watts per rack was a big deal.  Try putting an IBM blade server in that!  You'd be hard pressed to support ten CPUs in an entire rack.  [Interesting discussion here (free but registration required).]

In my experience, if system architects focus on performance per watt, you end up with designs that are also competitive on density and cost, but it seldom works the other way around.

Benoît FELTEN – Mr. Fiberevolution

Well, I'm still without a working computer of my own, but I've borrowed an office at our Paris facility that has a PC with a French keyboard layout, so let's see if I can type anything...

It's been a fascinating week with a some interesting meetings and a few new learnings from the AdvancedTCA Summit Europe, but best for my personal interests was a meeting with Benoît Felten who's blog, Fiberevolution, I've followed avidly since I discovered it last spring.

I hadn't realized Benoît is a telecoms consultant whose work has ranged over a wide variety of telecom topics.  Indeed, it appears his avocation, i.e. broadband access, is one of the few telecom areas not part of his day job.  So we had wide ranging discussion across all of telecom – really fun!

I also got a much better sense of what going on with FTTH in France.  I'd read individual blog posts (e.g. this, this, this, this), but I didn't grasp the extent to which the French have enacted laws and regulations that ensure local authorities are allowed to build telecom networks.  Boy, would the folks in Lafayette Louisiana have loved that.  But of course I've written on this subject before.

The only curious thing was we chatted in a French bistro in the plaza by the Arch at La Defense, but the house red wine was from Chile – that's a first in my book.  :)

Laptop crash - expect email delays

Upon arriving in France on Monday my laptop crashed.  Luckily I already had copies of my presentations for the AdvancedTCA Summit Europe on a memory stick:  But, I've been disconnected from email and my blog (and the web) since then.  I expect to get an hour or so per day on other people's PCs, but if you are awaiting something from me, it could be mid next week before I'm home, have my problems sorted out and then catch up.

What a nightmare  :(

Architecturally induced denial of service susceptability

Fundamental design decisions have a big impact on how a network behaves under stress.  I just noticed two otherwise unrelated posts both touching on network design issues that result in (or protect against) network collapse under stress.

Yesterday, I wrote about August's Skype outage where the system took over 36 hours to recover from a collapse.  The issue was too many clients attempting to reconnect at once.  Instead of clients “backing off” when supernodes weren’t immediately available, they kept hammering away, trying to log in.

Last week I wrote about the ways the Internet and the Telecoms networks respond to congestion in the backbone.  In December 2006 an earthquake off the coast of Taiwan broke multiple fiber cables causing a congestion collapse on the Internet backbone in significant parts of Asia.  The issue was too many computers attempting to reestablish TCP sessions with the result there was no capacity left for any session to actually send data.

In both cases, there were externally induced problems but, rather than recovering, there was the equivalent of a denial of service attack, self inflicted!

Both of these failures have a fairly simple solution, at least architecturally.  Under conditions of severe overload, the system must be able to restrict new attempts (new TCP sessions, new Skype logins, etc.) to some small percentage of the available capacity.  This allows the rest of the capacity to serve the logins, sessions or calls that do get through, with the result that what capacity remains is put to good use.

As I commented last week, this is one place where the telecoms industry has the correct architecture.  When disaster strikes subjecting some part of the network to overload, it's easy to restrict new call attempts on trunks into the congested area, for example by call gapping. This limits the amount of new traffic to that which the network can handle.  Thus, if only 30% capacity is available, at least the network handles 30% of the calls, not 3% or zero.

Here's one place where network architects can learn from established telecoms practice.

Skype outage: post mortem evolves

Since my last summary, some new information has emerged.  Gerry Blackwell interviewed Skype's Director of Operations, Michael Jackson, got some comment from Martin Geddes, and wrote an interesting article.

I've also had an email exchange with Julian Cain, which I reprint below, and a brief exchange with Philippe Biondi of SecDev.org who referred my to an interesting blog post (in French) by his colleague Cédric Blancher at  EADS France, Innovation Works (Suresnes).

New information from Michael Jackson (in Gerry Blackwell's article) includes the idea of five supernodes per cell (of 300 users):

Each supernode handles about 300 nearby users. Skype configures five in each cell for redundancy. So with upwards of nine million users online, it takes something like 150,000 supernodes to make Skype work.

The triggering event is still attributed to massive computer reboots after Microsoft's Patch Tuesday.  Everything else in the article is consistent with the best earlier explanations, including Julian's which I summarized here, i.e. as Blackwell puts it:

... the real culprit, Skype now says—was a resource allocation algorithm in the client software that could not adapt to such a set of circumstances. Instead of clients “backing off” on their attempts to validate on the network when supernodes weren’t immediately available and waiting for the ship to right itself, they kept hammering away, trying to log in.

And the solution, having clients back off when supernodes aren't immediately responsive, is obvious.  What's left to understand?

1.  I still haven't seen a plausible explanation of why Microsoft's "Patch Tuesday" resulted in problems on Thursday morning, only a lot of questions.  If the problem was induced by massive reboots, why didn't it happen on Wednesday morning?

2.  I still haven't seen a reasonable discussion of scaling.  As I wrote back in August,

I wonder, apart from the login server cluster as a single point of failure, is there also a scaling issue?  FastTrack's breakthrough was the use of supernodes to make the system more scalable.  But was that just one layer of scalability?  If so, what happens when there are 300 million on-line users and one million supernodes?  Perhaps Julian (or another P2P expert) could comment...

Indeed I emailed Julian about scaling and also about Joost.  This was his reply.
On 09/05/2007 05:06 PM, Julian Cain wrote:

Brough,

Skype and Joost are utterly different however Skype is more like Fasttrack(Kazaa). Joosts' Network architecture is mainly "Centralized", they have their own server farm of Supernodes as well as Authentication and Jabber servers. The nature of Joost is less dependent on peer to peer routing as it's basis is tuned towards QoS. Joost peers route traffic and relay UDP based payload as media data streams as well as keep a small cache of what they have recently viewed, however currently every Joost peer is directly connected back to the Joost home servers unlike Skype and Kazaa where once authentication occurs it's "out of our hands".

I agree on the extent of Skype scalability being very limited because of the nature of the Supernodes. At any one time the Supernodes hold ~300-500 child nodes and maintain an "Overlay" network which consists of another several hundred Supernode to Supernode connections. Ie* The Supernode network is very dense in order to provide for best means routing of least cost, however the flaw in this architecture is where the "Overlay" network reaches a capacity and is unable to reliably route traffic. I do not currently have any statistics on how the "Overlay" layer is not scalable but as more Supernodes arise the management of the "Decentralized Data Store" becomes a very hard task as well as keeping this "Overlay" in one single "in sync" network. This was proven with the Skype outage as the network was "trying to heal" it had to start from many 10s of thousands of "Overlay" networks which very slowly were able to sync again as a "single" network however is still an issue today with presence.

For the current Skype "Overlay" network to scale indefinitely while maintaining  a "Single" network infrastructure it needs in place an organizational hierarchy of Supernodes and a level of Service for each of these Supernodes. *Ie. If Skype Supernodes worked in a way such as in Fasttrack then when the network reached 100 million users it would began to crawl. This is due to the dense nature of the upper "Overlay". I can only assume that Skype has thought of this and that when the Supernode ratio is beginning to "bottle neck" then there would be some ordered Hierarchy as to what role each Supernode was playing, otherwise the more Supernodes the more dense the "Overlay" the more the data is relayed back and forth before considering the "Supernode Overlay" into it's own Denial of Service attack.

I hope this helps to some degree, let me know if you have any other questions.

~Julian

So to the extent I have time to look into P2P technology further, I plan to explore what's been written about hierarchy in P2P networks.  Here are some references (which I've found but have not read as yet):

RFC 4981 on Survey of Research towards Robust Peer-to-Peer Networks: Search Methods

Hierarchical Peer-to-peer Systems by L. Garces-Erice, E.W. Biersack, P.A. Felber, K.W. Ross, and G. Urvoy-Keller.

An efficient peer-to-peer file sharing exploiting hierarchy and asymmetry, by G. Kwon and K. D. Ryu in the Proceedings of the 2003 Symposium on Applications and the Internet, 27-31 Jan. 2003 Page(s): 226 - 233.
< unfortunately only available on an IEEE pay-for site >

Paris and the Euro-AdvancedTCA Summit next week

I'll be in France next week.  On Tuesday and Wednesday I'll be at the AdvancedTCA Summit Europe 2007 at Disney's Newport Bay Club near Paris.  I have other meetings in and around Paris on Monday afternoon and Thursday, then I'm flying back to Boston on Friday morning.

On Tuesday afternoon I'm organizing, and speaking in, Tutorial T2B - AdvancedTCA/MicroTCA in Next-Generation IMS Networks, and on Wednesday morning I'm representing PICMG in Session 203: COTS Ecosystem (organized by Mountain View Alliance).

If you are attending the AdvancedTCA Summit, please say hello.  Or if you are otherwise in Paris next week and want to meet, please send me an email or text chat on Skype.

Congestion in the backbone: Telecom and Internet solutions

When a network is subject to a rapid increase in traffic perhaps combined with a rapid decrease in capacity (for example due to a fire or a natural disaster), there is a risk of congestion collapse.  In a congestion collapse, the remaining capacity is so overloaded with access attempts that virtually no traffic gets through.  In the case of telephony, everyone attempts to call their family and friends in a disaster area.  The long standing telephony approach is to restrict new call attempts upstream of the congested area, for example by call gapping.  This limits the amount of new traffic to that which the network can handle.  Thus, if only 30% capacity is available, at least the network handles 30% of the calls, not 3% or zero.

There are comparable issues for the Internet backbone, but they are not so completely solved.  As Wikipedia puts it:

Congestion collapse was identified as a possible problem as far back as 1984 (RFC 896). It was first observed on the early internet in October 1986, when the NSFnet phase-I backbone dropped three orders of magnitude from its capacity of 32 kbit/s to 40 bit/s, and continued until end nodes started implementing Van Jacobson's congestion control between 1987 and 1988.

TCP congestion control solved day-to-day congestion collapse, but it didn't deal with disasters such as the Taiwan earthquake of December 2006.

As a result of problems in the Internet backbone in Asia caused by that earthquake, there's new discussion of Internet congestion.  I stumbled on one interesting discussion on the NANOG (North American Network Operators Group) mail list.

In this thread, the NANOG group seems to be arriving at conclusions similar to those of the telecom industry, i.e. throttle new traffic attempts upstream of the congested area.  From Fred Baker:

So plan B would be to in some way rate limit the passage of TCP SYN/SYN-ACK and SCTP INIT in such a way that the hosed links remain fully utilized but sessions that have become established get acceptable service (maybe not great service, but they eventually complete without failing).

And from Sean Donelan:

This would be a useful plan B (or plan F - when things are really FUBARed), but I still think you need a way to signal it upstream 1 or 2 ASNs from the Extreme Congestion to be effective.

...what should the alternate queue plan B be?

Probably not fixed capacity numbers, but a distributed percentage across different upstreams. ...

  Session protocol start packets (TCP SYN/SYN-ACK, SCTP INIT, etc) 1% queue
  Everything else (UDP, ICMP, GRE, TCP ACK/FIN, etc) normal queue

And finally why only do this during extreme congestion?  Why not always do it?

The thread contains over 50 messages and doesn't reach a specific call-to-action (that I could detect), but it does show similar problems and similar potential solutions.

I would have thought there'd be a few old telecom folks on the NANOG list, but I guess not...

Do Quants understand power laws?

Yesterday's Wall Street Journal (Friday Sept 7th) had two more articles on the August turmoil in financial markets (pay-for links here and here).  In the second article, they quote Former Federal Reserve Chairman Alan Greenspan,

"The behavior in what we are observing in the last seven weeks is identical in many respects to what we saw in 1998, what we saw in the stock-market crash of 1987, I suspect what we saw in the land-boom collapse of 1837 and certainly [the bank panic of] 1907," Mr. Greenspan told a group of academic economists in Washington, D.C., last night at an event organized by the Brookings Papers on Economic Activity, an academic journal.

That's all very well, but the current crew is using "sophisticated" computer models to drive their trading.  Nowhere in either article is there any discussion of these models and yet that may be the key to what's gone wrong.  In any case it recalls some interesting mathematics which appear in both nature and in commerce.

Power laws versus normal and log-normal distributions

The issue is how you model markets.  Small price changes occur very frequently while large price changes are very infrequently.  If you plot the number of changes of a given size against the size of the change on a log-log scale, you get a straight line, at least out some distance. Conventional finance assumes this fits a log-normal curve which eventually falls off.  But extensive analysis, by Benoit Mandelbrot and others, of historical price data in many markets suggests the data fits a fractal or scale-free model, i.e. follows a power law.  If so, the likelihood of extreme events (like Black Monday in October 1987) has been and is being consistently underestimated.

I've written on power laws before in "Scale free networks, long tails and big bucks - or maybe not," as they are the basis of Chris Anderson's, The Long Tail. 

Power laws point to big bucks for Amazon, but also to big problems for Long Term Capital Management which collapsed in 1998 as I wrote about in "Power law vs log-normal with really big bucks implications."

Are the quants using models based on log-normal or power law distributions?

A quick Google search suggests that very few even understand the issues.  Ed Wright and Yves Smith seem to get it, especially in the comments to their post.  But otherwise one has to wonder about the financial community...

If you'd like further information, Nigel Goldenfeld has an excellent review of Fractals and Scaling in Finance by Benoit B. Mandelbrot that he did for Physics Today back in 1998.

Navizon is building the wireless covereage map I asked for

Last week I lamented the sketchiness of most wireless signal coverage maps and went on to suggest someone should "Use handset software to build mobile coverage maps."  Now Shai Berger has pointed out someone who's already doing just that.  It's Navizon from Mexens Technology.  Their focus is wireless positioning, i.e. determining your location, but their approach includes downloadable software for smartphones, PDAs and handheld PCs.

Mexens is the provider of the first positioning system that combines WiFi, Cellular and GPS information. The Wireless Positioning System triangulates signals broadcasted from Wi-Fi access points and Cellular towers to help users find their way in most major metropolitan areas worldwide.
                  
It is based on a collaborative network called Navizon. This is why we like to describe it as a peer-to-peer positioning system. Our users are sharing information about the position of WiFi access points and cellular towers, and by doing so, they keep our map of the Wireless landscape up-to-date.

And they offer such information in the form of a Google Maps mashup!  Here's their take on mobile cell sites in a western part of Newton, Massachusetts USA.

With software on the WiFi enabled mobile devices they can capture every WiFi signal any of their contributors pass.  Here's their map of WiFi signals on the northern edge of Natick Massachusetts.  Red are secure access points and green are unsecure, i.e. open, access points.

Clearly, Navizon participants have driven through the area, but no one has gone into the residential neighborhoods, as I'm confident this community is riddled with broadband users and residential access points.

While the website is in English, coverage is global.  Here are 200 of the more than 37K signals they've mapped in this part of London:

And 200 of more than 87K signals mapped in western Paris:

This is fascinating!  But I have other things to do now, so I'll print out their 20 page white paper to read at a later point.

Following more feeds in less time — BlogRovR

I'm a feed junkie.  Perhaps not as bad a some, but at one point my Bloglines account peaked at over 250 feeds.  Meanwhile, I keep stumbling on additional feeds I wish I could follow, but there's just no time.  Long ago I established a "Probation" folder for feeds I might want to follow and a "Too busy" folder for feeds like Slashdot where even scanning headlines is a problem.  Gradually more and more feeds ended up in "Too busy" or just unsubscribed. 

Now I've found an interesting solution — BlogRovr from Stickis.  They advertise themselves as:

the perfect companion for blog reading. Tell Rovr which blogs you like.  From then on, as you browse, Rovr fetches from them stories that link to the what you're browsing and shows them to you right on the page.

This happens via a tiny tray that briefly slides in from the right when you hit a new page (blog or otherwise) for which there is related content in any of your BlogRovR subscription. See the example on the right.

After two weeks of use, my BlogRovr subscriptions now include not only the feeds I subscribe to in Bloglines, but every other RSS feed I've stumbled on that looks even mildly interesting.  The net effect is that, if a mildly interesting blog covers a subject I otherwise happen read about, I find out about the related article(s).

I'm now removing overlapping coverage from my Bloglines subscription.  Who needs to see everything from all top 30 VoIP bloggers when most (certainly the top ten) present slightly different shades of opinion on the same set of stories?

I now follow the busiest sites in headlines-only mode and rely on BlogRovR to point out related articles.

So far, this appears to be a winning combination!

Benefits of competition between local governments

I've just had an email discussion about part of an article I wrote last spring for the July issue of Internet Telephony magazine.  My topic was broadband Internet access policy.  Among other things, I questioned the legal and regulatory policy approach taken by nearly every country in the world.  At least one reader appreciated my point so let me excerpt it here.

... it would be a terrible idea to impose any particular solution, however good it seems, on a national basis.  <This> point ... was best expressed by U.S. Supreme Court Justice Louis Brandeis more than 70 years ago in New State Ice Co. v. Liebmann, 285 U.S. 262 (1932): "It is one of the happy incidents of the federal system that a single courageous state may, if its citizens choose, serve as a laboratory; and try novel social and economic experiments without risk to the rest of the country."

The United States has 50 states and more than 22,000 local municipalities. The richest laboratory for broadband policy would be to give 22,000 municipalities permission to do whatever they want, and then see what emerges. Federal policy might be required to give them permission; (i.e., to undo decades of communications regulation and restrain local lobbying by existing national monopolies), but competitive public policy choices would be preferable to risking all on a single national policy, especially in an area that's evolving as rapidly as broadband connectivity.

Of course, we do have one fallback.  Because there are several hundred national governments on the planet we can argue to our national governments, "We're falling behind.  Look at broadband in country X."  But it's hardly as compelling as the local-to-local argument "City X is booming while our population is migrating away and our tax base is shrinking."  Now that's something a politician can understand.

In 2005, I referred to this approach as edge-based policy.  I have yet to hear a counter argument...  Any takers?