Scale free networks. long tails and big bucks - or maybe not

Some of my blog posts serve to organize a set of references on a topic that's important to me.  This is such an occasion -- hopefully one or two others may share this interest.

Back in 2002, I really enjoyed Albert-Laszlo Barabasi's book on networks, Linked: How Everything Is Connected to Everything Else and What It Means, but haven't paid much attention to scale-free networks since then.  Yesterday I stumbled on this interesting presentation by Chris Anderson at "SciFoo" last week, which prompted me to revisit the subject.  The notes in Anderson's presentation include two great references (Limpert et al., Mitzenmacher) for the mathematically inclined, and the current Wikipedia article on Scale-free networks has excellent content and a lot more references.

New to me, are the diverse models that generate networks which are scale-free but differ in other meaningful characteristics.  Barabasi discussed preferential attachment, a model which generates scale-free networks with highly connected nodes near the core.  These networks are robust even if a large fraction of the nodes are removed.  However the Internet, while a scale-free network, has it's highly connected nodes closer to the periphery, so the preferential attachment model doesn't apply.  And as Wikipedia currently puts it,

Indeed, many of the results about scale-free networks have been claimed to apply to the Internet, but are disputed by Internet researchers and engineers.

The "big bucks" part of my headline was prompted by Chris Anderson's The Long Tail: Why the Future of Business Is Selling Less of More.  Anderson's thesis, originally published in Wired magazine, is that the popularity of items in certain markets, like media, follows a power law -- the same law that applies to the connectivity of nodes in a scale-free network!  In such a market, a few products have enormous sales but there is a very long tail of products that have only a few sales each.  This long tail goes way out, so taken together, sales on the long tail represent big bucks.

Since Anderson's book was published there's been considerable argument about the long tail.  Nicholas Carr has good summary of the argument between Wall Street Journal columnist Lee Gomer and Chris Anderson.  But Andrew Odlyzko (on a private economist list, quoted here) has the best overall answer to the questions Lee Gomer raised.

As with the Internet, the key question is: does the phenomenon follow a power law (as scale free networks do)?  If however, it follows a log-normal distribution, there's a lot less money to be made selling products out on the long tail as illustrated by this slide in Anderson's presentation:

As Anderson puts it:

But here’s the problem. I’ve been assuming that the powerlaw is the “natural” shape of all these markets. But there are other distributions that start off straight on a log-log chart, then fall off the line not necessarily because of a bottleneck effect but because they’re simply not powerlaws. The best example of this is the “lognormal” distribution, which often stays straight for several orders of magnitude before sloping downward. How to tell whether a market that falls off the line is a natural powerlaw shape distorted by a removable bottleneck or a natural lognormal market that will look like that no matter what you do? In markets such as film or television archives, this could be a billion-dollar question.

Indeed!

Comments

[Ville Saarikoski]

Please have a look at
http://www.smartmobs.com/archive/2006/12/12/thesis_on_email....html
It hopefully might interest you.

I just completed my PhD thesis in which I studied why the west failed with the mobile internet and Japan was successful. I conclude the thesis with a hypothesis:
The western mobile internet has contained no elements which are efficient in a scale free sense. By adding elements which are efficient (e.g. mobile email, wlan, flat rate pricing, search engines etc) you will be successful with the mobile internet business.
This is cause and effect- or at least very close

I noted you are combining scale free with long tail. I have
been doing that and it proves a very interesting thing to play with. If you add to it the works of Lessig and yochai Benkler and the idea of building digital commons, the picture gets more interesting.

Let me know if this raises any questions/ideas...

[Beaumont Vance]

You are absolutely correct that the issue of power laws and the work done by Barbasi is a critical issues in the markets. Few outside o academia seem to have noticed anything except that markets involving IP are not Gaussian. I am a Risk Manager for Sun Microsystems and just discovered Barbasi’s work. It has powerful implications for the markets. Thanks for bringing it up. Do you have any further references regarding work in this area?

Like you am a technical specialist who writes frequently out of my passion for these things.

[brough]

I have written two other related posts, mentioning interesting work by Mandelbrot. Look here:
8/18/2006: http://blogs.nmss.com/communications/2006/08/power_law_vs_lo.html
9/08/2007: http://blogs.nmss.com/communications/2007/09/do-quants-under.html