The battle of the middle... A Long Tail lesson...

Matt Hurst of Intelliseek has done some simple statistical analysis on the beta "PubSub LinkRanks 1000" and shows evidence of something that we've been noticing ever since we started computing the PubSub LinkRanks:

Matt Writes: "... not surprisingly, the higher you are ranked, the less likely you are to change rank - or the more stable the rank is."

If Matt had been able to see the full set of millions of ranks that we compute, he would have noticed a second "region" of very stable ranks. That region is at the bottom of the ranking scale -- sites that never get linked to by anyone. The harsh reality is that those with the lowest ranks tend to stay that way. Those with the highest rank tend to keep their high rank.

The really interesting part of the data collection represents those sites that are in the middle ranks. The mid-ranked sites have the most volatile ranks -- they gain and lose rank in drastic swings on a day to day basis. It is in the middle ranks that the real "battle for attention" is taking place. In those middle ranks, isolated from both the peaceful zone at the "top" occupied by those who have already "won" and the quiet zone at the bottom where live those who haven't even begun to fight, we see a constant and vibrant battle of ideas, personalities, voices and styles.

Of course, this vibrancy of the middle is what is to be expected given everything that we know today of power laws, network effects, and the long tail. The folk at the top of the pile have the "authority" or "popularity" that they do today in large part because they had it yesterday. The folk who are at the bottom of the pile are there today because that's where they were yesterday. It's the folk in the middle -- on their way up or down -- that see real movement on a day-to-day basis.

There has been a great deal of posting in the last few weeks that is critical of ranking systems. Some of the more astute critics have pointed out that one of the really unfortunate things about ranking systems is that they tend to focus attention at the top of the scale. The problem is that in almost all rankable systems that deal with information and ideas, the stuff at the top is where you'll find things that everyone already knows about or, those at the top of the scale tend to reflect points-of-view that are well known, broadly accepted and are generally predictable... Unfortunately, you can't just invert things and look at the bottom ranks as an alternative. The "signal-to-noise" ratio down there isn't very good...

The really useful place to look, if new information or new ideas is that you're interested in, is the middle ranks. As Aristotle said long ago: "Seek the mean." That's where the good stuff is.

bob wyman

Comments

[Greg Linden]

Hi, Bob. Any thoughts on how many blogs there are in this middle rank of good stuff weblogs?

On the Ask Jeeves weblog at

http://blog.ask.com/2005/07/what_feeds_matt.html

Jim Lanzone said that only about 1.1M weblogs have even one subscriber. The subset of interesting weblogs -- blogs that are interesting enough to have at least a couple dozen readers -- is likely a small fraction of that.

What do you consider to be the end of the middle rank? How many of the 16M weblogs out there would fall into the upper or middle ranks? How many of the 16M weblogs are interesting enough to be considered good stuff?

[Dan Isaacs]

"The harsh reality is that those with the lowest ranks tend to stay that way. Those with the highest rank tend to keep their high rank."

An excellent point, that I think is equally applicable to socio-economics, or pretty much anything else. :)