Polska wersja

Marcin Sochacki (Wanted) Blog, Internet, Technology

» Don't trust the ranking(.pl)!

Regardless of where you look for the marketing data on Polish Internet, you will most likely notice the statistics published by Gemius company and their site ranking.pl. One can't deny the fact that Gemius is practically a monopoly in large scale statistics in Poland, but on the other hand it doesn't automatically mean that their data is correct.

During the last month many people noticed big changes in global stats, reaching even several million users weekly. For example in the period of 27.03–02.04 they have counted 15.4 million users in Poland and two weeks later the number was already 19.2 million. The last week (24.04–30.04) has shown 18.8 million. Such big changes clearly show that something's wrong. What's worse, not only the number of users has grown suspiciously, but also the proportions have changed dramatically, e.g. IE gained 5% of share and the alternative browsers noted a corresponding drop. Gemius itself stays silent and doesn't explain what's going on. On their blog they prefer to write about their new tool called “The Fairy” for predicting trends. I'm afraid that not only future, but also current statistics are examples of crystal ball science.

Gemius's method is obviously imperfect, but the results are often quoted as if they were objective, unquestionable information. The stats base on two sources: JavaScript code embedded in sites cooperating with Gemius and their toolbar, which tracks all visited websites. One should note clearly that those two sources will never be 100% sure:

As a result, the ranking.pl stats should be treated with great reserve. If the weekly variation of several percent is not a substantial proof, then I don't know what is. The ultimate insolence in this context is their claim that “the statistical error is approximately 0.01 percent with a confidence interval at 99 percent.” Riiight, in my opinion the error is rather closer to 5% in reality.

I have a hypothesis regarding the big growth of users count and change in proportions in favour of IE. Due to two recent important premieres at Microsoft: MSIE 7 and Windows Vista, people update their systems on mass scale. I don't have Windows to test it out, but I'm curious if updating IE6 to IE7 also deletes cookies. Even if it doesn't we surely have a higher than usual number of OS reinstallations and clearing cookies. New cookies are counted as new users, which is not the case here.

One shouldn't obviously blame Gemius for the fact that statistics are not precise, because frankly speaking they can't be better. What's sad is when many people use their numbers as objective and truly perfect, just based on the de facto monopoly of Gemius in Poland. One should remember the old Mongolian highlanders' proverb: “lie, big lie, statistics”.

Gemius problems were mentioned by many Polish bloggers. Marek Futrega also wrote several posts in English, e.g. this one.

2007/05/05 20:53 | computers/www/

» Comments

My blog


/ (69)
  computers/ (40)
    apple/ (1)
    google/ (14)
    linux/ (3)
    microsoft/ (9)
    other/ (4)
    spam/ (1)
    wikipedia/ (3)
    www/ (5)
  i_like_it/ (15)
    film/ (6)
    fun/ (1)
    photo/ (2)
    sport/ (6)
  info/ (4)
  media/ (5)
    ad/ (5)
  places/ (5)
    aconcagua/ (1)
    dublin/ (2)
    tricity/ (2)



Color free zone

This site is Wit 2.0™ compliant!