Friday, November 27, 2009

Arguing over a Meaningless Concept

I continue to be surprised that Wikipedia has never been on the receiving end of a Chutzpah of the Week award. I gave one to Jimmy Wales this past April; but that was not for anything specifically having to do with Wikipedia. Rather, it was for an outrageous act of cultural amnesia, which, ironically, could have been prevented had Wales taken the time to use Wikipedia itself. This week, however, we seem to have an act of chutzpah at the administrative level of Wikipedia through a dispute raised by the Wikimedia Foundation. The dispute was reported this morning on the BBC News Web site. It concerned a claim in a research report by Dr. Felipe Ortega, which seems to have been trying to track the level of participation in Wikipedia over a sustained period of time. Ortega claimed that, in the first three month of 2009, 49,000 editors "departed" Wikipedia, comparing this with 4900 departures in presumably the same period of 2008. (My weasel words are an attempt to get around the vague language of the BBC report.)

At the very least Ortega was trying to quantify just how participatory the Wikipedia enterprise was and consider whether or not participation may be dropping off. This seemed to be enough to provoke an indignant response from Erik Moeller, deputy director of the Wikimedia Foundation, and Erik Zachte, one of its data analysts, through a blog post. Their refutation, however, seemed to have more to do with biting Ortega's finger than with looking where he was pointing:

The confusion arose over the differing definitions of what constitutes an editor. Dr Ortega counted everyone who made one change as an editor giving a total population of three million people.

By contrast, the Wikimedia Foundation counts only people who make five edits or more as an editor. This gives an editing population of about one million people across all languages. Of that total, the English edition of Wikipedia has about 40,000 editors.

If all Ortega was interested in was participation, then it seems unfair to rap his knuckles for trying to settle on a metric that he could use consistently over different periods of time. His only fault may have been describing that metric in language that irritated the Wikimedia Foundation. Thus, the chutzpah of their refutation came down to infantile schoolyard reasoning: It's our ball; we get to make the rules. Having done so, however, they do not have appeared to provide numbers that, by any standard of measurement, would refute Ortega's more general claim.

This may be much ado about not very much, but the chutzpah of going after scholarly research in the name of institutional pride still deserves recognition.


Anonymous said...

If the crowd stops sourcing, then isn't the new paradigm in trouble?

Stephen Smoliar said...

Anonymous, if by "in trouble" you mean "liable to be revealed as no paradigm at all," you may be right; the history of information technology could well end up chronicling the lineage of naked emperors!

Dan said...

I've read several reports of a drop in participation, most of which bemoan the trend, or see it as an ominous sign of reduced something or other. Seems to me we may be missing a possibility: that most things deemed worthy of recording in Wikipedia have now been recorded. It's just possible the backlog has been largely satisfied, and further contributions will be largely about new things. That could be a good thing. Most of what I'm looking for these days I find.