Trusting Wikipedia: a sea change

Here’s an interesting development. Long and acrimonious battles have been fought over the question “Can you trust Wikipedia?”. Now, at last, there’s a new answer to this question.

First, let’s recap the arguments in the original debate, and then I’ll explain why I think this news means a sea change for how Wikipedia will be judged and used in future.

Whether you can, or should, trust Wikipedia depends on several factors. Most importantly, it depends on what you mean by the term “trust”. For casual use, on uncontroversial subjects, sure you can trust Wikipedia. It’s particularly good at covering technology, biography, science and some history, for example. It is less good at politics, environmental issues or anything where camps of opinion are divided. Recognise that, and Wikipedia is your friend.

Can you cite it in academic articles? How about legal work? Unless you’re referencing public opinion, no, you can’t. Ever. Under any circumstances. You need to go to a primary source, or a relevant secondary source. No encyclopedia qualifies, especially not one written by its users, no matter how well it is done. Whether or not Wikipedia is better than the Encyclopedia Britannica isn’t even the issue. Even if it is, it still doesn’t qualify, and is even less citable because no single person or group is accountable for its content. (Unless, of course, you’re writing about Wikipedia itself.)

Can journalists use it as a source? Not unless you want to look like a lazy, incurious and gullible fool, you can’t. Even for really uncontroversial stuff, like definitions, rather go to a reliable dictionary. Same goes for business presentations. Wikipedia itself admits as much:

It is in the nature of an ever-changing work like Wikipedia that, while some articles are of the highest quality of scholarship, others are admittedly complete rubbish. We are fully aware of this. We work hard to keep the ratio of the greatest to the worst as high as possible, of course, and to find helpful ways to tell you in what state an article currently is. Even at its best, Wikipedia is an encyclopedia, with all the limitations that entails. It is not a primary source. We ask you not to criticise Wikipedia indiscriminately for its content model but to use it with an informed understanding of what it is and what it isn’t. Also, as some articles may contain errors; please do not use Wikipedia to make critical decisions.

However, it is impossible to deny that Wikipedia is very useful. In all these areas, Wikipedia makes an excellent starting point for research. It neatly summarises existing knowledge, and a well-referenced article provides both a comprehensive overview and pointers to further material that may be closer to a citable source.

Now the question remains: How reliable is this “preliminary” research? Especially on controversial subjects, my experience is: not very. I once looked for some basic facts and arguments around recycling paper, and thought Wikipedia might make an excellent compendium of the differing views. It did: I got three different answers, but unfortunately I got them on three different occasions during the course of a week. The only thing it proved conclusively was that there’s a lot of debate and misinformation around the subject, and that Wikipedia cannot be relied upon as a source in journalism.

The WikiLab at the University of California at Santa Cruz, however, has come up with a solution. It paints text backgrounds to indicate the relative reliability of the text. How does it judge this? It assigns a ranking to contributors based on the edits they’ve made to pages, and how long those edits have remained valid before being changed (if they were changed at all). Regular contributors who don’t get edited much therefore rank higher than a Stats SA employee removing all criticism of the South African government’s Aids policy, or the New York Times employee who thought “jerk jerk jerk jerk” was a helpful description of George Bush.

The site getting slashdotted didn’t contribute to my attempts to test it out — on the recycling pages, for example — but if the principle works it would be very useful. I think this really is a sea change. It makes Wikipedia dramatically more reliable and will forever alter the way people judge the information it so efficiently provides. It’s still not citable, by strict academic, legal or journalistic criteria, but it will surely be seen as a far more trustworthy source now.

Even more significantly, with a Wikitrust Rating, Web 2.0 users, already revered as Time Person of the Year for 2006, will have what may prove to be an even more significant measure of worth than their Google PageRank, Technorati authority, Digg ranking and Hot or Not score. It’s still not actual money, but it will look pretty good on your CV.

Kudos to Luca de Alfaro, B Thomas Adler, Marco Faella, Ian Pye and Caitlin Sadowski for not just adding to the hot air about Wikipedia, but doing something to address the problem.

(First published on my own blog on August 31 2007.)