Tuesday, December 11, 2007

A Librarian's Worst Nightmare

Yahoo! Answers, where 120 million users can be wrong.

By Jacob Leibenluft
When it does battle on the Web, Google rarely loses. Last year's closure of Google Answers, however, marked a rare setback for the search giant. An even bigger shock is that Yahoo! succeeded where Google failed. Yahoo! Answers—a site where anyone can post a question in plain English, including queries that can't be answered by a traditional search engine—now draws 120 million users worldwide, according to Yahoo!'s internal stats. The site has compiled 400 million answers, all searchable in its archives. According to the Web tracking company Hitwise, Yahoo! Answers is the second-most-visited education/reference site on the Internet after Wikipedia.

The blockbuster success of Yahoo! Answers is all the more surprising once you spend a few days using the site. While Answers is a valuable window into how people look for information online, it looks like a complete disaster as a traditional reference tool. It encourages bad research habits, rewards people who post things that aren't true, and frequently labels factual errors as correct information. It's every middle-school teacher's worst nightmare about the Web.

The site's home page, which offers a real-time snapshot of the dozens of questions posted every minute, provides a good sense of users' favorite topics: relationships, computers, homework, pregnancy. These queries reveal why something like Yahoo! Answers might draw so many visitors. The questions—"Why does the stomach make funny noises when it's hungry?" and "How do stoplights sense a car?" for instanceare difficult to answer with a traditional Web search. If you're looking for advice on your new haircut or help on the third question on your precalculus problem set, Yahoo! Answers might be your best option. Most strikingly, Answers draws a large enough crowd that you're likely to get an answer almost instantaneously. Post a semicoherent question and the responses will come within minutes, if not seconds.

For educators fretting that the Internet is creating a generation of "intellectual sluggards," the problem isn't just that Yahoo!'s site helps ninth-graders cheat on their homework. It's that a lot of the time, it doesn't help them cheat all that well.

Take a popular question asking about common customs and beliefs among Native Americans. In theory, this is the kind of query Yahoo! Answers is made for. It's more easily asked in the form of a complete sentence rather than in a series of search terms, and it has a factual answer some users might know.

How did Yahoo! Answers do? On the plus side, the question received an impressive 97 different answers, including a few knowledgeable responses and helpful references. But several of the postings were misleading, confused, or just plain wrong. If you started off uncertain, it's hard to imagine you would read the responses and feel any more confident. To top it off, the answer eventually chosen as the "best" was, enigmatically, "American pie."

In some academic areas—physics is one I've noticed—the Answers community consistently does an impressive job of providing accurate answers and a clear explanation of how to get them. But in other disciplines, the site's record as an educational tool is, to put it charitably, unreliable. A recent question about dual citizenship attracted 12 answers in just two hours; some of the responses were nearly accurate, many partially true, and others entirely false ("yes it is true they outlawed dual citizenship in 2001 due to people going to canada and the uk for free health care while they were not paying taxes in that country"). Another thread on the relationship between Iran, Saddam Hussein, and Osama Bin Laden offered a few insightful responses about Sunni-Shiite politics surrounded by enough noise—"No one really cares except for people like yourself!"—to confuse or annoy anyone who might pose the question earnestly.

Some people might look at this mixed record and think that Yahoo! Answers is just like Wikipedia. But the differences between the two sites say a lot—about why Wikipedia has been such a success, why the Web's leading reference site is so hard to replicate, and how Yahoo! Answers has become so popular despite its flaws.

Like Yahoo! Answers, Wikipedia isn't perfect. But for savvy browsers who know how to use it, Wikipedia is an invaluable source of factual information. In the last two years, there's been a heated debate over whether Wikipedia is as trustworthy as Encyclopedia Britannica. This obscures a crucial point: Wikipedia is at least reliable enough that such a question can be asked. Take my word for it—no one is going to make any such claims about Yahoo! Answers any time soon.

Wikipedia's greatest virtue is that it is self-editing and self-correcting. The site's draconian efforts to consolidate pages and remove entries that aren't deemed important have a crucial side effect: They focus users' energy on revision rather than addition. By contrast, Yahoo! Answers is more devoted to quantity than quality. It struggles to prevent repeat questions from appearing over and over again. And unlike Wikipedia, the Yahoo! community expends far less energy trying to hide dubious or just plain incorrect contributions, despite a community rating system designed to flag them. Often, a correct answer will be hiding somewhere on an Answers page, only to be obscured by a tide of wrong or off-topic material that never gets erased. Wikipedia pages are subject to constant revision. If a vandal screws with an entry, one of the site's busy janitors cleans it up. If new information becomes available or a new user devotes energy to making improvements, then a Wikipedia article will get better even years after it's first posted. Yahoo!, by contrast, "closes" questions to new answers after a week, although users occasionally post comments afterward. While the site's answers live forever on the Web, each question attracts only seven days' worth of collective wisdom.

The small, almost obsessive community that built Wikipedia created a culture of reliability. For contributors to see their writing on the site, they must submit information that's clear and accurate enough to survive the scrutiny of other users. Yahoo! Answers has created a more formal, yet far less successful, reward structure to identify top users. Every time you post an answer, you earn two points. If you win a "best answer" distinction, you get 10 points. (The person who asked the question gets the opportunity to select the best answer; if they choose not to, it is selected by community vote.) This system highlights the site's greatest strength and its greatest weakness: Everyone gets credit for answering, but there's not a huge push to make sure the answers are right.

As its devotees would point out, Yahoo! Answers allows you to ask questions Wikipedia would never touch. Many of the site's users are simply looking for advice, local knowledge (like a restaurant recommendation), or an opportunity to start a discussion. But for these questions, too, the quality of the responses varies widely, and users can be stuck struggling to separate the good answers from the bad.

Even though Yahoo! Answers is so frequently sloppy and inaccurate, it's still the juggernaut in its field. Despite a rapid proliferation of answer-giving sites—Amazon.com's recently inaugurated Askville just joined a crowded field that includes Answerbag, WikiAnswers, AnswerBank, and Ask Metafilter—Yahoo!'s is still by far the most popular. And in the question-answering game, size matters. While the others have a few clever features (like Answerbag's efforts to separate "educational" and "conversational" questions) or a more specialized community, the sheer magnitude of Yahoo!'s community gives it the upper hand.

After all, while Yahoo! Answers and its peers are classified as reference tools, what they actually provide is social networking. The thrill of Yahoo! Answers comes in the instant interaction: the scores of questions, the immediate back-and-forth discussions, the opportunity to feel like an expert, and, eventually, the promise a query will be labeled a "Resolved Question" no matter how difficult.

For a passive reader, this has the same value as listening to two random guys at a bar talk about what to do if you are driving during a tornado. You may not learn very much by eavesdropping—and you certainly shouldn't trust what you hear if disaster strikes—but that isn't really the purpose. The lesson Yahoo! Answers teaches is that, for millions of people on the Web, it's less important to get a good answer than to get someone to listen to your question in the first place.

Jacob Leibenluft is a writer from Washington, D.C.

Article URL: http://www.slate.com/id/2179393/

No comments: