Machinist

Google|Search Engines | 2008.01.07 • 17:11 EST

Wikipedia founder's search engine gets bad reviews

"We are aware that the quality of the search results is low," Search Wikia points out in a bold-faced notice on its site, but the concession isn't silencing many critics. The new search engine, an ambitious effort spearheaded by Wikipedia founder Jimmy Wales, has been so long in the making -- and so overhyped -- that on seeing the product for the first time today, critics couldn't contain their scorn.

TechCrunch's Michael Arrington calls it "one of the biggest disappointments I've had the displeasure of reviewing." And at Search Engine Land, Chris Sherman labels Search Wikia "essentially useless as a search engine," and he wonders if the project can ever succeed, and, indeed, if it's even necessary.

As Wales conceives it, Search Wikia is not just a new kind of search engine, it's an entirely new kind of Web project.

Jimmy Wales wants people out on the Internet to help build something as complex and useful as Google, in much the same way that people took his desultory online encyclopedia and transformed it, over the years, into the world's best reference source. In fact, this project is even more ambitious -- here people are working not only to edit text but to edit computer algorithms and policies, the arcane set of systems that companies like Google need an army of developers to run.

Can such a thing ever work? Wales can be a big talker, but mainly he's self-effacing. When I asked him, a few months ago, about the difficulties of his project, he admitted, "I could fail. I have no idea. But I'm going to have fun trying."

He also noted that the first version of the search engine wouldn't be very good at all. He was right. You can try it out here. I ran many searches and, like other testers, found that a great deal returned poor results.

As one example, type in Paul Greengrass. The first result is the Amazon entry for the "Bourne Ultimatum" DVD (which Greengrass directed), followed by several haphazardly ordered links to reviews, sketchy DVD stores, and questionable foreign sites. Search for the same term in Google and you find, first, a link to Greengrass' filmography at the Internet Movie Database, and next a Wikipedia entry, which tells you Greengrass is a kick-ass movie director. The results page is superb.

Search Wikia's spotty results are by design. The trouble with the sort of project Wales is building is that, even if it may one day succeed, it's got to start off sucking.

At its birth, the Google search engine pretty much beat out every competitor -- that's what made it so successful so fast. Search Wikia, like Wikipedia, will improve only if people help it. The site allows you to rate the search engine's results -- you can do so by clicking on the stars that come up next to some links. You can also alter its white list (which tells the site which pages to include in its results), and, more generally, you can help create new policies determining how the whole thing will work.

Wales wants people, now, to start doing that work. In an interview a few minutes ago, he told me, "We have enough features there that people will find useful in their day-to-day work. They'll find that a reason to stick around and use the product even while the search results are improving in quality."

In time -- a long time, at least two years, Wales says -- Search Wikia will return results that are as good as those of the other engines.

But search quality is not his only goal. Wales says we need an open-source, transparent search engine -- one that explains why it's returning the results it is -- because search determines how we understand the world. What we get on a Google results page is too consequential to keep the method behind those results hidden.

Really, then, the debate over Search Wikia is more about philosophy than functionality. I mentioned to Wales that he's got a chicken-and-egg problem -- he needs people to use the search engine in order to improve it, but people aren't going to use a search engine that gives them lousy results.

Sherman suggests that Wales' push for transparency and community may not be enough of an inducement for people to join the project:

But Wales believes people are yearning for transparency. He says that he has no worries that nobody will want to work on Search Wikia; what he worries about, in fact, is that he'll get more volunteers than the project can effectively manage.

Wales was right about this for an online encyclopedia. Eventually we'll know if he's right about search, too. But not soon.