Search Engines Just Aren’t That Into You

Scoble’s done it again; he’s gone and started a conversation, and this one is a pretty important and timely one for me… enough to rouse me from my blogging coma (or better said, obsessive work focus) to comment… Robert, in reviewing John Battelle’s The Search, makes the case for intelligent query refinement, giving a search for hotels in New York as an example:

So, what COULD search engines do? Well, first, give me some choices at the
top of the page. Why couldn’t search engines ask you these questions:

1) “are you looking for hotels in New York or named New York?”
2) Are you looking for hotels with free Wifi?
3) Are you looking for hotels with great views?
4) Are you looking for hotels nearby major tourist destinations?
5) Are you looking for hotels with above average ammenities like super
large bathtubs, well stocked minibars, etc.?

Danny Sullivan responds to Scoble with an emphatic YES!, stating in part (in a lengthy post worth a read if you’re “into” search):

“A search engine, unlike a librarian, can’t interrogate you. It can’t ask further questions to help you narrow in on what you are looking for… That’s why the regular trend of someone trotting out a super-magical “natural language” search engine is always laughable. The pitch generally goes something like, “This search engine is smart enough to know when you typed in a sentence about AIDS that you meant AIDS the disease rather than aids as in something that helps you.”

Danny mentions the past path of human editorial approaches (largely a bygone era as we humans are so expensive to employ…) and the currently in-vogue clustering approach.

Healthline doesn’t do hotels, it just does health. And it doesn’t do automated statistical clustering (though that’s not a religious stance, just a current state). Rather, it uses a well vetted medical taxonomy (overlayed by a lay-person synonymy) to do its query refinement.

Our query refinment will come in three buckets — broad, narrow and related — and in two flavors — visual and text taxonomy traversal.

So, for example, if you’re like David Hornik, and think you’re suffering from Narcissistic Personality Disorder (aka Venture Capitalitis), you can type it into Healthline and get refinements that not only help you get to the results themselves, but actually put your search in context by showing you medically (vs. statistically) where NPD fits in the world of all things “health”.

Of course, this is the tip of the iceberg (i.e., we’re closing quickly on v1.0 Beta); as we continue to build out our technology and UI, we’ll continue to expose additional value to the user through our semantic understanding of the incoming query.

We should have a build that’s sufficiently stable (and representative) to show off privately at Web 2.0; drop me a note if you’d like a 5 minute private demo.

P.S. Don’t think we’re so old-school as to think it’s all about our generated content (i.e., our taxonomy); tagging has a place in our world too.