Esotropiart

Why do attempts at developing natural language search engines almost always fail miserably? What is meant by "natural language"? Why should you care? Natural language, aka plain English (Portuguese, Spanish, Mandarin, and so on) is a logical construct of words, phrases and expressions that intelligent beings use to communicate meaning and intent. Most modern search engines have very little understanding of intent and thus return millions of results that have little or nothing to do with what a person is looking for. This is because search engines break apart typed queries into single words and weigh their importance. Many common words are thrown out of the equation, deemed as insignificant by the powers that be.

Natural language search engines attempt to, among other things, bring these overlooked words back to help determine the user's intent. Ask.com is one of the more well-known "natural language" search engines. In reality, Ask.com is little more than a group of experts who constantly build a vast database of links and answers to common queries, and the realtime intelligence of their technology leaves much to be desired. Ask.com has already experienced severely lacking success and has lost its place as a significant proponent of true natural language technology. A number of similar "pseudo-smart" services are on the scene - some more successful than others. Most rely on a combination of automated and human interaction with web content in order to form complex web indexes, a constant behind-the-scenes work in progress. Seemingly little has been done in the way of interpreting the language of a query in order to answer questions on the fly or make bold assumptions as to the desired web destination or information hiding in the typed words. Many search engine developers have concluded that such logical, realtime interpretation is downright impossible.

A few days ago I caught mention of yet another search engine startup called Powerset. What caught my eye was the sentence that mentioned something about natural language search. The company is still in its beginning stages, and there is no workable (publicly accessible) search engine or demonstration that I am aware of. A select few have been exposed to the promising technology's results, but little specific is written about the experience. The entrepreneurs behind Powerset are intentionally keeping things hush-hush, I suppose either in fear that someone will steal or squash their idea or so they can dodge humiliation if hopeless failure ensues. At any rate, Powerset has received quite a bit of attention from techies everywhere. The effort purportedly has quite a few million dollars of venture capitalist backing already. It will be interesting to see where they do or don't go with it. I for one am very interested in intelligent web search results, especially stemming from full English sentences and questions. I'm tired of wading through billions of unrelated results. To be honest, I've never been overly impressed with Google's (to name a giant) search results. Sure, the speed and number of results are great, but relevance to my intent is rarely something to write home about.