Why enterprise search is not internet search

There's a reason why Google is a verb. Yahoo may get more visitors than the stock price might lead you to expect, Microsoft is still pumping research and money into Live Search, and there are plenty of new search engines trying to steal even a few percentage points.

But with around 60 per cent of search traffic, Google is where most of us turn when we need to find something on the web. Wouldn't it make sense to use the same techniques to make it easier to find information within the enterprise?

Depending on which survey you look at, businesses waste the equivalent of 10 per cent of salary costs (says the Butler Group) or information workers waste around three to four hours a week a total of five weeks a year - because they don't find the information they're looking for a third to a half of the time (IDC and HP).

In small and medium enterprises in the UK, a YouGov survey this summer found 80 per cent of managers and directors waste up to an hour a day looking for documents. And according the same HP survey, 60 per cent of IT managers spend "a high proportion of their time" dealing with employee requests for help finding basic information on the network.

That means it's worth investing in the right search technology. That doesn't necessarily mean using Google's free SiteSearch on your intranet, familiar as the interface may be to users. For one thing, the information people need to work with is rarely in the intranet to start with, and more importantly, the techniques that work well on the web aren't as well suited to internal search.

Rank and relevance

Although Google doesn't disclose all the factors that it uses to match web pages and rank them in by what's most relevant, the original PageRank algorithm was based on how many other web sites link to a particular page. That's not going to help when you're looking at a file share or the list of outstanding customer complaints where no-one is explicitly making the links between documents to base ratings on, or when the most relevant document for you isn't one that's been accessed frequently by other users.

Deriving relevancy automatically is more attractive than building and maintaining a custom thesaurus of terms and concepts for your enterprise, but the latest enterprise search tools don't rely on pre-written dictionaries anyway.

Google says its Search Appliance for enterprise relies "very little" on PageRank and uses many other algorithms, although it doesn't give details. There have been some impressive customer testimonials for the Google Search Appliance Raytheon says 84 per cent of corporate users find the result they want in the top three links but that still means Google search in the enterprise is very different from Google search on the web. And for once Google is some way behind Microsoft, which became a market leader in enterprise search alongside by buying FAST.

"No-one is going to say Google can't develop good technology," admits Craig Carpenter, the vice president of marketing at enterprise search and knowledge management company Recommind. "They have all the resources in the world and the core technology is very good for web search. But building a relevancy model related to how many people are linking to particular pages is a very different challenge from the enterprise scenario."