Search, a burden or a work tool

Information search is an essential part of every type of business today. Employees or customers spend valuable time searching for useful information. Therefore a well-designed, reliable search engine with fast and accurate results is a fundamental part of every business and should be chosen with care.

The need for search applications has grown steadily since the emergence of Internet as the largest centralized information-gathering instrument. Although the problem of finding relevant information seems rather simple, the solution is normally much more complex especially when the number of documents is large and the search phrase is not exact. The larger the information domain, the more effective algorithm is needed to acquire an accurate result-set in a simple and effective manner.

Considering the amount of time spent managing, retrieving and analyzing information, one can only imagine the size of lost revenue due to delays in time-to market or low employee productivity. Unsatisfied customers failing to find what they seek will simply discard an unreliable search engine. In most cases the search engine simply will fail to find any results at all or the result-set is too large to be of practical use. The problem is the same whether the piece of information searched resides on a file server, relational database, corporate intranet, content management application, off-line electronic archive or any other information source.

The Challenge

While most organizations spend millions of euros on building advanced information systems containing valuable information, very few invest in an effective and coherent information management infrastructure to capture and utilize that information. After all, information existing on file servers, databases, electronic archives or any other information repository is not worth much if not found effectively when needed. Companies that make such investments often suffer from inefficient solutions that fall short of achieving high quality search results because:

• High quality search phrase is required in order to produce adequate results i.e. the end user should often provide exact keywords in order to find reliable information
• The search engine fails to find any results or provide comprehensive results. An empty result-set will cause frustration and makes the search process awkward and tricky. The end user will itself need to act as a search engine and sort through the results to find relevant information
• An enterprise-wide integration to cover all existing sources of information is not provided
• The solution requires large administrative overhead and are not simple to configure, integrate, deploy or install

A direct result of these issues is frustrated employees wasting valuable time spent on finding information or displeased online shoppers leaving for another online dealer where they can find the required services or products probably only a click away. It does not take a mastermind to figure out that the economical impact of these limitations is not sound for the business and must be dealt with seriously.

The solution

An enterprise search engine should comprise a number of key functions to address issues mentioned above. These functions should be provided on a robust platform guaranteeing optimum performance while maintaining ease of administration. A flexible Application Programming Interface that’s supports a variety of today’s integration technologies should be provided to easily integrate existing or new enterprise information repositories. A number of search engine providers offer attractive license pricing while the integration is costly and time consuming. Some providers offer solutions with complex and costly administration and maintenance or require high-priced third party products which decimate the value added by introducing a powerful search engine.

The search engine should deploy a sort of fuzzy algorithm which enables approximate search. The search phrase may be exact, near or far from the information record(s) being searched. It may also contain spelling errors.

The search solution should be customizable in order to fully implement the specific requirements of a client. A result-set may need to be tailored to different kind of end users with different authorization profiles. Same data accessed from different channels (mobile, corporate intranet, Internet etc.) may need to be presented, categorised, formatted or sorted differently. Another important customization is being able to implement specific filters that manipulate the search result to address specific business rules. The optimum match for a search phrase might not necessarily be the best search result. The result-set may need to be manipulated in to rank best sellers, best authors or most viewed items higher than simply the best match for a given search phrase. Business rules might also be different for different information categories, user types or access channels.

Global presence dictates another important feature that a search engine should provide. Multi-language support is essential to implement a coherent enterprise search solution which can easily be deployed in different countries and languages.

Considering the growth and the ever increasing amount of information, it is crucial that the solution chosen is scalable. A variety of attractive solutions in the marketplace work quite well until a threshold is reached after that the performance is poor or unacceptable.

Today’s dynamic marketplace requires business intelligence where user behaviours and statistics play an important role to present adequate search results.

Statistical matching suggest relevant search keywords based on search performed by other users. Normally an advanced pattern-matching algorithm is used to create a virtual network of relationships among each user's individual search keywords and corresponding search results. Each individual user has a distinctive search pattern which will be matched against other similar patterns. In this way the search engine is able to automatically predict and suggest results which may be relevant or interesting to other users. Added value is gained by increasing sales on similar products.

Adaptive ranking is used to sort the list of returned search results based on predefined ranking priorities. The search engine is configured to automatically sort down relevant items that are statistically rarely chosen. The ranking of items in a result-set becomes simply dynamic and will change depending on search statistics, user behaviour and predefined ranking and sorting priorities.

Case study

AdLibris is one of the largest online bookstores in Scandinavia recognized for offering the market's lowest prices, fast deliveries and premium customer support. The company was founded in 1997 and offers millions of book titles in a variety of categories. The increasing number of titles together with aggressive growth in client online orders brought also real challenges.

“We had been on the lookout for a good search engine for a long time but had not been able to find one that could meet all our demands. We needed a search engine that was able to give good results even if the user didn’t know the exact title or spelling of an author’s name, furthermore we needed the search engine to be language independent. Adeptic’s search engine didn’t only give us this but also delivered lightning fast searches and was easy to adapt to our environment”, says Fabian Fischer – Co-founder of AdLibris.

As opposed to conventional search where the user should normally act as a search engine narrowing down the search criteria, AdLibris requirements was to simplify the search process even further. The user may be able to input a search keyword close to the intended result regardless of the type of information, i.e. title, author ISBN etc. The search engine should suggest the closets possible hits.

AdLibris turned to Adeptic Technologies to implement a solution based on Adeptic Enterprise Search Suite (AESS) as its main search tool. Working together with Adeptic, AESS was integrated into AdLibris technical environment. The solution was tuned and configured based on AdLibris requirements. A total number of 20 indexes were generated. All indexes are refreshed on regular basis.

“We used MS SQL Server for all indexing operations before. Indexing five million books took almost 8 hours. After we installed AESS, the average indexing time for the same amount of data was reduced to 5 minutes . Before AdLibris implemented the AESS, almost 30% of all searches returned no result due to customers misspelling the search word. Now this problem is completely gone. The results are stunning,” continues Fabian Fischer