Finding a solution

The enterprise search market, as defined by IDC, encompasses search engines, categorisers and text analytics software. As we enlarge our understanding of where this market will head next, we have realised that anything that discerns meaning from unstructured information should be included in the market, so we’ve now added translation and globalisation software to this list.

These technologies, and the applications built on them constitute the next wave of development for the software markets. They will enable a fundamental shift in how people interact with computers. Technologies that manage data and transactions have reached their maturity, but they were never designed to handle the non-data side of the information world. The missing piece for enterprise software right now is the ability to interpret the actual meaning of words in text and eventually to even understand the meanings embedded in rich media such as pictures or videos. It’s an active area of research and development because at the moment searches are limited to metadata on rich media, not ion the images themselves.

What’s driving this interest in search and the growth of the search market? Certainly, most computer savvy people have become accustomed to using a web search engine as a portal to get at information. So the utility of search is a given. As is usually the case, demand comes from different drivers in any software market.

The application increases revenue

Good examples of this are in online retail solutions from the likes of Endeca, FAST, Autonomy, or Mercado. Just adding Web search to your site can increase sales and also improve customer support. Coveo, Google, Microsoft, and IBM have all come out with products that

The application decreases costs

Customer support is a fertile area for the use of search to connect people to the answers they need. Inquira, Recommind, Endeca, Knova, SPSS or SAS as well as many general search platforms provide this functionality. Another area that is growing quickly is in pharmaceutical research where quick access to information can save millions in just a few days. Temis is well known in this area. Xerox Grenoble is developing some very interesting medical applications of text analytics as well. But most of the search and text analytics vendors have clients in this industry.

Risk mitigation or fear

Compliance drove many enterprise sales this year. With regulations such as the TRED Act. Sarbanes-Oxley and Basel II, large enterprises in particular are now obligated to understand what’s going on in their email systems, and in conversations between their employees and their clients. They also need to mine various kinds of customers’ messages in order to understand warranty claims or reasons for customer churn. Being able to derail problems or avoid recalls of products means big savings for manufacturers. Of course, financial services firms are particularly interested in detecting fraud and suspect transactions as well, and the evidence for these is often in emails and voice recordings. Autonomy, Attensity, Attenex, are active in this area. Text analytics is also heavily used for government intelligence applications. Inxight, now part of Business Objects, has a sizable government business. So does Mark Logic.

Unified access to both data and content is another area, an offshoot of both risk mitigation and decreasing costs. Most employees simpy don't have the time or the expertise to access sometimes hundreds of separate silos of information within an organization. We are seeing some interesting applications for unifying access to all the information in any format within an enterprise, and sometimes for selected Web sites and news feeds outside it. Connotate is a good example of this kind of example. So is Siderean, which can scrape a web site and extract the essential information building blocks, and Schemalogic, which normalises the information so that similar things are tagged with the same tags.

Understanding text is the key to all of these. That’s no easy thing. If you’re a Proctor & Gamble or a Hewlett Packard, you get millions of messages from customers and your call centres are often handling millions of calls every month. So you can either hire all of the English graduates in the world or you have to automate the detection process.

Another new area of development is work environments that combine search, collaborative technologies, workflow engines and perhaps some domain knowledge. All of these are combined into work environments that are to support a particular task or workflow. You don’t have to open and close an application, cut and paste from your email into your Word document, from the Word document into your PowerPoint slide. The work environment creates a new kind of interface, a layer on top of multiple applications. So, these new applications act as a point of integration among multiple software products. Autonomy's Liquid Office lets you create these yourself, but there are also examples of pre-configured targeted applications that do this. Inmagic has one aimed at editors, researchers and publishers. It combines the right information sources with the desktop environment to support research and editorial work. We’re starting to see applications that allow users to engage in some sort of a dialogue with the computer in a natural language that they understand, rather than some sort of command-based communication. Another good example is LexisNexis Interact, which is an application that allows legal firms to prospect for new clients. It combines information from LexisNexis with a search engine, email and workflow so that the marketing officer of the legal firm can just work comfortably within the email environment.

Why this flowering of solutions to a multiplicity of information access and analysis problems? Previously, there wasn't enough understanding in the marketplace of their value to spur demand for robust commercial applications. And, until recently, the computing power, wasn’t widely available. But now it is and you’re starting to see really elegant solutions to a number of problems such as question answering, detecting terrorist networks, expertise location, and of course, better search. As technology becomes more sophisticated we can expect to see more solutions like this cropping up. This is just one of the areas where major progress is being made and it is only the beginning

Susan Feldman is VP Content Technologies Group and specialises in research on search and discovery software and digital marketplace technologies and dynamics. The Content Technologies group tracks and analyzes software that manages, organises, maintains, archives, distributes and creates access to unstructured information in any format.