Inspired by the success and convenience of search on the internet, there has been quite a few attempts to extend the search paradigm to tame the unruly clutter of documents in an enterprise. The basic premise is that knowledge workers can be much more effective in performing their jobs, if all the pertinent information for accomplishing the job is at their fingertips. The search on the internet however does not necessarily provide one with the desired information – it attempts to do its best to provide links to documents that might contain the desired information. Internet search deals with documents which are homogeneous in the sense that they are all uniquely addressable web pages. Furthermore, the search relies on document tags (keywords) and thus, the relevance of the results of a search query depends on how appropriately the tags characterize the document contents.
The basic search has evolved and become more sophisticated, particularly, in the areas of interpreting search requests, indexing documents, and presenting results as interesting visualizations. The question, however, still remains whether these developments are enough to make the knowledge workers more effective at their jobs. No doubt, search can save time in getting to the relevant documents, provided a right search query is posed. The question of knowledge worker effectiveness becomes lot more significant when search enters the realm of business analytics.
The Search Process
The search process begins with user articulating a search request and ends with users receiving results delivered by the search mechanism. An auxiliary process digests new documents and thus making them available for search.
User Articulation of Search Request
“Has anybody worked with Smith & Company? I would like to know their experience with them”, or “What’s the latest return policy for electronic products?” are typical examples of information that a knowledge worker might be seeking in the context of their job. A search system must allow the users to express their requests in a most natural way. This, of course, requires the search system to understand user’s intent and behind the scenes translate that intent into a formal query that can be executed by the underlying search engine. We are far from achieving this goal but major strides have been made in making it easier for users to express their requests. Techniques vary from auto-suggestions to use of natural language processing technologies.
Presentation of Search Results
Have you experienced frustration with search when you are not able to get to what you are looking for in spite of repeated attempts? After trying one request after another and scanning through many documents, you are still feeling lost. What you might be looking for may not at all be there but you don’t have any way of asserting that. At other times, you get lucky and get what you want instantly. The question is whether a smarter presentation of search results can improve user search experience. The simplest thing to do is to present search results in the order of their relevance. The advanced techniques involve inclusion of snippets from the documents and other semantic content that may help users avoid a full scan of documents. In some cases, presentation may include data visualizations.
Digesting Documents for Search
A document is as good as lost if it is not classified properly. Business documents can be quite large. So, for business users, it is not just a matter of getting to the document as a whole, but they would like to get to a specific part of the document which contains the relevant information. This means, for a document to be amenable to intelligent search, not only it must be indexed properly but also its parts. Alternatively, the parts of a document may be marked up during digestion phase and may have to be extracted during the execution of search query. Full-text search and other text processing techniques have made it possible to reach to the depths of any document.
The Lure of ‘Search’
The lure of ‘Search” is mesmerizing as it appears very simple at the user interaction level and there seems to be no learning curve on behalf of the end user. In contrast to that other mechanisms, such as BI, to find information from documents / data appear very laborious and unresponsive. Thus, there is a huge temptation to extend the search paradigm to the domain of BI and Business Analytics. Considering the skills and time it takes to produce a new BI report, search appears miles ahead of the game. The so-called self-service BI is targeting to eliminate the time lag between the expression of information need and its delivery and analytical search plays a pivotal role in the realization of this goal. In traditional BI, there are two roles are involved – a BI expert, and the end user of BI gleaned information. Search based self-service certainly makes the day-to-day job of a BI expert much easier but the claims made include elimination of the BI expert role in the information discovery process. These claims can only be justified if our end users of BI information are at least partially as savvy as the BI experts. In other words, in the disguise of end-user empowerment and responsiveness, we inadvertently shift responsibility to the end user to become more data savvy.
Why Search is not enough?
Search is not a panacea for discovering insights from data. It is not a substitute for business analytics, by any stretch of the imagination.
Business Analytics is all about supporting data-driven decision making which includes not only searching data for what it explicitly represents (descriptive analytics), but also what it implicitly encodes as the knowledge that can be used to predict future (predictive analytics), and also suggest preventive or coping measures (prescriptive analytics) to deal with the predicted future. Furthermore, to assist users in making effective and timely decisions, a business analytics system must reduce the cognitive dissonance between the decision-makers and the business analytics system (cognitive analytics).
Search can be an important tool in a decision making process but decision-making primarily driven from analytical search will be as inflexible and unresponsive from decision-making perspective as the traditional BI is for generating descriptive reports.
Search when it is well integrated in a decision-making process can be very useful but this integration is not a straight-forward plug and play activity. A deeper integration of the search with the decision-making process is a complex endeavor and has a profound impact on all aspects of the search process itself.
Author: Dr.Satyendra Rana