An Approach to the Design and Evaluation of an Enterprise Search Application

The purpose of enterprise search is to enable users to effectively find the information they need to perform their tasks while at the same time requiring a minimal effort and costs to be sustained by the company in terms of inefficiencies. Many of the problems of enterprise search have been addressed in previous experiences. The notion of relevance, for example, can be different from that used in web search where there are usually many documents relevant to a query, and the ranking tends to favor the most popular ones. Instead the typical enterprise search query has few correct answers. Dealing with enterprise information content has also some benefits to be exploited: the content is produced for dissemination purpose, unlike web content which is usually written to attract people. Moreover, we can obtain more contextual information on the queries: since the users are inside a company, we can obtain detailed profile information (e.g. role, experience, skills, team) or very precise location inside the company's building. Nonetheless, more effective algorithms that leverage those kind of benefits have to be developed. One of the motivations for the work reported here is the lack of previous works that deal with the problem of evaluating an enterprise search application. The reason for this lack may depend on the difficulties in building a significant test collection: each company can be interested in dealing with very different types of topics and data format with possibly peculiar content formats (e.g. in the case of the study reported in this paper CRM events were also dealt with, and the company may only be able to allocate limited resources to this type of activity. In the present work we present a real case study of design, development and evaluation of an application of enterprise search inside a company, and draw attention to the issues that had to be overcome toward the completion of the task.