To Re-rank or to Re-query: Can Visual Analytics Solve This Dilemma?

Evaluation has a crucial role in Information Retrieval (IR) since it allows possible point of failures of an IR approach to be identi ed and addressed thus improving the predictive capability of such approach. Developing tools to support users when analyzing results and investigating strategies to improve IR system performance can help make the analysis easier and more eective. In this paper we discuss a Visual Analytics-based approach to support the user when deciding whether or not to perform re-ranking to improve the system eectiveness measured after a retrieval run. The proposed approach is based on eectiveness measures that exploit graded relevance judgements and provide both a principled and intuitive way to support the user. A prototype is described and exploited to discuss some case studies based on TREC data.