Evaluation has a crucial role in Information Retrieval (IR) since it allows possible point of failures of an IR approach to be identi ed and addressed thus improving the predictive capability of such approach. Developing tools to support users when analyzing results and investigating strategies to improve IR system performance can help make the analysis easier and more eective. In this paper we discuss a Visual Analytics-based approach to support the user when deciding whether or not to perform re-ranking to improve the system eectiveness measured after a retrieval run. The proposed approach is based on eectiveness measures that exploit graded relevance judgements and provide both a principled and intuitive way to support the user. A prototype is described and exploited to discuss some case studies based on TREC data.

To Re-rank or to Re-query: Can Visual Analytics Solve This Dilemma?

DI BUCCIO, EMANUELE;DUSSIN, MARCO;FERRO, NICOLA;MASIERO, IVANO;
2011

Abstract

Evaluation has a crucial role in Information Retrieval (IR) since it allows possible point of failures of an IR approach to be identi ed and addressed thus improving the predictive capability of such approach. Developing tools to support users when analyzing results and investigating strategies to improve IR system performance can help make the analysis easier and more eective. In this paper we discuss a Visual Analytics-based approach to support the user when deciding whether or not to perform re-ranking to improve the system eectiveness measured after a retrieval run. The proposed approach is based on eectiveness measures that exploit graded relevance judgements and provide both a principled and intuitive way to support the user. A prototype is described and exploited to discuss some case studies based on TREC data.
Multilingual and Multimodal Information Access Evaluation. Proceedings of the Second International Conference of the Cross-Language Evaluation Forum (CLEF 2011)
9783642237072
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11577/2491278
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? ND
social impact