MetaTOC stay on top of your field, easily

Evaluation of context‐aware recommendation systems for information re‐finding

, ,

Journal of the American Society for Information Science and Technology

Published online on

Abstract

In this article we evaluate context‐aware recommendation systems for information re‐finding by knowledge workers. We identify 4 criteria that are relevant for evaluating the quality of knowledge worker support: context relevance, document relevance, prediction of user action, and diversity of the suggestions. We compare 3 different context‐aware recommendation methods for information re‐finding in a writing support task. The first method uses contextual prefiltering and content‐based recommendation (CBR), the second uses the just‐in‐time information retrieval paradigm (JITIR), and the third is a novel network‐based recommendation system where context is part of the recommendation model (CIA). We found that each method has its own strengths: CBR is strong at context relevance, JITIR captures document relevance well, and CIA achieves the best result at predicting user action. Weaknesses include that CBR depends on a manual source to determine the context and in JITIR the context query can fail when the textual content is not sufficient. We conclude that to truly support a knowledge worker, all 4 evaluation criteria are important. In light of that conclusion, we argue that the network‐based approach the CIA offers has the highest robustness and flexibility for context‐aware information recommendation.