Publication type: Book part
Type of review: Editorial review
Title: Black box evaluation for operational information retrieval applications
Authors: Braschler, Martin
Imhof, Melanie
Rietberger, Stefan
DOI: 10.1007/978-3-642-54798-0_9
Published in: Bridging between information retrieval and databases
Page(s): 192
Pages to: 207
Issue Date: 2013
Series: Lecture Notes in Computer Science
Series volume: 8173
Publisher / Ed. Institution: Springer
Publisher / Ed. Institution: Heidelberg
ISBN: 978-3-642-54797-3
Language: English
Subjects: IE; INIT
Subject (DDC): 020: Library and information sciences
Abstract: The black box application evaluation methodology described in this tutorial is applicable to a broad range of operational information retrieval (IR) applications. Contrary to popular, traditional IR evaluation approaches that are limited to measure the IR system performance on a test collection, the black box evaluation methodology considers an IR application in its entirety: the underlying system, the corresponding document collection, and its configuration/application layer. A comprehensive set of quality criteria is used to estimate the user’s perception of the application. Scores are assigned as a weighted average of results from tests that evaluate individual aspects. The methodology was validated in a small evaluation campaign. An analysis of this campaign shows a correlation between the testers’ perception of the applications and the evaluation scores. Moreover, functional weaknesses of the tested IR applications can be identified and then systematically targeted.
Fulltext version: Published version
License (according to publishing contract): Licence according to publishing contract
Departement: School of Engineering
Organisational Unit: Institute of Applied Information Technology (InIT)
Appears in collections:Publikationen School of Engineering

Files in This Item:
There are no files associated with this item.

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.