Publication type: | Book part |
Type of review: | Editorial review |
Title: | Black box evaluation for operational information retrieval applications |
Authors: | Braschler, Martin Imhof, Melanie Rietberger, Stefan |
DOI: | 10.1007/978-3-642-54798-0_9 |
Published in: | Bridging between information retrieval and databases |
Page(s): | 192 |
Pages to: | 207 |
Issue Date: | 2013 |
Series: | Lecture Notes in Computer Science |
Series volume: | 8173 |
Publisher / Ed. Institution: | Springer |
Publisher / Ed. Institution: | Heidelberg |
ISBN: | 978-3-642-54797-3 |
Language: | English |
Subjects: | IE; INIT |
Subject (DDC): | 020: Library and information sciences |
Abstract: | The black box application evaluation methodology described in this tutorial is applicable to a broad range of operational information retrieval (IR) applications. Contrary to popular, traditional IR evaluation approaches that are limited to measure the IR system performance on a test collection, the black box evaluation methodology considers an IR application in its entirety: the underlying system, the corresponding document collection, and its configuration/application layer. A comprehensive set of quality criteria is used to estimate the user’s perception of the application. Scores are assigned as a weighted average of results from tests that evaluate individual aspects. The methodology was validated in a small evaluation campaign. An analysis of this campaign shows a correlation between the testers’ perception of the applications and the evaluation scores. Moreover, functional weaknesses of the tested IR applications can be identified and then systematically targeted. |
URI: | https://digitalcollection.zhaw.ch/handle/11475/4499 |
Fulltext version: | Published version |
License (according to publishing contract): | Licence according to publishing contract |
Departement: | School of Engineering |
Organisational Unit: | Institute of Computer Science (InIT) |
Appears in collections: | Publikationen School of Engineering |
Files in This Item:
There are no files associated with this item.
Show full item record
Braschler, M., Imhof, M., & Rietberger, S. (2013). Black box evaluation for operational information retrieval applications. In Bridging between information retrieval and databases (pp. 192–207). Springer. https://doi.org/10.1007/978-3-642-54798-0_9
Braschler, M., Imhof, M. and Rietberger, S. (2013) ‘Black box evaluation for operational information retrieval applications’, in Bridging between information retrieval and databases. Heidelberg: Springer, pp. 192–207. Available at: https://doi.org/10.1007/978-3-642-54798-0_9.
M. Braschler, M. Imhof, and S. Rietberger, “Black box evaluation for operational information retrieval applications,” in Bridging between information retrieval and databases, Heidelberg: Springer, 2013, pp. 192–207. doi: 10.1007/978-3-642-54798-0_9.
BRASCHLER, Martin, Melanie IMHOF und Stefan RIETBERGER, 2013. Black box evaluation for operational information retrieval applications. In: Bridging between information retrieval and databases. Heidelberg: Springer. S. 192–207. ISBN 978-3-642-54797-3
Braschler, Martin, Melanie Imhof, and Stefan Rietberger. 2013. “Black Box Evaluation for Operational Information Retrieval Applications.” In Bridging between Information Retrieval and Databases, 192–207. Heidelberg: Springer. https://doi.org/10.1007/978-3-642-54798-0_9.
Braschler, Martin, et al. “Black Box Evaluation for Operational Information Retrieval Applications.” Bridging between Information Retrieval and Databases, Springer, 2013, pp. 192–207, https://doi.org/10.1007/978-3-642-54798-0_9.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.