Please use this identifier to cite or link to this item:
Publication type: Conference paper
Type of review: Peer review (publication)
Title: How (not) to measure bias in face recognition networks
Authors: Glüge, Stefan
Amirian, Mohammadreza
Flumini, Dandolo
Stadelmann, Thilo
et. al: No
DOI: 10.21256/zhaw-20277
Proceedings: Artificial Neural Networks in Pattern Recognition
Editors of the parent work: Schilling, Frank-Peter
Stadelmann, Thilo
Conference details: 9th IAPR TC 3 Workshop on Artificial Neural Networks for Pattern Recognition (ANNPR'20), Winterthur, Switzerland, 2-4 September 2020
Issue Date: 2-Sep-2020
Series: Lecture Notes in Computer Science
Series volume: 12294
Publisher / Ed. Institution: Springer
Publisher / Ed. Institution: Cham
Language: English
Subjects: Deep learning; Convolutional neural network; Fairness
Subject (DDC): 006: Special computer methods
Abstract: Within the last years Face Recognition (FR) systems have achieved human-like (or better) performance, leading to extensive deployment in large-scale practical settings. Yet, especially for sensible domains such as FR we expect algorithms to work equally well for everyone, regardless of somebody's age, gender, skin colour and/or origin. In this paper, we investigate a methodology to quantify the amount of bias in a trained Convolutional Neural Network (CNN) model for FR that is not only intuitively appealing, but also has already been used in the literature to argue for certain debiasing methods. It works by measuring the "blindness" of the model towards certain face characteristics in the embeddings of faces based on internal cluster validation measures. We conduct experiments on three openly available FR models to determine their bias regarding race, gender and age, and validate the computed scores by comparing their predictions against the actual drop in face recognition performance for minority cases. Interestingly, we could not link a crisp clustering in the embedding space to a strong bias in recognition rates|it is rather the opposite. We therefore offer arguments for the reasons behind this observation and argue for the need of a less naive clustering approach to develop a working measure for bias in FR models.
Fulltext version: Accepted version
License (according to publishing contract): Licence according to publishing contract
Departement: Life Sciences and Facility Management
School of Engineering
Organisational Unit: Institute of Applied Information Technology (InIT)
Institute of Applied Mathematics and Physics (IAMP)
Institute of Computational Life Sciences (ICLS)
Published as part of the ZHAW project: Libra: A One-Tool Solution for MLD4 Compliance
Appears in collections:Publikationen Life Sciences und Facility Management
Publikationen School of Engineering

Files in This Item:
File Description SizeFormat 
2020_Gluege-etal_Bias-in-face-recognition-networks_ANNPR.pdfAccepted Version3.78 MBAdobe PDFThumbnail

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.