Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation

Karakozis Yannis
Karakozis Yannis
Hata Kenji
Hata Kenji

CVPR, pp. 8916-8925, 2019.

Cited by: 8|Views51
EI
Weibo:
We provide a thorough analysis of a wide range of techniques

Abstract:

Computer vision models learn to perform a task by capturing relevant statistics from training data. It has been shown that models learn spurious age, gender, and race correlations when trained for seemingly unrelated tasks like activity recognition or image captioning. Various mitigation techniques have been presented to prevent models ...More

Code:

Data:

0
Introduction
  • Computer vision models learn to perform a task by capturing relevant statistics from training data.
  • These statistics range from low-level information about color or composition to contextual or societal cues.
  • Some real-world activities are more commonly performed by women and others by men
  • This realworld gender distribution skew becomes part of the data that trains models to recognize or reason about these activities.1.
Highlights
  • Computer vision models learn to perform a task by capturing relevant statistics from training data
  • As computer vision systems are deployed at scale and in a variety of settings, especially where the initial training data and the final end task may be mismatched, it becomes increasingly important to both identify and develop strategies for manipulating the information learned by the model
  • We provide a thorough comparison of existing methods for bias mitigation, including domain adversarial training [46, 41, 1], Reducing Bias Amplification [52], and domain conditional training similar to [40]
  • We evaluate on the CelebA [32] benchmark for attribute recognition in the presence of gender bias (Sec. 5)
  • We provide a benchmark and a thorough analysis of bias mitigation techniques in visual recognition models
  • Attributes that have skew greater than 80% always benefit from the DOMAININDEPENDENT model
  • What happens if the domain is non-discrete? What happens if the imbalanced domain distribution is not known at training time – for example, if the researchers failed to identify the undesired correlation with gender? What happens in downstream tasks where these models may be used to make prediction decisions? We leave these and many other questions to future work
Results
  • The authors would expect that a model trained on CIFAR-10S would take advantage of the available color cues and perform even better than 93.0%, ideally approaching 95.1% accuracy of a model trained on all color images.
  • One interesting change is that OVERSAMPLING yields 78.6 ± 0.4%, significantly lower than the baseline of 79.4%, so the authors investigate further.
  • Attributes that have skew greater than 80% always benefit from the DOMAININDEPENDENT model.
  • The authors note that the OVERSAMPLING model in this case achieves high mAP of 77.6% and bias amplification of -0.061, outperforming the other techniques
Conclusion
  • The authors provide a benchmark and a thorough analysis of bias mitigation techniques in visual recognition models.
  • The authors draw several important algorithmic conclusions, while acknowledging that this work does not attempt to tackle many of the underlying ethical fairness questions.
  • What happens if the domain is non-discrete?
  • What happens if the imbalanced domain distribution is not known at training time – for example, if the researchers failed to identify the undesired correlation with gender?
  • What happens in downstream tasks where these models may be used to make prediction decisions?
  • The authors leave these and many other questions to future work.
  • Thank you to Arvind Narayanan and to members of Princeton’s Fairness in AI reading group for great discussions
Summary
  • Introduction:

    Computer vision models learn to perform a task by capturing relevant statistics from training data.
  • These statistics range from low-level information about color or composition to contextual or societal cues.
  • Some real-world activities are more commonly performed by women and others by men
  • This realworld gender distribution skew becomes part of the data that trains models to recognize or reason about these activities.1.
  • Results:

    The authors would expect that a model trained on CIFAR-10S would take advantage of the available color cues and perform even better than 93.0%, ideally approaching 95.1% accuracy of a model trained on all color images.
  • One interesting change is that OVERSAMPLING yields 78.6 ± 0.4%, significantly lower than the baseline of 79.4%, so the authors investigate further.
  • Attributes that have skew greater than 80% always benefit from the DOMAININDEPENDENT model.
  • The authors note that the OVERSAMPLING model in this case achieves high mAP of 77.6% and bias amplification of -0.061, outperforming the other techniques
  • Conclusion:

    The authors provide a benchmark and a thorough analysis of bias mitigation techniques in visual recognition models.
  • The authors draw several important algorithmic conclusions, while acknowledging that this work does not attempt to tackle many of the underlying ethical fairness questions.
  • What happens if the domain is non-discrete?
  • What happens if the imbalanced domain distribution is not known at training time – for example, if the researchers failed to identify the undesired correlation with gender?
  • What happens in downstream tasks where these models may be used to make prediction decisions?
  • The authors leave these and many other questions to future work.
  • Thank you to Arvind Narayanan and to members of Princeton’s Fairness in AI reading group for great discussions
Tables
  • Table1: Performance comparison of algorithms on CIFAR-10S. All architectures are based on ResNet-18 [<a class="ref-link" id="c20" href="#r20">20</a>]. We investigate multiple bias mitigation strategies, and demonstrate that a domain-independent classifier outperforms all baselines on this benchmark
  • Table2: On CIFAR-10S, we consider other transformations instead of the grayscale domain: (1) cropping the center of the image, (2,3) reducing the image resolution [<a class="ref-link" id="c44" href="#r44">44</a>], followed by upsampling or (4) replacing with 32x32 ImageNet images of the same class [<a class="ref-link" id="c10" href="#r10">10</a>]. We use the inference of Eqn 6 for DOMDISCR and Eqn 9 for DOMINDEP, and report mean per-class per-domain accuracy (in %). Our conclusions from Sec. 4 hold across all domain shifts
  • Table3: Attribute classification accuracy evaluated using mAP (in %, ↑) weighted to ensure an equal distribution of men and women appearing with each attribute, and Bias Amplification (↓). Evaluation is on the CelebA test set, across 34 attributes that have sufficient validation data; details in Sec. 5.2
Download tables as Excel
Related work
  • Mitigating Spurious Correlation. Recent work on the effects of human bias on machine learning models investigates two challenging problems: identifying and quantifying bias in datasets, and mitigating its harmful effects. In relation to the former, [5, 31] study the effect of classimbalance on learning, while [52] reveal the surprising phenomenon of bias amplification. Additionally, recent works have shown that ML models possess bias towards legally protected classes [29, 6, 4, 7, 33, 8]. Our work complements these by presenting a dataset that allows us to isolate and control bias precisely, alleviating the usual difficulties of quantifying bias.

    On the bias mitigation side, early works investigate techniques for simpler linear models [23, 50]. Our constructed dataset allows us to isolate bias while not simplifying our architecture. More recently, works have begun looking at more sophisticated models. For example, [52] propose an inference update scheme to match a target distribution, which can remove bias. [40] introduce InclusiveFaceNet for improved attribute detection across gender and race subgroups; our discriminative architecture is inspired by this work. Conversely, [12] propose a scheme for decoupling classifiers, which we use to create our domain independent architecture. The last relevant approach to bias mitigation for us is adversarial mitigation [1, 51, 13, 16]. Our work uses our novel dataset to explicitly highlight the drawbacks, and offers a comparison between these mitigation strategies that would be impossible without access to a bias-controlled environment. Fairness Criterion. Pinning down an exact and generally applicable notion of fairness is an inherently difficult and important task. Various fairness criteria have been introduced and analyzed, including demographic parity [24, 51], predictive parity [15], error-rate balance [19], equality-ofodds and equality-of-opportunity [19], and fairness-throughunawareness [35] to try to quantify bias. Recent work has shown that such criteria must be selected carefully; [19] prove minimizing error disparity across populations, even under relaxed assumptions, is equivalent to randomized predictions; [19] introduce and explain the limitations of an ‘oblivious’ discrimination criterion through a non-identifiability result; [35] demonstrate that ignoring protected attributes is ineffective due to redundant encoding; [11] show that demographic parity does not ensure fairness. We define our tasks such that test accuracy directly represents model bias. Surveying Evaluations. We are inspired by previous work which aggregate ideas, methods and findings to provide a unify survey of a subfield of computer vision [22, 38, 43, 21]. For example, [45] surveys relative dataset biases present in computer vision datasets, including selection bias (datasets favoring certain types of images), capture bias (photographers take similar photos), category bias (inconsistent or imprecise category definitions), and negative set bias (unrepresentative or unbalanced negative instances). We continue this line of work for bias mitigation methods for modern visual recognition systems, introducing a benchmark for evaluation which isolates bias, and showing that our analysis generalizes to other, more complex, biased datasets.
Funding
  • This work is partially supported by the National Science Foundation under Grant No 1763642, by Google Cloud, and by the Princeton SEAS Yang Family Innovation award
Reference
  • Mohsan Alvi, Andrew Zisserman, and Christoffer Nellaker. Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings. In ECCV, 2018. 1, 2, 4, 7, 8
    Google ScholarLocate open access versionFindings
  • Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. In ECCV, 2018. 1
    Google ScholarLocate open access versionFindings
  • Steffen Bickel, Michael Brckner, and Tobias Scheffer. Discriminative learning under covariate shift. Journal of Machine Learning Research, Sep 2009. 3
    Google ScholarLocate open access versionFindings
  • Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In NeurIPS, 2016. 1, 2
    Google ScholarLocate open access versionFindings
  • Mateusz Buda, Atsuto Maki, and Maciej A. Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. arXiv preprint arXiv:1710.05381, Oct. 2017. 2
    Findings
  • Joy Buolamwini and Timnit Gebru. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 2018. 1, 2
    Google ScholarLocate open access versionFindings
  • Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186, 2011, 2
    Google ScholarLocate open access versionFindings
  • Elliot Creager, David Madras, Jorn-Henrik Jacobsen, Marissa A Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In ICML, 2019. 2
    Google ScholarLocate open access versionFindings
  • Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In CVPR, 2017
    Google ScholarLocate open access versionFindings
  • Luke Nicholas Darlow, Elliot J. Crowley, Antreas Antoniou, and Amos J. Storkey. CINIC-10 is not imagenet or CIFAR-CoRR, abs/1810.03505, 2018. 7
    Findings
  • Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness Through Awareness. In Innovations in Theoretical Computer Science Conference, 2012. 1, 2, 4
    Google ScholarLocate open access versionFindings
  • Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. Decoupled classifiers for group-fair and efficient machine learning. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 2018. 2
    Google ScholarLocate open access versionFindings
  • Harrison Edwards and Amos Storkey. Censoring Representations with an Adversary. In ICLR, 2016. 2
    Google ScholarLocate open access versionFindings
  • Charles Elkan. The foundations of cost-sensitive learning. In In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pages 973–978, 2001. 3
    Google ScholarLocate open access versionFindings
  • Pratik Gajane and Mykola Pechenizkiy. On formalizing fairness in prediction with machine learning. arXiv preprint arXiv:1710.03184, 2017. 2
    Findings
  • Yaroslav Ganin and Victor Lempitsky. Unsupervised Domain Adaptation by Backpropagation. In ICML, 2015. 1, 2
    Google ScholarLocate open access versionFindings
  • Aditya Grover, Kristy Choi, Rui Shu, and Stefano Ermon. Fair generative modeling via weak supervision. arXiv preprint arXiv:1910.12008, 2019. 1
    Findings
  • Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, and Stefano Ermon. Bias correction of learned generative models using likelihood-free importance weighting. In NeurIPS, 2019. 1
    Google ScholarLocate open access versionFindings
  • Moritz Hardt, Eric Price, Nati Srebro, et al. Equality of opportunity in supervised learning. In NeurIPS, 2016. 2, 4
    Google ScholarLocate open access versionFindings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In CVPR, 2016. 3, 4, 8
    Google ScholarLocate open access versionFindings
  • Derek Hoiem, Yodsawalai Chodpathumwan, and Qieyun Dai. Diagnosing error in object detectors. In ECCV, 2012. 2
    Google ScholarLocate open access versionFindings
  • Minyoung Huh, Pulkit Agrawal, and Alexei A. Efros. What makes imagenet good for transfer learning? arXiv preprint arXiv:1608.08614, 2017. 2
    Findings
  • Aditya Khosla, Tinghui Zhou, Tomasz Malisiewicz, Alexei A Efros, and Antonio Torralba. Undoing the Damage of Dataset Bias. In ECCV, 2012. 2
    Google ScholarLocate open access versionFindings
  • Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Scholkopf. Avoiding discrimination through causal reasoning. In NeurIPS, 2017. 2
    Google ScholarLocate open access versionFindings
  • Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In CVPR, 2019. 1
    Google ScholarFindings
  • Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014. 8
    Findings
  • Alex Krizhevsky and Geoffrey Hinton. Learning Multiple Layers of Features from Tiny Images. 2009. 2
    Google ScholarFindings
  • Klas Leino, Emily Black, Matt Fredrikson, Shayak Sen, and Anupam Datta. Feature-wise bias amplification. ICLR, 2018. 1
    Google ScholarLocate open access versionFindings
  • Sam Levin. A beauty contest was judged by AI and the robots didnt like dark skin, September 2016. 2
    Google ScholarFindings
  • Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In CVPR, 2019. 1
    Google ScholarLocate open access versionFindings
  • Xu-Ying Liu, Jianxin Wu, and Zhi-Hua Zhou. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, 39(2), 2009. 2
    Google ScholarLocate open access versionFindings
  • Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015. 2, 6, 7, 8
    Google ScholarLocate open access versionFindings
  • David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In ICML, 2018. 2
    Google ScholarLocate open access versionFindings
  • Safiya Umoja Noble. Algorithms of Oppression: How search engines reinforce racism. NYU Press, February 2018. 1
    Google ScholarFindings
  • Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In KDD, 2008. 2
    Google ScholarFindings
  • Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In CVPR, 2019. 1
    Google ScholarLocate open access versionFindings
  • Amelie Royer and Christoph H Lampert. Classifier adaptation at prediction time. In CVPR, 2015. 5
    Google ScholarLocate open access versionFindings
  • Olga Russakovsky, Jia Deng, Zhiheng Huang, Alexander C. Berg, and Li Fei-Fei. Detecting avocados to zucchinis: What have we done, and where are we going? In ICCV, 2013. 2
    Google ScholarLocate open access versionFindings
  • Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211– 252, 2015. 7, 8
    Google ScholarLocate open access versionFindings
  • Hee Jung Ryu, Hartwig Adam, and Margaret Mitchell. Inclusivefacenet: Improving face attribute detection with race and gender diversity. In Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML), 2018. 1, 2
    Google ScholarLocate open access versionFindings
  • Hee Jung Ryu, Margaret Mitchell, and Hartwig Adam. Improving Smiling Detection with Race and Gender Diversity. arXiv preprint arXiv:1712.00193, 2017. 1, 2, 7
    Findings
  • Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure. Neural computation, 14(1):21–41, 2002. 5
    Google ScholarLocate open access versionFindings
  • Gunnar A. Sigurdsson, Olga Russakovsky, and Abhinav Gupta. What actions are needed for understanding human actions in videos? In ICCV. IEEE, Oct 2017. 2
    Google ScholarLocate open access versionFindings
  • Jong-Chyi Su and Subhransu Maji. Adapting models to signal degradation using distillation. In BMVC, 2017. 7
    Google ScholarLocate open access versionFindings
  • Antonio Torralba and Alexei A. Efros. Unbiased Look at Dataset Bias. In CVPR. IEEE, 2011. 2
    Google ScholarLocate open access versionFindings
  • Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. Simultaneous deep transfer across domains and tasks. In CVPR, 2015. 2, 4, 7, 8
    Google ScholarLocate open access versionFindings
  • Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In CVPR, 2019. 1
    Google ScholarLocate open access versionFindings
  • Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In CVPR, 2019. 1
    Google ScholarLocate open access versionFindings
  • Gary M Weiss, Kate McCarthy, and Bibi Zabar. Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs? 3
    Google ScholarFindings
  • Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning Fair Representations. In ICML, 2013. 1, 2
    Google ScholarLocate open access versionFindings
  • Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 2018. 1, 2, 4, 5, 7
    Google ScholarLocate open access versionFindings
  • Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. In EMNLP, 2017. 1, 2, 3, 4, 5, 6, 7, 8
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments