Certifying and Removing Disparate Impact
ACM Knowledge Discovery and Data Mining, pp.259-268, (2015)
What does it mean for an algorithm to be biased? In U.S. law, unintentional bias is encoded via disparate impact, which occurs when a selection process has widely different outcomes for different groups, even as it appears to be neutral. This legal determination hinges on a definition of a protected class (ethnicity, gender) and an explic...更多
下载 PDF 全文
- Given data set D = (X, Y, C), with protected attribute X, remaining attributes Y, and binary class to be predicted C (e.g., “will hire”), the authors will say that D has disparate impact if
- 1 Note that under this definition disparate impact is determined based on the given data set and decision outcomes.
- A data set is (1/2 − β/8)-predictable if and only if it admits disparate impact, where β is the fraction of elements in the minority class (X = 0) that are selected (C = 1).
- Duke Power Co. , the US Supreme Court ruled a business hiring decision illegal if it resulted in disparate impact by race even if the decision was not explicitly determined based on race
- The Duke Power Co. was forced to stop using intelligence test scores and high school diplomas, qualifications largely correlated with race, to make hiring decisions
- We introduce and address two such problems with the goals of quantifying and removing disparate impact
- We show that any decision exhibiting disparate impact can be converted into one where the protected attribute leaks, i.e. can be predicted with low balanced error rate
- We summarize our main idea with the following intuition: If Bob cannot predict X given the other attributes of D, A is fair with respect to Bob on D
- The authors run a classifier that optimizes BER on the given data set, attempting to predict the protected attributes X from the remaining attributes Y.
- Once Bob’s certification procedure has made a determination of disparate impact on D, Alice might request a repaired version Dof D, where any attributes in D that could be used to predict X have been changed so that Dwould be certified as -fair.
- Let Dλ = (X, Y , C) be the partially repaired data set for some value of λ ∈ [0, 1] as described above.
- The utility of a classifier gλ : Y → C with respect to some partially repaired data set Dλ is γ = 1 − BER(gλ(y), c).
- The authors will consider the certification algorithm and repair algorithm’s fairness/utility tradeoff experimentally on three data sets.
- The resulting BER is compared to DI(g) where g : Y → C, i.e., the disparate impact value as measured when some classifier attempts to predict the class given the non-protected attributes.
- Disparate impact (DI) for all data sets is measured with respect to the predicted outcomes on the test set as differentiated by protected attribute.
- Figure 5 shows the fairness and accuracy results for both combinatorial and geometric partial repairs for values of λ ∈ [0, 1] at increments of 0.1 using all three classifiers described above.
- On the Adult Income data set the repairs based on Naıve Bayes have better accuracy at high values of fairness than the repairs based on Logistic Regression.
- On the German and Adult data sets the results show that for any fairness value a partially repaired data set at that value can be chosen and a classifier applied to achieve accuracy that is better than competing methods.
- A natural avenue for future work is to investigate generalizations of the repair procedures for datasets with different attribute types, such as categorical data, vector-valued attributes, etc.
- 1.1 Results
We have four main contributions. We first introduce these problems to the computer science community and develop its theoretical underpinnings. The study of the EEOC’s 80% rule as a specific class of loss function does not appear to have received much attention in the literature. We link this measure of disparate impact to the balanced error rate (BER). We show that any decision exhibiting disparate impact can be converted into one where the protected attribute leaks, i.e. can be predicted with low BER.
Second, this theoretical result gives us a procedure for certifying the impossibility of disparate impact on a data set. This procedure involves a particular regression algorithm which minimizes BER. We connect BER to disparate impact in a variety of settings (point and interval estimates, and distributions). We discuss these two contributions in Sections 3 and 4.
- A natural avenue for future work is to investigate generalizations of our repair procedures for datasets with different attribute types, such as categorical data, vector-valued attributes, etc. This research was funded in part by the NSF under grant BIGDATA-1251049
- S. Barocas and A. D. Selbst. Big data’s disparate impact. Technical report, available at SSRN: http://ssrn.com/abstract=2477899, 2014.
- T. Calders, F. Kamiran, and M. Pechenizkiy. Building classifiers with independency constraints. In ICDM Workshop Domain Driven Data Mining, pages 13–18, 2009.
- T. Calders and S. Verwer. Three naive bayes approaches for discrimination-free classification. Data Mining journal; special issue with selected papers from ECML/PKDD, 2010.
- C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In Proc. of Innovations in Theoretical Computer Science, 2012.
- R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. Liblinear: A library for large linear classification. J. of Machine Learning Research, 9:1871–1874, 2008.
- H. Hodson. No one in control: The algorithms that run our lives. New Scientist, Feb. 04, 2015.
- T. Joachims. A support vector method for multivariate performance measures. In Proc. of Intl. Conf. on Machine Learning, pages 377–384. ACM, 2005.
- F. Kamiran and T. Calders. Classifying without discriminating. In Proc. of the IEEE International Conference on Computer, Control and Communication, 2009.
- T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma. Fairness-aware classifier with prejudice remover regularizer. Machine Learning and Knowledge Discovery in Databases, pages 35–50, 2012.
- T. Kamishima, S. Akaho, and J. Sakuma. Fairness aware learning through regularization approach. In Proc of. Intl. Conf. on Data Mining, pages 643–650, 2011.
- B. T. Luong, S. Ruggieri, and F. Turini. k-nn as an implementation of situation testing for discrimination discovery and prevention. In Proc. of Intl. Conf. on Knowledge Discovery and Data Mining, KDD ’11, pages 502–510, 2011.
- A. Menon, H. Narasimhan, S. Agarwal, and S. Chawla. On the statistical consistency of algorithms for binary classification under class imbalance. In Proc. 30th. ICM, pages 603–611, 2013.
- W. Miao. Did the results of promotion exams have a disparate impact on minorities? Using statistical evidence in Ricci v. DeStefano. J. of Stat. Ed., 19(1), 2011.
- J. Pearl. Understanding simpson’s paradox. The American Statistician, 2014.
- D. Pedreschi, S. Ruggieri, and F. Turini. Integrating induction and deduction for finding evidence of discrimination. In Proc. of Intl. Conf. on Artificial Intelligence and Law, ICAIL ’09, pages 157–166, 2009.
- D. Pedreschi, S. Ruggieri, and F. Turini. A study of top-k measures for discrimination discovery. In Proc. of Symposium on Applied Computing, SAC ’12, pages 126–131, 2012.
- J. L. Peresie. Toward a coherent test for disparate impact discrimination. Indiana Law Journal, 84(3):Article 1, 2009.
- J. Podesta, P. Pritzker, E. J. Moniz, J. Holdren, and J. Zients. Big data: seizing opportunities, preserving values. Executive Office of the President, May 2014.
- A. Romei and S. Ruggieri. A multidisciplinary survey on discrimination analysis. The Knowledge Engineering Review, pages 1–57, April 3 2013.
- Supreme Court of the United States. Griggs v. Duke Power Co. 401 U.S. 424, March 8, 1971.
- Supreme Court of the United States. Watson v. Fort Worth Bank & Trust. 487 U.S. 977, 995, 1988.
- Supreme Court of the United States. Ricci v. DeStefano. 557 U.S. 557, 174, 2009.
- Texas House of Representatives. House bill 588. 75th Legislature, 1997.
- The Leadership Conference. Civil rights principles for the era of big data. http://www.civilrights.org/press/2014/civil-rights-principles-big-data.html, Feb.27, 2014.
- The U.S. EEOC. Uniform guidelines on employee selection procedures, March 2, 1979.
- R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In Proc. of Intl. Conf. on Machine Learning, pages 325–333, 2013.
- M.-J. Zhao, N. Edakunni, A. Pocock, and G. Brown. Beyond Fano’s inequality: bounds on the optimal F-score, BER, and cost-sensitive risk and their implications. J. of Machine Learning Research, 14(1):1033–1090, 2013.