Assessing data diversity and model fairness in machine learning (ML) requires access to sensitive demographic attributes, which are often unavailable due to privacy constraints. While several methods have been proposed to estimate these properties, the field lacks a unified and reproducible evaluation framework. To fill this gap, we introduce Proxy-based Assessment for Inclusion, Representation, and Equity (PAIRE), a standardized benchmark for evaluating Fairness and Diversity (FD) estimators that operate without individual-level sensitive attributes. Leveraging PAIRE, we evaluate state-of-the-art demographic estimators on binary classification (UCI Adult, 45k instances) and multiclass ranking (TREC Fair Ranking, 1.15M instances, 21 regions), measuring estimation accuracy and vulnerability to attribute inference. Advanced methods demonstrate superior diversity estimation, reducing estimation error by up to 81%. However, this performance can be inverted in fairness assessment, with naive counting-based methods achieving up to 41% lower error than advanced quantification-based estimators, highlighting that strong performance on direct prevalence estimation does not guarantee reliability for downstream fairness assessment. Finally, privacy attacks formalized with PAIRE highlight that aggregate demographic estimators can be exploited to infer individual sensitive attributes with high accuracy (F1macro>0.9). Overall, PAIRE establishes a challenging benchmark for attribute-unaware FD estimation, providing a holistic evaluation in sensitive applications.
In multi-criteria decision-making (MCDM) scenarios, the adaptive extraction of interpretable decision analysis processes from data is critical for applications such as recommendation systems, marketing, and medical diagnosis. Compromised is the adaptability of existing data-driven MCDM methods, as they usually predetermine a single problem type, a specific utility function, and prescribed interaction patterns of criteria, on the basis of which they develop the corresponding algorithm. To address this, we present a generalized data-driven MCDM (G-MCDM) model and a corresponding end-to-end algorithm, Differentiable Utility Network (DUN), which jointly handles sorting and ranking problems, diverse utility functions, and the interactions among any input criteria by learning optimal functional forms directly from data. Experimental results across 16 real-world datasets demonstrate that DUN surpasses MCDM benchmarks by an average of 9.21% on sorting tasks and reduces ranking errors by 38.1%, while achieving accuracy competitive with machine learning baselines. Notably, DUN liberates MCDM from expert-dependent assumptions by adaptively deriving optimal utility functions and interaction patterns from data—and even potentially uncovering previously unknown utility functions that domain knowledge fails to prescribe.
