Response to the Comment on Evaluating Cardiovascular Health Disparities Using Estimated Race / Ethnicity : A Validation Study

To the Editor: As users of the Bayesian Improved Surname and Geocoding Algorigthm (BISG) method we eagerly read Bykov et al. We were surprised that the paper never evaluates the method currently recommended in the literature. The authors compare results using known race/ethnicity results (unusually, collapsed to white vs. all other) to 2 alternative uses of the BISG-estimated probabilities. The first approach dichotomized the BISG probabilities. Elliott et al and McCaffrey and Elliott both warned against this approach, in part because classification leads to errors like those shown in Bykov et al. The second method examined uses random draws from the BISG probabilities to impute race. This method fails, not because the BISG probabilities are incorrect, but because it is an improper imputation method, as the outcomes were not used to inform imputed values. McCaffrey and Elliott suggest using the BISG probabilities directly as regressors. The authors considered this method only as a “sensitivity” analysis. Unfortunately they only present P-values for this method without point estimates and intervals, making meaningful comparisons difficult. There is a question of whether this study design would be capable of a meaningful evaluation of BISG even if complete results and more appropriate methods had been presented. In general the power of tests using BISG will be lower and the width of confidence intervals will be wider than analyses using known race. Further complicating the power problem is the focus on an interaction term likely to have larger standard errors than main effects. A data set of this size is unlikely to permit valid comparisons of the methods involved. In summary, we think it would be unfortunate if readers unfamiliar with this methodology interpreted Bykov et al as a broadly applicable criticism of a potentially quite useful method. John L. Adams, PhD Jason Kramer, MS Kaiser Permanente Center for Effectiveness & Safety Research, Pasadena, CA