Assessment of Renal Cell Carcinoma by Texture Analysis in Clinical Practice: A Six-Site, Six-Platform Analysis of Reliability

BACKGROUND. Multiple commercial and open-source software applications are available for texture analysis. Nonstandard techniques can cause undesirable variability that impedes result reproducibility and limits clinical utility. OBJECTIVE. The purpose of this study is to measure agreement of texture metrics extracted by six software packages. METHODS. This retrospective study included 40 renal cell carcinomas with contrast-enhanced CT from The Cancer Genome Atlas and Imaging Archive. Images were analyzed by seven readers at six sites. Each reader used one of six software packages to extract commonly studied texture features. Inter- and intrareader agreement for segmentation was assessed with intraclass correlation coefficients (ICCs). First-order (available in six packages) and second-order (available in three packages) texture features were compared between software pairs using Pearson correlation. RESULTS. Inter- and intrareader agreement was excellent (ICC, 0.93-1). First-order feature correlations were strong (r >= 0.8, p < .001) between 75% (21/28) of software pairs for mean intensity and SD, 48% (10/21) for entropy, 29% (8/28) for skewness, and 25% (7/28) for kurtosis. Of 15 second-order features, only cooccurrence matrix correlation, gray-level nonuniformity, and run-length nonuniformity showed strong correlation between software packages (r = 0.90-1, p < .001). CONCLUSION. Variability in first- and second-order texture features was common across software configurations and produced inconsistent results. Standardized algorithms and reporting methods are needed before texture data can be reliably used for clinical applications. CLINICAL IMPACT. It is important to be aware of variability related to texture software processing and configuration when reporting and comparing outputs.