Dose-volume-based evaluation of convolutional neural network-based auto-segmentation of thoracic organs at risk

•Dice score and Hausdorff distance do not correlate with dose-volume-based results.•Auto-contours close to the tumor or in entry/exit beams should be checked.•Heart and esophagus must be checked for locally advanced non-small cell lung cancer.•Bronchi must be checked for peripheral early-stage non-small cell lung cancer.•Every treatment plan still passed the clinical goals for the manual organs at risk.