Benchmarking Segmentation Models with Mask-Preserved Attribute Editing
CVPR 2024(2024)
摘要
When deploying segmentation models in practice, it is critical to evaluate
their behaviors in varied and complex scenes. Different from the previous
evaluation paradigms only in consideration of global attribute variations (e.g.
adverse weather), we investigate both local and global attribute variations for
robustness evaluation. To achieve this, we construct a mask-preserved attribute
editing pipeline to edit visual attributes of real images with precise control
of structural information. Therefore, the original segmentation labels can be
reused for the edited images. Using our pipeline, we construct a benchmark
covering both object and image attributes (e.g. color, material, pattern,
style). We evaluate a broad variety of semantic segmentation models, spanning
from conventional close-set models to recent open-vocabulary large models on
their robustness to different types of variations. We find that both local and
global attribute variations affect segmentation performances, and the
sensitivity of models diverges across different variation types. We argue that
local attributes have the same importance as global attributes, and should be
considered in the robustness evaluation of segmentation models. Code:
https://github.com/PRIS-CV/Pascal-EA.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要