Masked Modeling for Self-supervised Representation Learning on Vision and Beyond
CoRR(2023)
摘要
As the deep learning revolution marches on, self-supervised learning has
garnered increasing attention in recent years thanks to its remarkable
representation learning ability and the low dependence on labeled data. Among
these varied self-supervised techniques, masked modeling has emerged as a
distinctive approach that involves predicting parts of the original data that
are proportionally masked during training. This paradigm enables deep models to
learn robust representations and has demonstrated exceptional performance in
the context of computer vision, natural language processing, and other
modalities. In this survey, we present a comprehensive review of the masked
modeling framework and its methodology. We elaborate on the details of
techniques within masked modeling, including diverse masking strategies,
recovering targets, network architectures, and more. Then, we systematically
investigate its wide-ranging applications across domains. Furthermore, we also
explore the commonalities and differences between masked modeling methods in
different fields. Toward the end of this paper, we conclude by discussing the
limitations of current techniques and point out several potential avenues for
advancing masked modeling research. A paper list project with this survey is
available at .
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要