Toward Tiny and High-quality Facial Makeup with Data Amplify Learning
CoRR(2024)
Abstract
Contemporary makeup approaches primarily hinge on unpaired learning
paradigms, yet they grapple with the challenges of inaccurate supervision
(e.g., face misalignment) and sophisticated facial prompts (including face
parsing, and landmark detection). These challenges prohibit low-cost deployment
of facial makeup models, especially on mobile devices. To solve above problems,
we propose a brand-new learning paradigm, termed "Data Amplify Learning (DAL),"
alongside a compact makeup model named "TinyBeauty." The core idea of DAL lies
in employing a Diffusion-based Data Amplifier (DDA) to "amplify" limited images
for the model training, thereby enabling accurate pixel-to-pixel supervision
with merely a handful of annotations. Two pivotal innovations in DDA facilitate
the above training approach: (1) A Residual Diffusion Model (RDM) is designed
to generate high-fidelity detail and circumvent the detail vanishing problem in
the vanilla diffusion models; (2) A Fine-Grained Makeup Module (FGMM) is
proposed to achieve precise makeup control and combination while retaining face
identity. Coupled with DAL, TinyBeauty necessitates merely 80K parameters to
achieve a state-of-the-art performance without intricate face prompts.
Meanwhile, TinyBeauty achieves a remarkable inference speed of up to 460 fps on
the iPhone 13. Extensive experiments show that DAL can produce highly
competitive makeup models using only 5 image pairs.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined