2
Differentiable Augmentation for Data-Efficient GAN Training
Akbobek Abilkaiyrkyzy
MSc. Machine Learning student
GAN overview
Problem statement
https://developers.google.com/machine-learning/gan/gan_structure
Challenges
Overfitting: discriminator D memorizes the data
Challenges
Limited data: labelling and collecting cost
GANs Heavily Deteriorate Given Limited Data
https://hanlab.mit.edu/projects/data-efficient-gans/slides.pdf
Solution
Data augmentation
Solutions
https://miro.medium.com/max/3692/1*Ukc49J8TzyxiOD30EqOWwQ.png
Regularization
GAN trainings are usually unstable.
Regularization
Different regularization techniques are used to penalize the sudden change in discriminator's output within a local region of input.
Evaluation metrics
Inception Score (IS)
involves using a pre-trained deep learning neural network model for image classification to classify the generated images
Evaluation metrics
Frechet Inception Distance (FID)
a metric for evaluating the quality of generated images and specifically developed to evaluate the performance of generative adversarial networks.
DiffAugment
Figure 2. “Overview of DiffAugment for updating D (left) and G (right). DiffAugment applies the augmentation T to both the real sample x and the generated output G(z). When we update G, gradients need to be back-propagated through T, which requires T to be differentiable w.r.t. the input.” Source of image & caption: Zhao et al (2020).
Experiments
Data Augmentations
Augmentation of real samples x:
Augmentation of real samples x + fake samples G(z):
"Reals only"
Motivates to augment both real and fake samples.
The generator is encouraged to match the augmented & distorted distribution, NOT the true distribution.
Research
"Reals & Fake"
Motivates to propagate the gradient to our generator.
The discriminator learns to perfectly classify between reals that are augmented and fakes that are augmented, but it fails to identify fakes that are not augmented.
What is next?
Analysis
Datasets
Augmentation types
Translation
ImageNet
CIFAR-10, CIFAR-100
Color+Translation+Cutout
Considerations
FFHQ, LSUN-Cat
Color+Translation+Cutout
Approach results
Method 2
Method 1
- Method 1: the same augmentation artifacts appears on the generated images.
- Method 2: the unbalanced optimization cripples training.
- Method 3: Augment reals + fakes for both G and D.
Method 3
Low-shot generation
https://hanlab.mit.edu/projects/data-efficient-gans/slides.pdf
Low-shot generation
DiffAugment's performance
Conclusion
Results
Conclusion
- DifAugment is effectively prevents model from overfitting using limited amount of data.
- The augmentation must be differentiable.
- Type of augmentation matters.
Future prospects
Broader Impact
https://hanlab.mit.edu/projects/data-efficient-gans/slides.pdf
References
1
Differentiable Augmentation for Data-Efficient GAN Training, Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, Song Han, NeurIPS 2020.
References
Data-Efficient GANs! https://towardsdatascience.com/data-efficient-gans-d10acd595361
2
3
Guide to Differentiable Augmentation for Data-Efficient GAN Training, https://analyticsindiamag.com/guide-to-differentiable-augmentation-for-data-efficient-gan-training/