List of 14 Generative Adversarial Networks (GANs) Applications

For the past few days, we’ve been trying to create pixel arts of our own and we thought of using Generative Adversarial Networks (GANs) to help with the process. Since the last time we worked on a project involving GANs was 3 years ago, we decided to do a quick Google search on the topic for refreshing our memory.

We were in complete awe to see all the new applications that have stemmed off of the traditional GANs (first introduced by the GANfather, Ian Goodfellow). Therefore, we thought of compiling all the new applications of GANs so that you and us both can leverage them for future use cases.

Here’s a relatively large list of 14 different Generative Adversarial Networks (GANs) applications:

---

1. Neural Photo Editor using Introspective Adversarial Networks – GitHub

According to the GitHub repo, Neural Photo Editor is a simple interface for editing natural photos with generative neural networks.

Some commands of the tool are as follows:

• You can paint the image by picking a color and painting on the image, or paint in the latent space canvas (the red and blue tiles below the image).
• The long horizontal slider controls the magnitude of the latent brush, and the smaller horizontal slider controls the size of both the latent and the main image brush.
• You can select different entries from the subset of the celebA validation set (included in this repository as an .npz) by typing in a number from 0-999 in the bottom left box and hitting “infer.”
• Use the reset button to return to the ground truth image.
• Press “Update” to update the ground-truth image and corresponding reconstruction with the current image. Use “Infer” to return to an original ground truth image from the dataset.
• Use the sample button to generate a random latent vector and corresponding image.
• Use the scroll wheel to lighten or darken an image patch (equivalent to using a pure white or pure black paintbrush). Note that this automatically returns you to sample mode, and may require hitting “infer” rather than “reset” to get back to photo editing.

---

2. Anime character generation using IllustrationGAN – Github

According to the GitHub repo, Illustration GAN is a simple, clean TensorFlow implementation of Generative Adversarial Networks with a focus on modeling illustrations.

The model is based on DCGANs, but with a few important differences:

1. No strided convolutions. The generator uses bilinear upsampling to upscale a feature blob by a factor of 2, followed by a stride-1 convolution layer. The discriminator uses a stride-1 convolution followed by 2×2 max pooling.
2. Minibatch discrimination. See Improved Techniques for Training GANs for more details.
3. More fully connected layers in both the generator and discriminator. In DCGANs, both networks have only one fully connected layer.
4. A novel regularization term applied to the generator network.

---

3. Text2Image (text to image) generation using StackGAN – GitHub, Paper

Stacked Generative Adversarial Networks (StackGAN) is a variant of GANs that can generate 256×256 photo-realistic images conditioned on text descriptions.

StackGAN uses a two-stage approach where the Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs and generates high-resolution images with photo-realistic details.

---

4. 3D Image Generation using 3D-GAN – GitHub, Paper

3D Generative Adversarial Network (3D-GAN) is a type of GANs which generates 3D objects from a probabilistic space by leveraging recent advances in volumetric convolutional networks and generative adversarial nets.

Here’s a video detailing the application:

---

5. Image Editing using Invertible Conditional GAN – GitHub, Paper

Invertible Conditional GANs (IcGANs) uses a combination of an encoder with conditional GANs to re-generate real images with deterministic complex modifications.

---

6. Face Aging using Age-cGAN – Paper

In this paper, conditional generative adversarial network (cGAN) has been used for identity-preserving face aging

The contributions of the paper are as follows:

• The first GAN to generate high-quality synthetic images within required age categories.
• Use of a novel latent vector optimization approach that allows Age-cGAN to reconstruct an input face image preserving the original person’s identity.

---

7. Image-to-Image Translation using Conditional Adversarial Networks (pix2pix) – GitHub, Paper

According to the authors, the image-to-image translation is performed by using conditional adversarial networks as a general-purpose solution to image-to-image translation problems.

These networks not only learn the mapping from an input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations.

---

8. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (CycleGAN) – GitHub, Paper

Just like pix2pix, CycleGAN also uses pixel-level domain transfer and has multiple use cases.

Some applications of CycleGAN are as follows:

• Painting to photo
• Collection style transfer
• Object transfiguration
• Season transfer
• Photo enhancement

---

9. Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks (MGAN) – GitHub

Using MGAN, a texture from an image can be applied to another image. For example, the texture from Van Gogh’s “Olive Trees with the Alpilles in the Background” applied to other images results in the following:

---

10. Pixel-Level Domain Transfer (PixelDTGAN) – GitHub

Using PixelDTGAN, one can generate images of objects using an input image similar to pix2pix or CycleGAN.

In the above example, the images on the left are the input, and the images on the right are generated clothes using PixelDTGAN.

---

---

11. Deep Photo Style Transfer – GitHub, Paper

Deep Photo Style Transfer is one of the many interesting applications of GANs since the results are very mesmerizing.

According to the authors, the paper introduces a deep-learning approach to photographic style transfer that handles a large variety of image content while faithfully transferring the reference style. Their approach builds upon the recent work on painterly transfer that separates style from the content of an image by considering different layers of a neural network. However, as is, this approach is not suitable for photorealistic style transfer. Even when both the input and reference images are photographs, the output still exhibits distortions reminiscent of a painting.

---

12. Image Inpainting (hole filling) using Context Encoders – GitHub

Image Inpainting using Context Encoders allows computers to fill the missing parts of an image using GANs.

---

13. Image super-resolution through Deep Learning (srez) – GitHub

According to the GitHub readme, this project uses deep learning to upscale 16×16 images by a 4x factor. The resulting 64×64 images display sharp features that are plausible based on the dataset that was used to train the neural net.

The below example shows what this network can do. From left to right, the first column is the 16×16 input image, the second one is what you would get from a standard bicubic interpolation, the third is the output generated by the neural net, and on the right is the ground truth.

The application is based on Deep Convolutional Generative Adversarial Network (DCGAN).

---

14. High-resolution image generation using GANs – GitHub

According to the authors, ‘We describe a new training methodology for generative adversarial networks. The key idea is to grow both the generator and discriminator progressively: starting from a low resolution, we add new layers that model increasingly fine details as training progresses. This both speeds the training up and greatly stabilizes it, allowing us to produce images of unprecedented quality, e.g., CelebA images at 1024².‘

This is what GANs were originally intended for and looks like we’ve made a lot of progress in the quality of the application.

---

In Conclusion

The above 14 different types of GANs applications were quite intriguing and there are certainly more applications out there. If you have recommendations to add to the list, please feel free to comment them down.

---

Do you want to learn Python, Data Science, and Machine Learning while getting certified? Here are some best selling Datacamp courses that we recommend you enroll in:

1. Introduction to Python (Free Course) - 1,000,000+ students already enrolled!
2. Introduction to Data Science  in Python- 400,000+ students already enrolled!
3. Introduction to TensorFlow for Deep Learning with Python - 90,000+ students already enrolled!
4. Data Science and Machine Learning Bootcamp with R - 70,000+ students already enrolled!