Introduction to Generative Adversarial Networks and their Applications: What Happens When Two AI Play a Game?

By: Brandon Campanella
Advised by: Dr. Franz Kurfess
While at: California Polytechnic University San Luis Obispo

Abstract:

Generative Adversarial Networks (GAN) are a relatively new invention created by Ian Goodfellow in his seminal paper published in 2014 [1]. At its core a GAN is two neural networks, one a discriminator and the other a generator, that play a sort of game amongst each other where both learn and strengthen from the success and the failures of each other. In my blog post I hope to accomplish 3 main goals, acquaint my audience with the general structure and theory behind GANs, talk about what makes them unique such as their capabilities and weak points , and lastly talk about their applications. In the applications section I link to two Colab Notebooks along with an extended discussion on selected code snippets. Further I talk about the solutions that GANs can provide to knowledge graphs with a special emphasis on synthesis of new graphs and vertices.

Introduction to the World of Generative Models:

Over the past decade machine learning models have proliferated throughout our daily lives from personal assistants that utilize voice recognition, or cars that use computer vision to see the road and obstacles ahead of them[2,3]. However many of the models that touch our lives can be seen as classification models, taking a sound wave and identifying a word or evaluating an image and identifying if it contains an object. More precisely, a classification model is a model that when given a specific example of a distribution can output the likelihood amongst a set of classes. Put mathematically, a classification model tries to estimate P(y|x) where y is an instance from the set of class labels and x is an example from the distribution of one of the classes. Classification is just that, taking an unseen example from a distribution, and attempting to give it a correct label from a set of given classes.

Generative models on the other hand are less commonly seen in an average day to day interaction. Generative models attempt to create a new target from a given distribution of items, with the goal of creating a natural representation of an instance of the target distribution. Very roughly speaking a generative model is a model that attempts to extract an item from a distribution that has a high probability. Usually the goal is for the extracted item to be new and unseen, instead of a trivially outputting an instance from the training data. Put mathematically a generative model attempts to estimate joint distribution P(x, y) so a sample x can be drawn with a high probability of belonging to the chosen distribution.

Generative models could be described as being orthogonal to classification models, instead of simply being a passive critic, the model is being an active creator of the data. Published applications of generative models are wide ranging with the most studied field falling in the image domain, from the generation of life like images, to image super resolution, and text to image generation[19, 33]. Some other fields that generative models are actively being explored in are chemical molecule generation, music generation, and to create more robust and secure machine learning models [15][16].

TLDR: Discriminative models try to learn the boundary between classes while generative models try to lean the distribution for the class itself.

Currently generative modeling is being taken over by the Generative Adversarial Network architecture style, making new possibilities like conditional generation and style transfer possible. But generative modeling has been explored for years with different styles of models, a few of which are worth mentioning here as the spiritual precursors to GANs.

The most effective generative modeling technique for images prior to GANs was Auto Encoders (AE) and Variational Auto Encoders (VAE), which adds an element of stochasticity to an auto encoding network. An auto encoder, originally coined as nonlinear principal component analysis, is a feed forward network that attempts to refine an instance of a data set into its defining features in a lower dimensional space, and then attempts to recreate the original data instance from the compressed representation[11]. In 2013 a novel idea to add a level of stochasticity to the lower dimensional representation in autoencoders was used to create variational auto encoders. VAEs took image generative modeling to the next level, and allowed sampling and interpolation from the lower dimensional representation[12].The main difference between traditional AE and VAE is that “ instead of encoding an input as a single point, we encode it as a distribution over the latent space”[17].

AE/VAEs have good success at creating low dimensional low loss representations of instances of data. AE/VAEs can be a generative model by sampling random instances from the lower dimensional space and hoping that the uncompressed result was representative of the original distribution. Another technique was to interpolate between two instances of known valid lower dimensional representations.

The results for AE as a generator are underwhelming with the model having difficulty creating novel valid instances of images from random points in the compressed dimension[12]. VAE produced better generative results that AE but due the the theoretical complexity of VAEs, the resulting instance tends to be blurry and non-high resolution. Further the irregular latent space of auto encoders makes it tricky to perform techniques like style transfer and so GANs have preoccupied much more scientific literature in the past years[13][17].

Neural Networks Background:

Briefly I believe it’s important to touch upon deep neural networks (DNN) to give the reader a better intuition about the training and internal workings of GANs. A deep neural network is composed of multiple layers of linear multivariable equations with a nonlinear function applied between successive layers. Generally speaking a standard fully connected layer consists of weights and bias for each output dimension, which can be trained and tuned to meet functional requirements. A layer that takes two inputs and produces a single output would be f(x₁, x₂) = w₁*x₁ + w₂*x₂ + b, with a nonlinearity after it, like the rectified linear unit f(x) = max(0, x). Interestingly it can be shown that a sufficiently wide and deep NN can become a universal approximator meaning with the correct set of weights it can approximate any function or distribution with arbitrary closeness [18].

In order for DNN models to “learn” there must be an objective function usually in the form of a loss function that is trying to be minimized. Some NN loss functions have their roots in probabilistic theory, with a popular loss function being cross entropy. When an instance of data is run through a simple feed forward NN a loss is able to be calculated, and with that loss a gradient can be formulated. In order to minimize that loss the gradient is subtracted from each of the modifiable parameters to take a step in the lower loss direction. It is important to note that all layers and functions within a NN must be differentiable everywhere, and so functions that are not natively differentiable (like ReLU) are approximated. Many larger models can have millions of trainable parameters and so calculating each individually can be extremely expensive, with no hope of creating a closed form derivative equation. Thankfully there is a technique called back propagation that utilizes the principle of the chain rule to efficiently find each weights gradient and take a weighted step in that direction [14]. Remember chain rule states that d/dx [ f(g(x))] = f’(g(x)) * g’(x)

Here’s the (contrived) problem setup of a single layer NN:
Find min of f(x,y) = -(y * tanh(w₁x₁ + w₂x₂ + b)) ,for all (x,y) in the dataset. y is -1 or 1. Data x is fixed and so we modify the parameters w and b. To know where to modify, we take the gradient of w and b and then update via (w — alpha * w_grad) and (b — alpha * b_grad). Alpha is a tunable hyperparameter usually called the learning rate.

Introduction to GAN Architecture:

Now with a foundation laid down we can formally introduce architecture of a generative adversarial network. Informally the thought process behind a GAN that two models, one a classifier and the other a generator, learn from each other while training. In the image generation case, the classifier makes a binary decision if an item presented to it is in the image distribution or not, while the generator tries its best to create an image that will fool the discriminator. If they both learn appropriately the generator will be forced to create images very similar to the real distribution of the training images.

Mathematically this can be presented as a zero sum non-cooperative game, where one side only wins if the other looses. The formula below represents the objective function that the discriminator is trying to maximize while the generator is trying to minimize.

From here

In a fully trained ideal case these competing networks would converge to a Nash equilibrium, where any move by either party would lead to an increase in loss[1].

A visual representation of the the structure of a GAN would be this image provided by Google.

From here

Now in order to solidify the mental image you hopefully have, I will run through an analogy. In this example instead of a neural network it will be two individuals who act in the place of the generator and the discriminator. The generator (George) has never seen or drawn a flower before and a discriminator (Debby) is an amateur flower critic. George is trying to fool Debbie between the image of a flower he created and the image a professional artist creates. At first it George’s drawings would be a random mess, and the critic Debbie should be able to tell the difference between George’s drawings and the expert’s drawings relatively quickly. But Debbie also gives George feedback, perhaps stating something along the lines of ”A flower should have petals and a stem, maybe some thorns”, and eventually George will learn from Debbie’s feedback and create an expert level image. At this point in time George now creates examples comparable to Debbie’s decision skill. So now Debbie will take time learn the difference between the George’s new examples and the other expert artist’s examples. This continues until equilibrium is reached.

Here I will try and rid some of the hand wavy descriptions of how a the discriminator imparts knowledge on to the generator. The discriminator learns the same way any image classifier would, it directly calculates a loss and back propagates the information to its weights within its own model. However when the generator takes its turn to learn it must use the discriminator as part of a loss function and utilize the backpropagated pixel gradients from the discriminator to learn the gradient updates for its own weights.

When the generator is training, the gradient of the discriminator is taken over the input instead of over the parameters.

From our previous example let

L(y’,y) = -( y * y’ )

D(x) = tanh(w₁*x₁ + w₂*x₂ + b)

G(z) = (u₁*z₁ + u₂*z₂ + b₁ , u₃*z₁ + u₄*z₂ + b₂ )

When updating the generator we take gradients of D w.r.t. x₁ and x₂ instead of w₁, w₂ and b. We then pass d/dx D to the generator G using chain rule. So the total network when the generator is training is L(D(G(z)), y) where y is the true label that indicates this instance is generated distribution. To get the gradients of the generators weight we are trying to find

d/du L == d/du L(D(G(z)), y) == dL/dD * dD/dG * dG/du

Drawbacks and difficulties of GAN

Previously I talked about how the generator and the critic learn in tandem from each other, each taking turns and advancing from their mistakes. While training a GAN it is important that neither the critic or the generator overpower each other otherwise learning will become very slow or even come to a halt[1]. Worse yet the models could oscillate[8]. Training this network architecture essentially takes all the previous problems that a single classification model has and multiplies it. Gradients are more likely to explode or become very small.

Also important to realize is that there is no fundamental objective function to assess goodness of GAN’s generated example. If a loss value becomes lower that is not necessarily indicative that the model is training well, the loss values are just representative of the performance of the generator and the discriminator relative to each other, not relative to the output distributions similarity to the training distribution. To assess the goodness of the entire model it must be visually inspected, or judged by a pretrained outside classifier. Lastly since is very important that generator in the discriminator train balanced relative to each other, the architectures and hyperparameters must be chosen not just with respect to the problem but with respect to each other.

Code for a Deep Convolutional GAN

Colab Notebook — click here

Sneakers from Fashion-MNIST

At the link above there is a Google Colab notebook that allows a user to train and observe a simple convolutional generator and discriminator. It is written using the Keras API which is a wrapper layer over the Tensorflow library. The images that the model is training over is the Fashion-MNIST dataset that consists of 70,000 28x28 grayscale images of clothing items. To expedite the model’s leaning cycle, this network trains only over the 6,000 images of sneakers. Fashion-MNIST was chosen instead of the original hand written digits MNIST because digits MNIST has been criticized as having a too easy image distribution, and in order to show the representation power from a very simple GAN the more complex distribution of Fashion-MNIST was selected [28]. I should note that this notebook and the subsequent notebook in the later sections were inspired by [8, 29, 30, 31, 32] .

In the gist above are some of the most important functions that are included in the Colab script that I have chosen to highlight. The first function is the generator architecture, that takes as input a random normally distributed size 100 vector, then reshapes and up samples using the transposed convolution operation until the desired image size is achieved.

Next is the discriminator, it follows the standard convolutional neural network architecture but note that its output is a single dimensional vector. The discriminators output vector is a binary choice between the original dataset image distribution and the generated image distribution.

Lastly I have included the function that runs a single step of training to identify the how the two networks fit together in practice. In one step a batch of images are generated and another batch of images is sampled from the dataset. Next the discriminator is run separately over the two batches of data, and the losses are calculated via their loss functions (not shown here). Lastly the gradient is calculated and applied over each network.

For the rest of the code, including the data preprocessing, the loss functions, and the rest of the training harness please see the Collab notebook linked above. Below is an example of images created from this GAN after training for 100 epochs.

Sneakers created by the generator

Applications of GANs

Now I will give a wide ranging tour of some applications of GANs

Image Generation

Image generation is the most explored area of GAN research, being the original domain of application in the first paper that outlined the GAN architecture[1]. There are many GAN image use cases, from image super resolution, text to image, and style transfer to name just a few. Below I will sample just a few applications that I find exceptionally interesting.

The previous Colab notebook is an example of a very simple image generation task of low resolution 28x28 grayscale images. Recent advances in GAN architectures and conditional GANs make it possible to generate high definition, life like color images. A cutting edge example would be the StyleGAN2 architecture by researchers at Nvidia[10]. Below are images of fake people, none of them exist and are instead artificially generated random examples.

Stylegan 2, generation of human faces. From here

Another very illustrative example would be the work done in the paper “Image-to-Image Translation with Conditional Adversarial Networks” and their pix2pix architecture[33]. It provides an highly adaptable conditional architecture that accepts an image as the conditional input, and outputs a transformed image based on the input. Some use cases that they provide are automatically changing an images lighting from day to night, or transforming an aerial photograph into a street map. A particularly fun example from their paper is a model trained to turn automatically generated edges into photographs. The model is powerful enough to even generate photographs from poor hand drawn edges with impressive results.

Hand drawn images, to photographs. From here

Deepfakes

This paper would be lacking if it did not mention one of the most hotly discussed applications of GANs, currently used by both respected industry partners and nefarious actors. The application is the generation of Deepfake videos and images. The definition of deepfake is “ a deep learning based technique able to create fake videos by swapping the face of a person by the face of another person[25].” Deepfakes utilizing GANs among other architectures have been able to create incredibly life like manipulated videos that are hard for a human to detect and can easily fool a viewer that the video is unaltered. This tool has shown great positive potential in the cinematography industry to be used for applications like making an actor appear younger or to even make a deceased actor appear in a film[26]. It is for these reasons that companies like Disney have been utilizing their scientist to perform research on deepfakes, producing results like those found in Disney's 2020 paper that invented a technique to create high resolution (1024x1024) cinema quality deepfake videos[27].

On the flipside deepfakes have been commonly mentioned in popular media for their nefarious uses. Deepfakes have been used to put celebrity faces on sexually explicit videos, in addition to swapping out politicians faces to create fake news, hoaxes and to attempt to commit financial fraud[25]. Because of the potential harm, there is growing research literature on how to identify and detect modified deepfake content. Even in the attempt to find deepfake videos GANs are being utilized, although admittedly in this classification problem it is more common to see researchers use standard convolutional neural networks[25].

Audio Generation

In audio generation there have been some recent papers that present promising results towards life like music creation. Audio generation is a particularly difficult field because humans are highly sensitive to pitch and discontinuation of sounds, so even small abnormalities will be immediately apparent[20]. In 2019 researchers at Google AI presented a paper titled “GANSynth: Adversarial Neural Audio Synthesis” that generated audio in an artificial timbre at a given pitch[20]. This conditional GAN was given a musical note representing a pitch and was tasked with generating a certain audio timbre at that pitch, like a piano, or a guitar or a choir of people singing. Audio Samples from GANSynth, a music generating network, can be found here[22].

Image from GANSynth paper here

GAN and Knowledge Graphs

In this section we will sample the current state of GANs working with knowledge graphs going from general to more specific applications. Most intuitively is the task of knowledge graph generation.

Quickly we will go over the definition of a graph, and the definition of knowledge graph. Put mathematically “graph g = (Vg, Eg,Xg,Ag) is defined by a set of vertexes Vg (also referred to as nodes) and by a set of edges (or arcs) Eg connecting pairs of nodes[23].” The edges can be ordered (meaning unidirectional) or unordered, and is commonly represented by an adjacency matrix, which is |Vg| by|Vg| matrix with binary inputs where 1 implies there exists an edge between vertices and a 0 implies no edges. The definition of a knowledge graph (KG) is a bit more nebulous due to there being no universally accepted precise definition. An inclusive definition that most researchers could agree on would be that a knowledge graph is “ a graph of data intended to accumulate and convey knowledge of the real world, whose nodes represent entities of interest and whose edges represent relations between these entities[24]”.

One example would be the 2017 paper “GraphGAN: Graph Representation Learning with Generative Adversarial Nets” that attempts to fit a generator G(v|vᶜ) into creating a new vertex v that is likely to be connected with a given vertex vᶜ [7]. The discriminator D(v|vᶜ) “tries to distinguish well-connected vertex pairs from ill-connected ones, and calculates the probability of whether an edge exists between v and vᶜ.[7]” An example use case for this given a social media network of friends, where each person is a vertex and an edge represents friendship, generate several new examples of friends for person v.

Another useful example would be to use GANs to generate negative samples for knowledge graph embedding models to train on as proposed in “KBGAN: Adversarial Learning for Knowledge Graph Embeddings”[5]. The problem that this paper tries to solve is to create more believable negative graph examples as compared to the common practice of randomly replacing a head or tail entity.

Framework for molecule synthetization, from Here

The last example is application is to use GANs and knowledge graphs to synthesize new chemicals with desirable properties. The 2018 paper “MolGAN: An implicit generative model for small molecular graphs” created a framework where a generator attempts to output a graph that represents a molecule with a high likelihood of coming from a trained on distribution of molecules and adheres to desirable chemical properties[4]. In this application the graphs’ nodes and edges correspond to atoms and bond types.

The MolGAN framework includes two critics on the generated example; one is a traditional discriminator that attempts to distinguish if the generated chemical could reasonably be a valid chemical in the training distribution, and the second is a reward function that penalizes the generator if the chemical items do not have desired molecular properties. While there are other architectures of GANs that use multiple discriminators and multiple differing loss functions, this papers work is unique in that the reward network critic uses an external software to identify metrics of chemicals deemed good[4].

Exploring GAN with Knowledge Graph Discriminator

DCGAN with KG Discriminator — click here

Above I provide a Google Colab notebook to test what happens when the discriminator portion of a GAN is a knowledge graph neural network. The rest of the notebook is essentially the same as the previously linked code, except for the functions shown below and that it uses the original MNIST dataset to give the network as best a shot at working as possible. Specifically this network only trains on the digit 7 to speed up computation time.

Above we see two functions. The first is a function that builds an adjacency matrix to turn an image into an unordered graph represented in the form of a matrix. The second builds a shallow network that infers images from the graph matrix. The theory behind this discriminator is out of the scope of this article but an in depth explanation and the inspiration behind this code can be found at this medium article [31].

From the tests in the Colab notebook we can see that the discriminator as a stand alone network is able to decipher between the 10 classes of images well, with a validation test set accuracy of greater than 88%. However when embedded into a GAN architecture we can see that the results it produces are quite poor, with the generated images not clearly representing handwritten digits. Below is an example output of the GAN after 100 epochs.

Generator output from GAN with KG Discriminator

The cause of this is unclear but a few possible theories could be that the representational power of the KG discriminator was not strong enough to be used in a GAN, or perhaps the learning rates and other associated hyper parameters need to be differently optimized to adjust to this new architecture. My personal hypothesis is that the adjacency matrix multiplication portion of the discriminator, meaning the knowledge graph representation, is not good for gradient flow during backpropagation when the generator is trying to train.

Conclusion

In this article we reviewed the major mathematical concepts of deep neural networks, and applied that knowledge to the mathematical theory behind genitive adversarial Networks. We explored GANs at a high level, and wrote Colab script to train a simple generative network. Furthermore we looked at many different application of GANs with a special emphasis on applications related to knowledge graphs. Last we examined a Colab notebook that used knowledge graph style image discriminator within a generative adversarial network.

Links for Further Reading

•http://cs231n.stanford.edu/2019/

•https://www.deeplearningbook.org/

•https://www.tensorflow.org/tutorials/generative/dcgan

References

1. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative Adversarial Networks. arXiv [stat.ML]. 2014. http://arxiv.org/abs/1406.2661
2. Lemley, Joe, et al. “Deep Learning for Consumer Devices and Services: Pushing the Limits for Machine Learning, Artificial Intelligence, and Computer Vision.” IEEE Xplore, IEEE Consumer Electronics Magazine, Apr. 2017, ieeexplore.ieee.org/abstract/document/7879402.
3. Rao, Qing, and Jelena Frtunikj. “Deep Learning for Self-Driving Cars.” Proceedings of the 1st International Workshop on Software Engineering for AI in Autonomous Systems, 2018, doi:10.1145/3194085.3194087.
4. De Cao N, Kipf T. MolGAN: An implicit generative model for small molecular graphs. arXiv [stat.ML]. 2018. http://arxiv.org/abs/1805.11973
5. Cai L, Wang WY. KBGAN: Adversarial learning for knowledge graph embeddings. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Stroudsburg, PA, USA: Association for Computational Linguistics; 2018.
6. Wang Q, Ji Y, Hao Y, Cao J. GRL: Knowledge graph completion with GAN-based reinforcement learning. Knowledge-based systems. 2020;209(106421):106421.
7. Wang H, Wang J, Wang J, Zhao M, Zhang W, Zhang F, Xie X, Guo M. GraphGAN: Graph representation learning with generative adversarial nets. arXiv [cs.LG]. 2017. http://arxiv.org/abs/1711.08267
8. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X. Improved Techniques for Training GANs. arXiv [cs.LG]. 2016. http://arxiv.org/abs/1606.03498
9. Pasini M. Style transfer with GANs on HD images — towards data science. Towards Data Science. 2019 Jul 23 [accessed 2021 Jan 28]. https://towardsdatascience.com/style-transfer-with-gans-on-hd-images-88e8efcf3716
10. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T. Analyzing and improving the image quality of StyleGAN. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2020.
11. Kramer MA. Nonlinear principal component analysis using autoassociative neural networks. AIChE journal. American Institute of Chemical Engineers. 1991;37(2):233–243.
12. Kingma DP, Welling M. Auto-Encoding Variational Bayes. arXiv [stat.ML]. 2013. http://arxiv.org/abs/1312.6114
13. Rezende DJ, Mohamed S. Variational inference with normalizing flows. arXiv [stat.ML]. 2015. http://arxiv.org/abs/1505.05770
14. Ma T, Avati A, Katanforoosh K, Ng A. CS229 Lecture Notes. Stanford.edu. [accessed 2021 Feb 5]. http://cs229.stanford.edu/notes2020spring/cs229-notes-deep_learning.pdf
15. DH. Ackley, GE. Hinton, et al. “Applications of Generative Adversarial Networks (GANs): An Updated Review.” Archives of Computational Methods in Engineering, Springer Netherlands, 1 Jan. 1985, link.springer.com/article/10.1007/s11831–019–09388-y.
16. Rocca, Joseph. “Understanding Variational Autoencoders (VAEs).” Medium, Towards Data Science, 7 July 2020, towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73.
17. Xiao, Chaowei, et al. “Generating Adversarial Examples with Adversarial Networks.” ArXiv.org, 14 Feb. 2019, arxiv.org/abs/1801.02610.
18. Kratsios, Anastasis, and Eugene Bilokopytov. “Non-Euclidean Universal Approximation.” ArXiv.org, 7 Nov. 2020, arxiv.org/abs/2006.02341.
19. Karras, Tero, et al. “A Style-Based Generator Architecture for Generative Adversarial Networks.” ArXiv.org, 29 Mar. 2019, arxiv.org/abs/1812.04948.
20. “GANSynth: Making Music with GANs.” Magenta, 25 Feb. 2019, magenta.tensorflow.org/gansynth.
21. Reed, Scott, et al. “Generative Adversarial Text to Image Synthesis.” ArXiv.org, 5 June 2016, arxiv.org/abs/1605.05396.
22. Cysmith. “Cysmith/Neural-Style-Tf.” GitHub, github.com/cysmith/neural-style-tf.
23. Bacciu, Davide, et al. “A Gentle Introduction to Deep Learning for Graphs.” Neural Networks, vol. 129, 2020, pp. 203–221., doi:10.1016/j.neunet.2020.06.006.
24. Hogan, Aidan, et al. “Knowledge Graphs.” ArXiv.org, 24 Jan. 2021, arxiv.org/abs/2003.02320.
25. Tolosana, Ruben, et al. “DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection.” ArXiv.org, 18 June 2020, arxiv.org/abs/2001.00179.
26. Linder, Courtney. “Disney Is Getting Into the Deepfake Game.” Popular Mechanics, Popular Mechanics, 1 July 2020, www.popularmechanics.com/technology/security/a33013507/disney-deepfake-face-swap/.
27. Naruniec, J, et al. “High-Resolution Neural Face Swapping for Visual Effects.” Disney Research Studios, Nov. 2020, studios.disneyresearch.com/2020/06/29/high-resolution-neural-face-swapping-for-visual-effects/.
28. Zalandoresearch. “Zalandoresearch/Fashion-Mnist.” GitHub, github.com/zalandoresearch/fashion-mnist.
29. Brownlee, Jason. “How to Develop a Conditional GAN (CGAN) From Scratch.” Machine Learning Mastery, 1 Sept. 2020, machinelearningmastery.com/how-to-develop-a-conditional-generative-adversarial-network-from-scratch/.
30. “Deep Convolutional Generative Adversarial Network : TensorFlow Core.” TensorFlow, www.tensorflow.org/tutorials/generative/dcgan.
31. Knyazev, Boris. “Tutorial on Graph Neural Networks for Computer Vision and Beyond (Part 1).” Medium, Medium, 13 May 2020, medium.com/@BorisAKnyazev/tutorial-on-graph-neural-networks-for-computer-vision-and-beyond-part-1–3d9fada3b80d.
32. Radford, Alec, et al. “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks.” ArXiv.org, 7 Jan. 2016, arxiv.org/abs/1511.06434v2.
33. Isola, Phillip, et al. “Image-to-Image Translation with Conditional Adversarial Networks.” ArXiv.org, 26 Nov. 2018, arxiv.org/abs/1611.07004.