![]() Autoregressive models such as PixelRNN instead train a network that models the conditional distribution of every individual pixel given previous pixels (to the left and to the top).Variational Autoencoders (VAEs) allow us to formalize this problem in the framework of probabilistic graphical models where we are maximizing a lower bound on the log likelihood of the data.Every time the discriminator notices a difference between the two distributions the generator adjusts its parameters slightly to make it go away, until at the end (in theory) the generator exactly reproduces the true data distribution and the discriminator is guessing at random, unable to find a difference. Now, our model also describes a distribution p ^ θ ( x ) \hat(x) p ^ ( x ). In the example image below, the blue region shows the part of the image space that, with a high probability (over some threshold) contains real images, and black dots indicate our data points (each is one image in our dataset). Mathematically, we think about a dataset of examples x 1, …, x n x_1, \ldots, x_n x 1 , …, x n as samples from a true data distribution p ( x ) p(x) p ( x ). In both cases the samples from the generator start out noisy and chaotic, and over time converge to have more plausible image statistics: But before we get there below are two animations that show samples from a generative model to give you a visual sense for the training process. There are a few other approaches to matching these distributions which we will discuss briefly below. In the end, the generator network is outputting images that are indistinguishable from real images for the discriminator. These two networks are therefore locked in a battle: the discriminator is trying to distinguish real images from fake images and the generator is trying to create images that make the discriminator think they are real. But in addition to that-and here’s the trick-we can also backpropagate through both the discriminator and the generator to find how we should change the generator’s parameters to make its 200 samples slightly more confusing for the discriminator. For instance, we could feed the 200 generated images and 200 real images into the discriminator and train it as a standard classifier to distinguish between the two sources. Here we introduce a second discriminator network (usually a standard convolutional neural network) that tries to classify if an input image is real or generated. One clever approach around this problem is to follow the Generative Adversarial Network (GAN) approach. The question is: how should we adjust the network’s parameters to encourage it to produce slightly more believable samples in the future? Notice that we’re not in a simple supervised setting and don’t have any explicit desired targets for our 200 generated images we merely want them to look real. Suppose that we used a newly-initialized network to generate 200 images, each time starting with a different random code. The intuition behind this approach follows a famous quote from Richard Feynman: To train a generative model we first collect a large amount of data in some domain (e.g., think millions of images, sentences, or sounds, etc.) and then train a model to generate data like it. Generative models are one of the most promising approaches towards this goal. The only tricky part is to develop models and algorithms that can analyze and understand this treasure trove of data. This tremendous amount of information is out there and to a large extent easily accessible-either in the physical world of atoms or the digital world of bits. It’s easy to forget just how much you know about the world: you understand that it is made up of 3D environments, objects that move, collide, interact people who walk, talk, and think animals who graze, fly, run, or bark monitors that display information encoded in language about the weather, who won a basketball game, or what happened in 1970. One of our core aspirations at OpenAI is to develop algorithms and techniques that endow computers with an understanding of our world.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |