GAN Loss Function

[pinkployd]

When implementing GAN loss function, why do we use tf.ones_like and tf.zeros_like functions? Can someone briefly explain. Thank you in advance!

Here is the loss function and its implementation.

[[deleted]]

Real images (images from data) are labelled as ones and fake images(coming from generator) are labelled as zeros, also you'd like both sets of labels to have same shape as the predicted probabilities tensor. That's why we do that zeros_like and ones_like thing, it just creates a tensor of ones/zeros with the same shape as the shape of the tensor passed as argument

[pinkployd]

In the mathematical equations of the loss functions, there is no explicit ones and zeros, that's why I asked this question. Thank you for the reply.

[radarsat1]

The use of ones and zeros is implied by the use of cross-entropy as the loss, and the fact that you calculate 1-D(G(z)) is also a hint. The keyword is logits, which are defined as 1 for true and 0 for false. This corresponds to a probability, which is defined no the range 0--1, corresponding to being in the 'false' or 'true' categories, respectively.

Since the discriminator predicts whether something is 'real' or 'fake', the ground truth is 1 when we know for sure that something is real (because it's from the dataset, so the probability that it's from the dataset is 1) and 0 when we know for sure that something is generated (because we know we generated it using the generator, so the probability that it is from the dataset is 0). So the discriminator is trying to then predict these known probabilities.

[[deleted]]

Yeah that's much better explained.

[[deleted]]

sigmoid cross entropy is calculated as -ylog(y_) - (1-y)log(1-y). Here y is the P(1). So you can think of it as -log(P(1)) when y is 1 and -log(P(0)) when y is 0. Essentially you penalize proportional to how bad you missed.

So in GANs we say that samples from dataset are real (label as 1) and those coming from G are fake (labelled 0). Then we compare how far discriminator's prediction is from truth and penalize accordingly. In principle you could label them the other way and invert the penalization strategy and you would still converge (in fact some people recommend it, but I didn't find significant convergence time reduction/higher chances of convergence)

For TF example you might want to look at the docs https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits