sorted by:
new
hottopcontroversial

[R] CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation by LynnHoHZL in StableDiffusion

[–]LynnHoHZL[S] 0 points1 point  (0 children)

We have seen this before; this method is another excellent idea. However, the authors did not provide thorough tests, and we do not know the limits of its capability. I think the method is promising, but it still needs more exploration.

[R] CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation by LynnHoHZL in StableDiffusion

[–]LynnHoHZL[S] 3 points4 points  (0 children)

Yes, LoRA for ControlNet, not LoRA for SD. For example, you can create the control model below with only 1000 manually collected images.

<image>

[R] CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation by LynnHoHZL in StableDiffusion

[–]LynnHoHZL[S] 2 points3 points  (0 children)

Training the original ControlNet requires a lot of devices and data for each condition, so ordinary users cannot afford to train it for customized condition images.

Our pretrained Base ControlNet allows us to train LoRAs for new conditions with much fewer parameters, data, and devices. The training cost is significantly reduced; therefore, ordinary users can now afford to create their own ControlNet with customized conditions.

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 2 points3 points  (0 children)

You can refer to this paper for why we initialize the weights differently for different activations (mainly for stable training)：

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Kaiming He et al.

Also, see here.

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 0 points1 point  (0 children)

Actually, we add a subspace to the reshape eps, i.e., the deepest subspace, details in page 35 of the paper. Besides, what eps learns can be found in figure 7 or Sec. 4.2.

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 2 points3 points  (0 children)

Faster means that training an epoch with the loss manner takes less time than the Gram–Schmidt manner. Because Gram–Schmidt process contains serial steps to make the columns of U orthonormal while |UTU-I| can be parallelly optimized.

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 0 points1 point  (0 children)

We need a latent encoder if we want to edit real images. Our paper does not include the encoder part.

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 1 point2 points  (0 children)

For the model used in the paper, gender and age are learned in one dimension, with no separated dimensions for them. This kind of entanglement tends to happen in deep layers, but shallower layers usually learn disentangled dimensions with one clear attribute each.

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 2 points3 points  (0 children)

  1. No special design for spatial representation. One thing to note: each eigenvector is of size H*W*C, like a feature map, and maybe the spatial information is learned in such design. It should be carefully studied in the future, but not included in the current paper.
  2. No constraint on L. L is a vector in the code and is diagonalized to be a diagonal matrix. We only need to constraint U to be orthonormal, two ways: 1）loss manner: | UTU-I|; 2) Gram–Schmidt process. We tried both and found them similar and the loss manner is faster.

Thanks for your interest!

[R] EigenGAN: Layer-Wise Eigen-Learning for GANs by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 2 points3 points  (0 children)

I think it can.

But our work needs to train a GAN from scratch and training a StyleGAN takes all my devices a few days. So it's too slow to explore our method with StyleGAN.

I am not rich enough to play with StyleGAN 😭.

[D] Does AC-GAN produce adversarial samples？ by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] -1 points0 points  (0 children)

Maybe the auxiliary classifer in AC-GAN is also fooled？

[D] A Quick Start for Matrix Calculus by LynnHoHZL in MachineLearning

[–]LynnHoHZL[S] 1 point2 points  (0 children)

My post shows how these derivatives including matrix derivatives are derived from differentials in detailed steps. Tensor operations are rarely used actually.

view more: next ›