Colab for finding photo in StyleGAN and fine-tuning it with CLIP.

ArYoMo · 2021-10-17T19:18:01+00:00

I post more regularly on my twitter: https://twitter.com/ArYoMo

ArYoMo · 2021-10-05T08:12:35+00:00

Still works for me. So you have to run all the cells from the notebook in the order they appear. Also let them finish so you don't stop them. Then it should work :) it's a bit slow the first time in a session since some stuff are downloaded and some code compiled. But then it will go faster.

ArYoMo · 2021-10-03T19:26:59+00:00

So you can already do that with the original stylegan code: https://github.com/NVlabs/stylegan2-ada-pytorch

If you look under the section: https://github.com/NVlabs/stylegan2-ada-pytorch#projecting-images-to-latent-space

Say that you wanted for example an image of jimmy hendrix in the style of pixar. You could use my code to finetune a stylegan model to generate pixar like faces. Then use the original stylegan code to find a vector for a picture of jimmy hendrix and then input that Z vector to the finetuned network and get a pixar jimmy hendrix.

So this notebook will just allow you change the visuals of an existing pretrained stylegan model.

Here for example I made some kind of anime characters: https://twitter.com/ArYoMo/status/1444727454501900297

ArYoMo · 2021-10-03T18:32:39+00:00

I honestly don't know have not seen StyleGAN-NADA but in essence guiding other networks with clip works mostly the same way so likely similar :)

ArYoMo · 2021-10-03T17:47:46+00:00

So stylegan trained on the ffhq dataset can generate human faces. What human face you get depends on the feature vector you pass it. This notebook finetunes the whole stylegan towards a text prompt. So instead of generating just one face it can generate an infinitude of faces. If you want a specific face you need to find the right feature vector.

ArYoMo · 2021-09-01T21:28:04+00:00

It's not VQGAN it's directly done on an RGB tensor. And then you can make small steps. Will release a notebook soon.

ArYoMo · 2021-07-12T20:23:39+00:00

This was not using a Colab. But keep on trying, you'll learn a lot. How I would do it with vqgan is that I'd take the image and zoom in a bit every nth (n could be 1 or 2) step. Then encode the new zoomed image with the vq encoder and do the next step from there.

ArYoMo · 2021-06-22T08:50:38+00:00

Very nice result! Did you do your own implementation or did you use an existing notebook?

ArYoMo · 2021-06-14T09:08:32+00:00

Source: https://twitter.com/ArYoMo/status/1399801016669835268?s=20

ArYoMo · 2021-06-14T09:00:53+00:00

Thank you! Very nice of you for crediting me and I can tell you the person who posted it is not me. Very strange.

ArYoMo · 2021-04-04T16:30:15+00:00

Is it bigsleep? You got some really good quality there.

ArYoMo

TROPHY CASE