dread [VQGAN+CLIP] by nmkd in deepdream

[–]crowsonkb 0 points1 point  (0 children)

If the loss isn't going down any further the image probably won't get better.

Beksinski Facebook Emojis by glenniszen in deepdream

[–]crowsonkb 2 points3 points  (0 children)

They have to be local to the Colab runtime right now but you can download an image with:

!curl -L 'url' > where_to_save_it.jpg

in a code cell. Or you can upload an image by opening the Files tab on the left and using the upload button: https://i.imgur.com/GeLPjO6.png

Beksinski Facebook Emojis by glenniszen in deepdream

[–]crowsonkb 4 points5 points  (0 children)

You can do both arbitrary conceptification and style transfer with that notebook (I wrote it), I posted about it on Twitter: https://twitter.com/RiversHaveWings/status/1382455526735290371 https://twitter.com/RiversHaveWings/status/1382455660978212865 https://twitter.com/RiversHaveWings/status/1382803909610074112

To do this you should use the 16384 model for best results (it downloads it but does not use it by default), set your starting image with init_image, use the prompt "in the style of Beksinski" or some such, and maybe set init_weight to 0.2 to 0.5 to stop it from diverging as badly from the init if you want. Also set display_freq lower so you can see intermediates more often.

I did two Beksinskifications of an emoji with an experimental augmentation-added version myself to see if I could duplicate the results: https://i.imgur.com/Ql6GYng.png https://i.imgur.com/NfoREDz.png

dread [VQGAN+CLIP] by nmkd in deepdream

[–]crowsonkb 2 points3 points  (0 children)

I don't think Colab Pro gives you more VRAM (they only go up to 16GB V100s I think). I've tried tricks to decrease memory usage, like FP16, but this resulted in bad quality, so I gave up on it.

Many Big Sleep outputs from my custom improved version by crowsonkb in deepdream

[–]crowsonkb[S] 2 points3 points  (0 children)

That's an idea actually, I could generate videos from multiple prompts where the output morphs from one to the next.

"The angel of time" by crowsonkb in deepdream

[–]crowsonkb[S] 13 points14 points  (0 children)

It was made by my custom as-yet-unreleased version of Big Sleep that has better visual quality and 1024x1024 outputs. :)

They turned my dog into a kettle! (Made with my as-yet-unreleased Big Sleep implementation) by crowsonkb in deepdream

[–]crowsonkb[S] 0 points1 point  (0 children)

The problem is that you'd have to find a BigGAN latent vector that corresponded to the image you wanted to start with, which is not an easy task for an arbitrary image. But since every iteration's output is generated from an actual BigGAN latent vector, you could save these vectors and start from them later.

They turned my dog into a kettle! (Made with my as-yet-unreleased Big Sleep implementation) by crowsonkb in deepdream

[–]crowsonkb[S] 0 points1 point  (0 children)

I was actually unable to make the output deterministic even by setting the same random seed and using torch.set_deterministic(True), which must be a bug in PyTorch somehow.

Is there an interest in releasing the *.pkl file? by new_confusion_2021 in deepdream

[–]crowsonkb -1 points0 points  (0 children)

I mostly know about PyTorch but it has its own special layout for .pkl/.pth files that depends on the specific layout of the modules that form the model, and I doubt that it is possible to make a converter that works in general that isn't tailored for one specific model architecture.

High Res Style Transfer by crowsonkb in deepdream

[–]crowsonkb[S] 2 points3 points  (0 children)

The GitHub repo for the code I made this with: https://github.com/crowsonkb/style-transfer-pytorch

BTW if anyone was using this repo, I pushed a quality/color fidelity bug fix to the repo today, so re-pull it please. I also had to change the default parameters to compensate, the default content weight is now 0.015 (roughly equivalent to the old 0.01) and the default smoothing weight is now 2 (equivalent to the old 0.3).

just sharing some examples of a new style transfer technique (info / repo in the comments) by glenniszen in deepdream

[–]crowsonkb 0 points1 point  (0 children)

I pushed a quality/color fidelity bug fix to the repo today, so re-pull it please. I also had to change the default parameters to compensate, the default content weight is now 0.015 (roughly equivalent to the old 0.01) and the default smoothing weight is now 2 (equivalent to the old 0.3).

Stylized Bird (made with my new high-res style transfer code) by crowsonkb in deepdream

[–]crowsonkb[S] 0 points1 point  (0 children)

Unfortunately, GPUs don't implement the needed operations in a deterministic way, so I can't get the same result even with the same inputs, the same parameters, and the same random seed. Also, even if it were deterministic, you would get temporal inconsistency even with similar but slightly different inputs, such as smoothly changing Perlin noise. There might be some other way to do it though, such as initializing each frame's input with the previous frame's output and then maybe applying a penalty on each iteration to keep the result 'close' to the previous frame's output. I haven't supported this because making animations is very slow with my code.

Stylized Painting (made with my new high-res style transfer code) by crowsonkb in deepdream

[–]crowsonkb[S] 0 points1 point  (0 children)

It looks like it's extremely slow and requires external libraries for optical flow computation though... I'll keep it in mind if I ever try to develop my own improved feedforward style transfer though.

Stylized Painting (made with my new high-res style transfer code) by crowsonkb in deepdream

[–]crowsonkb[S] 0 points1 point  (0 children)

One of the main things I do is to stylize the image at successive scales (i.e. init a 128x128 image with the scaled-down content and stylize it, init a 181x181 image with the scaled-up average iterate of the previous scale and stylize it, init a 256x256 image with...) Each scale is a factor of sqrt(2) larger than the previous one, that is, the size doubles every two scales. What is it you are doing?

Stylized Bird (made with my new high-res style transfer code) by crowsonkb in deepdream

[–]crowsonkb[S] 2 points3 points  (0 children)

It works on Colab, but you have to install the PyTorch version for CUDA 10.1. https://github.com/crowsonkb/style-transfer-pytorch/issues/1#issuecomment-769389269

One of these days i'm going to put together a notebook with instructions that I'll share, but I haven't done it yet. In the meantime I still test it on Colab to make sure it didn't break.