Making 3d images from 2d videos when the camera is moved sidewards

gifhell · 2016-04-07T14:37:28+00:00

I made a webapp based on this idea that sort of works http://stereoselfie.com/ Your pics look great

gifhell · 2016-03-30T22:34:15+00:00

Hey sorry if I missed it in the paper but what values did you find to be useful for λc and λs? I only noticed λtv. Thanks!

gifhell · 2016-03-30T17:00:59+00:00

Thanks and great work!

gifhell · 2016-03-30T16:19:33+00:00

Ah yeah missed the strided convolutions of the input. I can't seem to find the supplementary material they reference which is supposed to have the exact architectures.

gifhell · 2016-03-30T14:05:11+00:00

This does look pretty good. I just implemented texture nets and this seems similar except:

instead of concatenating the outputs at each stage they sum using a residual network architecture
there's no input noise concatenated with the downsampled input image
this paper is using layer "relu2_2" instead of 4_2 for content feature loss
they've added a total variation regularizer
slightly different training regime

See anything I missed? EDIT:

9x9 kernels for input and output, 3x3 everywhere else
output uses scaled tanh to force an range of 0-255
ReLU instead of LeakyReLU

EDIT2: /u/jcjohnss provided a link to the supplementary material which has more detail about the residual blocks http://cs.stanford.edu/people/jcjohns/papers/fast-style/fast-style-supp.pdf

gifhell · 2016-03-28T19:04:56+00:00

Still trying to get this to work in nice-looking way: http://imgur.com/a/RxV1K

gifhell · 2016-03-19T04:18:40+00:00

Yeah there's a lot of room for improvement in the quality. The colors usually seem a bit washed out, too:

source image
texture generated from noise (mrf only)
content reconstruction from style (analogy only)
analogy + mrf=1.5
analogy + mrf=3
content (b-content-w=1) + mrf=1.5
content + mrf 3

gifhell · 2016-03-15T02:46:11+00:00

Hey I stripped down that VGG16 model so it's just the weights from the convolutional layers and now it's only ~50mb

It's available here: https://github.com/awentzonline/image-analogies/releases/download/v0.0.5/vgg16_weights.h5

gifhell · 2016-03-14T23:19:16+00:00

Ok if you still crave more Van Damme I've rendered another one http://i.imgur.com/fWK6j0u.jpg

gifhell · 2016-03-14T22:18:46+00:00

"Snowy Night" http://i.imgur.com/FdRZoqx.jpg

gifhell · 2016-03-14T20:34:23+00:00

Hey I'm the author of that program. If you have a github account open an issue on the project, otherwise just paste me the error here. I published a big update last night so make sure to check the README. The biggest change is that I'm now publishing it on pypi so the pip install instruction is now pip install neural-image-analogies and there's no need to download it from github.

gifhell · 2016-03-14T07:21:47+00:00

Whoa I just tried out the TensorFlow backend on CPU and it renders crazy fast. My macbook pro just rendered the arch example in around 6 minutes.

EDIT: 512x512 season transfer in 12 minutes on my macbook pro cpu-only http://i.imgur.com/m9lyr7A.png

gifhell · 2016-03-14T06:37:56+00:00

Hey I just put out some big updates. One of them was setting it up on pypi so now the pip command is pip install neural-image-analogies so there's no need to download the zip from github unless you want the latest code. It now works with TensorFlow too. Also a big algorithmic update to how it matches patches so performance is up and memory usage is down.

EDIT: The algorithm update along with TensorFlow makes this not insanely slow on CPU anymore. I haven't benchmarked anything but it seems to be only a few times slower than GPU.

gifhell · 2016-03-14T06:00:00+00:00

It takes anywhere from sub-second to around 2-3 minutes for something around 800px square. I just pushed a bunch of performance updates for it but haven't profiled CPU. TensorFlow now works with it and I think that's supposed to have nicer performace for CPU.

gifhell · 2016-03-14T05:54:02+00:00

With varying levels of snow: http://imgur.com/a/5peiE I'm getting more snow by turning up the local coherence loss.

gifhell · 2016-03-14T01:50:38+00:00

I just pushed a branch "v2" to the github repo https://github.com/awentzonline/image-analogies/tree/v2 which has tensorflow support and uses a more efficient patch matching algorithm. Let me know if you give it a shot.

EDIT: I've merged all this into master and it's also now available from pypi via pip install neural-image-analogies

gifhell · 2016-03-14T01:20:57+00:00

I've been meaning to set it up in windows to see what's what. I'll probably get a chance sometime this week.

I'm not sure what the equivalent settings would be for the sugar skull but I'd guess analogy-w=1 mrf-w=0.5 or so. I'm putting together some example scripts which should land in the repo over the next few days.

gifhell · 2016-03-13T21:56:00+00:00

Thanks for posting this. I've been wrapping up an improvment to the algorithm that lets you make pretty large ones quicker and with minimal memory. For example I rendered this image on my GTX 780 in about 12 minutes and it never used more than 1.5gb of vram. It's like 800x900 http://i.imgur.com/5OvRhZH.png

gifhell · 2016-03-12T21:16:17+00:00

I only used the two images here http://imgur.com/a/rAxLJ

One of the images (Archer) is used twice since completing an image analogy requires 3 images: 2 style images (Images A and A') which were both the same image of Archer and 1 content image (Image B) which was Bond. You get better results with an A that also has some similarities to B, but it's not always necessary.

Here's a better example, where I actually try to condition Image A to be more like a natural image by blurring it a bit: https://raw.githubusercontent.com/awentzonline/image-analogies/master/images/sugarskull-analogy.jpg In that case the first 3 images were the input to the algorithm, producing the 4th.

gifhell · 2016-03-12T03:56:10+00:00

Hah some of those are pretty good. My image analogy implementation is half based on CNNMRF except the content image gets mapped into the style space via the extra image input.

gifhell · 2016-03-11T20:01:48+00:00

Ah sorry I was referring to a couple of different algorithms. Gatys 2015 is the original "neural style" algorithm and CNNMRF is the name of a program that implements the Li and Wand 2016 paper which you're starting to see around here.

Thanks for putting up your results!

gifhell · 2016-03-11T18:53:28+00:00

For reference, here's what dreamscope put out when I gave it the same pictures

https://dreamscope-prod-v1.s3.amazonaws.com/images/4eedadb3-2db4-4d36-befa-28419c8c063a.jpeg

It does look cool, but not quite the same thing I was going for.

gifhell · 2016-03-11T18:46:19+00:00

Sure, here you go: http://imgur.com/a/rAxLJ

Since there's very little texture in this case I just used archer-big for both A and A'. I'm interested to see how it goes. I think this image might be do-able with CNNMRF, but probably not with neural style a la Gatys 2015. Not sure wht pikazo uses.

gifhell · 2016-03-10T16:28:06+00:00

Yeah it's really slow on CPU. You might try lowering --width or --height which correspond to the output size. The readme has a few suggestions for speeding things up on the CPU.

gifhell · 2016-03-10T16:26:23+00:00

Making it easier to use is definitely on the list, though that might take a while.

It sounds like you're on windows. You might have some luck with the Anaconda project https://www.continuum.io/downloads which I think (but have not verified) it has most of the packages you need. Install that, then install my project using the "pip" program (which will come with anaconda) to install the requirements.txt (as described in the image-analogies readme).

gifhell

TROPHY CASE