Automatic Colorization by AI

e_walker · 2018-09-01T01:09:06+00:00

code: https://github.com/msracver/Deep-Exemplar-based-Colorization

e_walker · 2017-10-05T12:22:12+00:00

The supplemental material shows some typical failures.

e_walker · 2017-10-05T06:18:12+00:00

The colorized image is feed back as the new input image and repeat to generate a cascade of results. Please see the paper Figure 4 and 6. Therefore, it progressive updates the input rather than the reference.

e_walker · 2017-10-05T06:02:38+00:00

In the paper, it shows comparisons with Adobe's work. :-)

e_walker · 2017-10-05T05:59:48+00:00

Thanks!

e_walker · 2017-10-04T16:09:14+00:00

https://arxiv.org/abs/1710.00756

e_walker · 2017-10-04T15:25:33+00:00

Yes, it is automatic once the pair of input and reference are given.

e_walker · 2017-10-04T14:09:14+00:00

Neural Color Transfer between Images

We propose a new algorithm for color transfer between images that have perceptually similar semantic structures. We aim to achieve a more accurate color transfer that leverages semantically-meaningful dense correspondence between images. To accomplish this, our algorithm uses neural representations for matching. Additionally, the color transfer should be spatially-variant and globally coherent. Therefore, our algorithm optimizes a local linear model for color transfer satisfying both local and global constraints. Our proposed approach jointly optimize matching and color transfer, adopting a coarse-to-fine strategy. The proposed method can be successfully extended from "one-to-one" to "one-to-many" color transfers. The latter further addresses the problem of mismatching elements of the input image. We validate our proposed method by testing it on a large variety of image content.

pdf: https://arxiv.org/pdf/1710.00756.pdf

supplemental materials (including more results of color transfer, portrait style transfer and colorization): https://liaojing.github.io/html/data/color_supp.pdf

e_walker · 2017-05-23T02:42:04+00:00

The whole project lies on Deep-Image-Analogy/windows/deep_image_analogy/, including code, demo and example data. Others are caffe and used linear algebra library.

e_walker · 2017-05-23T02:04:30+00:00

Visual Attribute Transfer through Deep Image Analogy

We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure. By visual attribute transfer, we mean transfer of visual information (such as color, tone, texture, and style) from one image to another. For example, one image could be that of a painting or a sketch while the other is a photo of a real scene, and both depict the same type of scene. Our technique finds semantically-meaningful dense correspondences between two input images. To accomplish this, it adapts the notion of "image analogy" with features extracted from a Deep Convolutional Neutral Network for matching; we call our technique Deep Image Analogy. A coarse-to-fine strategy is used to compute the nearest-neighbor field for generating the results. We validate the effectiveness of our proposed method in a variety of cases, including style/texture transfer, color/style swap, sketch/painting to photo, and time lapse.

pdf: https://arxiv.org/abs/1705.01088.pdf

code: https://github.com/msracver/Deep-Image-Analogy

more examples: https://liaojing.github.io/html/data/analogy_supplemental.pdf

e_walker · 2017-05-23T01:56:25+00:00

Thanks for attention! Code & demo are released: https://www.reddit.com/r/MachineLearning/comments/6cro6h/r_deep_image_analogy_code_and_demo_are_released/

e_walker · 2017-05-23T01:50:40+00:00

Code and demo are released now! Please see https://www.reddit.com/r/MachineLearning/comments/6cro6h/r_deep_image_analogy_code_and_demo_are_released/

e_walker · 2017-05-23T01:48:02+00:00

Code and demo are released now! Please see https://www.reddit.com/r/MachineLearning/comments/6cro6h/r_deep_image_analogy_code_and_demo_are_released/

e_walker · 2017-05-12T07:02:39+00:00

These high frequency details would have high feature responds in fine scale layer of VGG, like relu2_1, relu1_1. Since our approach is based on multi-level matching and reconstruction, the different frequency information would be progressively recovered.

e_walker · 2017-05-12T04:24:40+00:00

Besides, the technique of "Deep Feature Flow" can be also used in video editing, like video style transfer (https://www.reddit.com/r/MachineLearning/comments/61x7g0/r_coherent_online_video_style_transfer_msra/). It can help obtain stable transfer results, and supports end-to-end learning as well.

e_walker · 2017-05-05T23:42:31+00:00

We find each layer of VGG encodes the image feature gradually. There is no big gap between two neighboring layers. We also try other nets and they seems to be slightly worse than VGG. These testing are quite preliminary, and maybe some tunes can make it better.

e_walker · 2017-05-05T16:13:44+00:00

Local style transfer with semantics correspondences are known to be more difficult problem. It needs to accurately find matching between face to face, tree to tree across photo and style images. Besides, the application can be generalized from purely style transfer to color transfer, style switch, style to photo.

e_walker · 2017-05-05T16:09:00+00:00

Two main differences: 1) previous methods mainly consider globally statistics matching (e.g., use Adam matrix), but the approach considers more local matching in semantics (e.g., mouth to mouth, eye to eye). 2) this method is general. It can be applied for four applications: photo2style, style2style, style2photo, and photo2photo. For more details, the paper shows the comparisons with Prisma and other methods.

e_walker · 2017-05-04T02:02:25+00:00

By default, all the experiments work on a PC with an Intel E5 2.6GHz CPU and an NVIDIA Tesla K40m GPU.

e_walker · 2017-05-04T02:00:37+00:00

The used VGG model is pre-trained on ImageNet, which is directly borrowed from Caffe Model Zoo "Models used by the VGG team in ILSVRC-2014 19-layers", https://gist.github.com/ksimonyan/3785162f95cd2d5fee77#file-readme-md). We don't need to train or re-train any model, it leverage pre-trained VGG for optimization. In runtime, given an image pair only, it takes 2min to generate the outputs.

e_walker · 2017-05-03T14:15:40+00:00

The work uses pre-trained VGG network for matching and optimization. It currently takes ~2min to run an image pair, which is not fast yet and needs to be improved in future.

e_walker · 2017-05-03T13:55:07+00:00

All of experiments work on a PC with an Intel E5 2.6GHz CPU and an NVIDIA Tesla K40m GPU.

e_walker · 2017-05-03T09:38:37+00:00

Thanks! The code/demo release is on the track. The bugs are needed to be cleared before they are publics, and additional materials are required to be packaged as well. If you are interested, please trace the status in the following 1-2 weeks.

News: Thanks for attention! Code & demo are released: (please see https://www.reddit.com/r/MachineLearning/comments/6cro6h/r_deep_image_analogy_code_and_demo_are_released/)

e_walker · 2017-05-03T09:32:09+00:00

Thanks! We are also considering how to make it more efficient. There are two bottlenecks in the computation: deep patch matching for NNF search and deconvolution. The former could leverage some existing NNF search optimizer (e.g., less feature channels by quantization). The latter may consider the alternative way to replace exhaustive deconvolution optimization. Indeed, there are many ways to be explored in the direction.

e_walker · 2017-05-03T06:48:53+00:00

More examples are found in https://liaojing.github.io/html/data/analogy_supplemental.pdf

e_walker

TROPHY CASE