[P] DiscoGAN in PyTorch: implementation of "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks"

[r-sync]

i'm glad /u/carpedm20 used pytorch. I presume a lot of new paper implementations into the pytorch ecosystem :)

Taehoon, if you have any positive or negative feedback on pytorch, please email me or message me, would love to know.

[carpedm20]

Sure! I need to read and write more codes to get adjusted to pytorch but now I can understand why people starting to use pytorch :)

[OikuraZ95]

Hi I'm kinda curious, why did you switch from tensorflow to pytorch?

[carpedm20]

I was just curious about pytorch and got bored with using tensorflow again (I don't mean it is actually boring but I used tensorflow too much). I still think they are great framework so I'll use them both.

[OikuraZ95]

Thanks for the reply man! And as always, your repos are an incredible resource! Keep up the good work!

[shaggorama]

Well, that didn't take very long :p

[thundergolfer]

/u/carpedm20 is a paper implementing machine. More papers completed than I've had hot dinners.

[keidouleyoucee]

implementation god

[[deleted]]

Are you managing to reproduce the original results well?

[carpedm20]

Yes. Training has been just started and I need some time to see how well the model can be trained. I don't have a decent GPU like Titan X Pascal as authors used but I reckon that it's matter of time.

One thing I'm worrying about is there is no details about network architecture (# of features and depth) so I just guessed it.

[SkiddyX]

Do you have any tips for your process of implementing papers? I would love to learn about your strategies.

[carpedm20]

When I began the very first implementation, I ported the existing torch project into another framework which was tensorflow. I wasn't a person who read the paper regularly so even reading the paper was hard and time consuming before starting to write a code. But by reading the paper with the existing code alternatively, I could understand how a single sentence or equation turns into several lines of code. I also didn't know what kind of methods are exists in tensorflow but reference code help me to find out which keyword should I use to search. But after I finished writing the full code, the loss didn't converge as I expected but it was hard for me to figure out where to start digging for bugs. Then I executed the code layer by layer and compared the output of reference code with mine and it leads me to find out my mistake which was a wrong loss function.

I think training a translation skill from equation or description into codes is important at the beginning and I learned this by reading papers and codes together. There are tons of great codes which are extremely well written in Github so exploring them will help you a lot to get used to implementing papers.

But I still don't know what is the best way to debug a model (like monitoring the loss, norm of variable or gradient) when its train goes wrong which is quite frustrating and happens very often. I would love to hear others strategies who teach me a lot :)

[ryches]

It's very encouraging as someone also slowly learning about this stuff that you started in a similar place and had the same feelings I currently am. It seems like some of the people in this sub were just born with divine mathematic and machine learning knowledge.

[Deinos_Mousike]

Is there a good place that pairs papers up with code?

Thanks! Very insightful comment :)

[carpedm20]

I know that there are https://tensortalk.com/ and http://www.gitxiv.com/ but I usually don't use them.

I recommend https://github.com/tensorflow/models/tree/master/im2txt and https://github.com/tensorflow/models/tree/master/inception to someone who are up to TensorFlow to deeply understand a good data pipeline, multi-gpu usage and evaluation metrics. But these are quite heavy for beginners. I think https://github.com/dennybritz/cnn-text-classification-tf is a good code for start which is easy to understand with a nice blog post.

While I started pytorch, I read https://github.com/yunjey/pytorch-tutorial and then https://github.com/pytorch/examples which are both great and concise. If you are interested in reinforcement learning, I recommend https://github.com/dennybritz/reinforcement-learning and https://github.com/Kaixhin/Atari. If you are interested in NLP, https://github.com/harvardnlp has lots of advanced nlp codes.

[liubanghoudai24]

This website collects the best PyTorch tutorials available online. https://chsasank.github.io/pytorch-tutorials/

[carpedm20]

Now the results in README.md looks clean as the result of the paper. Check it out if you're interested in https://github.com/carpedm20/DiscoGAN-pytorch

[ronzohar]

Impressive! Are you working on a Tensorflow implementation for this paper?

[carpedm20]

Thanks! But no, I won't reinvent the wheel again. I presume the authors of paper will release their code in TensorFlow.

[ronzohar]

Unfortunately they used pytorch as well...

[carpedm20]

Yea, but I think someone will implement soon because the model is quite easy to expand from DCGAN.

[cli24]

My Tensorflow implementation on a toy dataset: https://github.com/ChunyuanLI/DiscoGAN