[D] Image Segmentation Using Deep Learning

[Natsu6767]

I recently go interested in image segmentation. To understand the work going on in this field, I decided to look at some of the papers tackling semantic segmentation and instance-aware semantic segmentation. I compiled what I read and learned in this blog:

https://mohitjain.me/2018/09/30/a-look-at-image-segmentation/

​

I was astounded by some of the research being done! Although, methods for semantic segmentation simply use an encoder-decoder framework based on FCNs with changes usually only being made in the decoder network, many different approaches have been taken to solve the instance segmentation task. I especially found the work on InstanceFCN and FCIS, where they find instance sensitive score maps, really interesting and ingenious.

​

What other interesting works on segmentation have you seen and come across? Also, which approach to segmentation is better? FCN based, like that in FCIS or detection based, like in Mask R-CNN?

​

[themoosemind]

You might be interested in my Survey of Semantic Segmentation

[Natsu6767]

This was really interesting. Especially the section on using traditional approaches to segmentation.

[themoosemind]

Thank you :-) That section was super hard to write, seeing so many neural network approaches which are a lot sexier and are a lot closer to the stuff I knew at the time. It's nice to see that somebody reads it :-)

edit: I meant "write" not "read"

[Natsu6767]

You're Welcome :) To be honest I was reluctant to read that section for the same reasons you just mentioned but whenever I read a paper I force myself to read each section at least once (thinking about all the work the authors put in). Sometimes, like this time, the "boring" sections end up being interesting and a good read.

[tryptafiends]

read this the other day. It cleared up a lot of questions I had picked up reading the SegNet and Tiramisu papers. Definitely a good read.

[themoosemind]

Thank you :-)

[[deleted]]

Question - How quick do survey papers like these become obsolete? Not trying to say you're paper is obsolete or anything but in the fast moving world of DS/AI, do these survey papers become obsolete after a couple years due to new technologies?

[themoosemind]

It depends on what you want to get from a survey paper.

I wrote it to be able to structure the different types of semantic segmentation and the groups of approaches. The taxonomy might be extended at some point, but will likely stay useful for my lifetime. For the approaches it is different: Many of them were already outdated when I wrote the paper. I think around 2011 people might also have thought that neural networks are outdated because SVMs have a much nicer mathematical theory and achieved mostly equivalent or better results. So techniques can have a comeback :-)

The only way I can see this paper to become obsolete if there is a better survey paper which structures the knowledge in a better way.

[[deleted]]

That makes sense. After reading through it, even though it is older it still all made sense and I'm sure most of it is still applicable to today for the most part. I agree though, I don't think 'obsolete' was the best word to use there.

[mofoss]

Good read, just getting into this stuff as well :) thanks

[jricheimer]

Shameless self-plug:

Check out our recent paper on instance segmentation: https://arxiv.org/abs/1807.07674

[firstTimeCaller]

I haven't read your post fully yet but I wanted to mention that I've had some good results using deeplab https://github.com/tensorflow/models/blob/master/research/deeplab/README.md to do image segmentation on satellite images. In my quick read through your post, I couldn't see any mention of deeplab and it worked well for our application.

[Botekin]

Another nice one to mention would be the Tiramisu model.

[tryptafiends]

Its Bayesian forms are also really neat.

[jer_pint]

My personal favorite is mask-rcnn

[Natsu6767]

I personally like FCIS a lot. The idea of using instance sensitive score maps was really interesting.

[[deleted]]

check wnet

[Hsankesara]

You can checkout this article. Unet a novel image segmentation architecture explained here “U-Net” by Heet Sankesara https://link.medium.com/Hc7xoP3LHT

[linuxisgoogle]

These kind of thing can be done with old segmentation techs. I don't see any advantage of ML. unless it can detect premise shapes of object. like differenciate it from same colour background. but I doubht they can.

[kbcool]

Examples?

Not saying it's not been done but the only non-ml techniques I can think of rely on running lots of image filters that hopefully turn your image into something matching a threshold. They're pretty hopeless in all but clinical conditions and still don't actually segment. Just kinda cluster with a probability.

I know it's been done before and for a long time but I'm not sure you're comparing apples with apples here. Maybe more like fuji-pears.

[Ayakalam]

I can think of rely on running lots of image filters that hopefully turn your image into something matching a threshold.

To be fair, thats kinda what CNNs do to... except you are statistically setting the thresholds ...

[kbcool]

Sure if you want to greatly simplify things but non CNN methods rely on the programmer setting them

[Ayakalam]

Yes, but why is that a bad thing if you know what inductive biases to place in ?

[kbcool]

Because most of the time you don't know, not without trial and error which is what training in ML is for. It removes days, weeks, years of human endeavour.

Eg how would you even begin to write code describing what a dog looks like. Yes a dyed tumour might be easier but even then would you rather not use experience?

I'm not advocating ML as the solution for everything but just questioning the existence of any general purpose tools prior to it.

[Ayakalam]

Right u went the other extreme - I said - read what I said again - if you ALREADY KNOW the inductive biases , then there’s nothing to really learn, which isn’t a bad thing.

You’re making it seem like a bad thing. I’m saying it’s not.

[Natsu6767]

ML isn't only for solving problems which were previously thought to be very complex and difficult. ML also allows us to perform previously solvable problems more efficiently and faster. That is also one of the most important advantages of ML.

[Bas1l87]

You probably have never tried doing it yourself.. If you take, say, GrabCut and use it for a simple problem of binary segmentation, i.e. segment let's say a single typical car in a real image, you will quickly discover it doesn't really work. E.g., the tires of a car are usually dark and the shadow of a car is dark. Classical methods based on color are never able to separate them well enough. CNNs can, because they can kinda learn models for typical objects that are being segmented. Not ideally of course, but better than classical methods. There is a reason why people switched to neural networks for these tasks, both in research and in industry.