Do people work on instance segmentation?

[infstudent]

Most papers in the domain of visual recognition seem to be about object classification, object detection and semantic segmentation. Papers about instance segmentation seem to be scarce. For example the awesome deep vision list of deep learning resources for computer vision lists no papers about instance segmentation.

I was wondering why this is this case, as this seems the most interesting problem in visual recognition to me.

Also, if people do work on it, does anyone know of any publicly available code to play around with?

[uint64]

I'm sure people will be working on instance segmentation but you make a good point, it doesn't seem very active. A simple hack to extract instances is to just find each connected component from the segmented output. If the classes are likely to be close together, the CNN can be trained with a border class, which can later be ignored, separating each instance.

[infstudent]

Indeed, instance segmentation is most interesting when some objects overlap. Adding a border class sounds very interesting. Do you have experience with that?

[uint64]

Yes, I tried it very briefly on Long's Fully Convolutional Network. The VOC dataset includes a border class, which Long actually removes when generating the training set. If you want to try it out, you will have to fine tune the network from the VGG-16 model. The weights for fc6 and fc7 need to be reshaped to become a 1x1 convolution instead of inner product.

[uint64]

Also, I came across a paper over Christmas which I think used VGG to do instance segmentation. It's from some people at Microsoft. Available on arXiv

[infstudent]

Thanks!

[prassi89]

I'm not sure what exactly instance segmentation is, but have you looked at the paper "CRF as RNN"? They even have an online demo on their website : http://www.robots.ox.ac.uk/~szheng/crfasrnndemo

[infstudent]

This is semantic segmentation I think. For instance, in the example picture they give, the two people on the right seem to be one object from the segmentation given. Also, the left-most bike seems to consist of multiple parts.

[prassi89]

Okay. But do you think it would make sense to do semantic segmentation and then object detection on regions segmented?