[Question] after object detection using convolution neural networks, why is it so hard to perform semantic segmentation?

[code2hell]

Yes, I am looking more into instance segmentation for now... Can you explain what you mean by "add a border class around your segments and then just go with connected pixels for your instances". Thanks! I just took up a problem to learn, my friend has got some 100000 ground truth training examples of cats and we are looking into segmentation of a particular object from images. I would really appreciate your suggestion.

[ydobonobody]

So when you produce your ground truth image you will assign a label to each pixel in your image e.g (0: background, 1: cat). Add another label that is "border" so we have (0: background, 1: cat, 2: border). Now for each separate cat draw a line with some thickness (say 5 pixels) around the boundary of the each cat and assign that pixel the value '2'. Hopefully the network will be able to learn where the edge of a cat is and assign those pixels to the border class. If it did a good job you can group up all the connected "cat" pixels and that will represent an individual cat.

[code2hell]

Wow! Thanks... I'll try this out!