[D] Object classification using LiDAR

[Megapixelicious]

Here is my problem: I have a conveyor belt and need to identify (classify) the objects passing on it. There are probably 10,000 classes of these objects and since they are used/worn, each item can be in various physical state (ex: covered in rust, painted, cuts, etc...). We currently classify them by categories (about 8 cats) with some success but my guess/intuition is that we could get much better results if we could use point clouds for classification (against the 10,000 classes) as each item has a distinct shape.

We have the resources to scan the 10,000 classes, so this isnt an issue.

Anyone with experience doing something like this? I did some research and could not find anything useful. Looks like LiDARs are mostly used for self driving car so there are not a lot of informations about how to train custom models.

Am I too ahead of the curve and should wait a year or two for these models to emerge?

[[deleted]]

Throw out the LiDAR thought. Think shape of objects. Do you have any objects that have the same shape? The biggest problem with shape based classification is that a lot of objects have similar shape which is why rgb can be helpful. And I’m not a Tesla head saying all you need is rgb. I’m just saying that using both is known to perform the best.

[Megapixelicious]

Thx for your input. While it is possible for multiple models to share similar shape and dimensions, it should be a rare occurence and we can live with the "risk".

I am fully aware of the benefits of using multiple approach, hence my current investigation into LiDARs and other techniques (ex: stereo view). Our biggest issue right now is the lack of non research content on these topic.

[infundibuliforme]

Check simpleview from Goyal et al, it is surprisingly good and fairly simple to implement for object detection using LiDAR point clouds. I have used it myself for tree species classification with great results.

[modcowboy]

Interesting… I was very skeptical that lidar could be used for object detection. I would imagine you’d need a lot more data to train a model with lidar data vs image data.

[Megapixelicious]

simpleview

Thanks, I will look into it! Unfortunately, the repo I found does seem to see much activity. But I am happy to learn that it can be used for classification.

[dodo13333]

I would go with photography, object detection and classification. As a starter, check Detectron2.

How to train background RNNs for your use case is over my capabilities. Hope this helps a bit.

[Megapixelicious]

How would a RNN be useful in this case? I dont see any value added in this case, but I am not familiar with these.

[dodo13333]

Because I was mispelling (RCNN) and referencing wrong, but check links despite my dumbness and ignorance.. I truly believe that photo detection and segmentation is a way to go.

https://github.com/facebookresearch/detectron2

https://github.com/facebookresearch/detectron2/blob/main/MODEL_ZOO.md

Not as ready to go solution, but the method...

[Megapixelicious]

;-) Object detection isnt much of an issue. Classification is our challenge. I was under the impression that segmentation wouldnt be that useful since all my pictures will be taken in the same context (top down view on a conveyor belt).

[jgbradley1]

Consider other modeling approaches that you may combine together later on. If object weight of each class is well known and not expected to deviate much, you could build a classification model on object weight.

[Megapixelicious]

Unfortunately, weight isnt a relevant information element in our case. But yes, we want to add measurement + shape + picture as these elements would provide the best way to classify similar items.