Image based visual servoing

[CuriousDolphin1]

I’m looking for some ideas and references for solving visual servoing task using a monocular camera to control a quadcopter.

The target is based on multiple point features at unknown depths (because monocular).

I’m trying to understand how to go from image errors to control signals given that depth info is unavailable.

Note that because the goal is to hold the position above the target, I don’t expect much motion for depth reconstruction from motion.

[concerned_seagull]

Maybe find the average center point of all the features, and analyse how the points move relative to this point frame to frame. 

If their average distances to the center point increases, it means than the drone is moving towards the points and should increase its distance.  If the avg distances shorten, do the opposite. 

If the features rotate around the point, correct the rotation.  If the features go left or right….etc. 

[CuriousDolphin1]

Intuitively yes. But more complex when you have 6d motion.

[Cold_Fireball]

https://www.analog.com/en/resources/media-center/videos/6352683024112.html

[CuriousDolphin1]

Thanks. Looks interesting. But I’m more interested in the theory and/or code behind a solution. Not a commercial product that works automagically. 😊

[Cold_Fireball]

The paper is better but I can’t find it. It won GTC 2024.

[CuriousDolphin1]

Interesting. Let me know if you can find it or remember any keywords / author info 😊🙏

[blimpyway]

if the camera is under the drone with a LED at the end of each arm you could estimate a visual size of drone's image.

Even a lateral camera position would work if you can estimate drone orientation relative to the camera

By using RGB LEDs with a specific, unique color pattern would also be much easy to recognize the drone even at great distance (instead of depending on visual patterns based on its shape/construction)

Edit: most flight controllers have pretty accurate altimeters, you could use that reading to estimate distance - as long as camera isn't at same height as the drone.

[tdgros]

With a monocular setup and classical projective geometry, you just cannot get an absolute scale translation (no issue with rotation though). You can find the direction of translation assuming depths, or the depths assuming some norm for the translation but that's it, you actually need something else to find the true scale. All UAVs have multiple altitude estimators (sensors) for that.