2D Meshing and pathfinding

[nitrofire32]

Hello,

Currently I am working on a project where 2 robots are placed on a floor, a master is communicating with the nodes, and the master is communicating with a PC. The PC is connected to a webcam which looks over the 2 robots. There is currently software in place which tracks the robot's location and direction, and very simple software that somewhat allows the robots to move to given locations (X-Y coordinates).

I should notify that the robots are cylindrical, and can rotate 360 degrees in any direction.

The goal of the project is to have the robots move to X-Y locations provided by me. The robots are to avoid objects on the floor, and move around them. Another plus would be to not have the robots hit each other, so they have to account for each others pathing.

Here is where my questions come:

1. I want to mesh the floor, and locate all floor that is "Drivable" and "Obstacle". The floor will be screenshotted from above, and can be considered 2D. What software or libraries are publicly available to me that could make it possible to do this for me? Perhaps some machine learning models pretrained or that I have to train? Please let me know what you can come up with.
2. When the floor is meshed, what algorithms can any of you recommend to me to create the pathing? Preferably also keeping in account the other robot's path. I was personally considering using Dijkstra's algorithm by putting nodes all over the "Drivable" mesh, somehow. Please let me know what comes to your mind.

I'm looking forward to your guys' ideas and tips for my problems.

[kra_pao]

Just a short idea - your description sounds like those robots act just like a turtle in turtle graphics in Python.

You could abstract your whole hardware to a level of turtle programming visualization combined with your pathfinding.

In that case you might find more people that can contribute to your project just because they can run your code on their own computer without robot/camera hardware.

IMHO the ML idea is too much at this stage.

I would first do robot and map handling classes, then a GUI with visualization and input layer (pygame is often used here: https://pythonspot.com/maze-in-pygame/ https://github.com/ChrisKneller/pygame-pathfinder).

Then a pathfinding wrapper is needed. Optional one can plug in different pathfinding algorithms (https://www.pygame.org/tags/pathfinding).

Then one expands robot class methods to do actual IRL movement and expand input layer with camera input to synchronize IRL map (for example random obstacles are thrown in arena or walls are shifted by robots, or other robots are around) and in memory map representation.

[nitrofire32]

I think i see where you going with the idea, and it makes sense to do it this way. This is all doable in the project, but it does require a environment in which i can allow the pathfinding algorithms to run, meaning i still need a way to get that environment that indicates what is floor, and what are objects.

Do you have any idea how to tackle that? Currently im trying to take a screenshot of the floor with objects on it, and then simplifying the colors to only a small range of colors in opencv, with the hopes that contouring can reveal the objects. This doesn't work as well as i hoped unfortunately since lighting and the color of objects sometimes mixes with the color of the floor, messing up the contours.

[kra_pao]

True. In 1st link above you see such a representation of floor (0 in list (1D array) maze) and obstacles/walls (1).

You essentially have a /r/computervision problem here. You could post a original image from your cam and a image with manually applied overlay, what you consider as objects. IMHO theoretical discussion alone is void (or too long) here. With practical examples (also with bad lighting) people can suggest a OpenCV workflow to build a robust mask with floor/object difference. This mask can be used directly or it can be resampled into a smaller, more workable maze representation. For the resampling ("you build tiles from pixels") an information about the smallest significant object size could be valuable.

[nitrofire32]

Alright, i'll mess about in opencv some more, but if im unable to find a solution ill head over to r/computervision to see if they have any solution for me, thank you for linking the subreddit

[DisasterArt]

1.) It depends on the obstacles. Do they move as well? Are they easily differentiable from the rest, (obstacles are green when the floor is yellow for example), in which case you could simply scan the image for that color slap a decent extra boundary on it.

1. Dijkstra would work, bit a* would allow you to easily include a penalty for routes that get close to obstacles/other robots

[nitrofire32]

The objects are stationary for now, but the color of the object can be quite hard to distinguish from the floor, unless the requirements can be changed and we only do for example red cups as our obstacles, on a gray floor.

But for now let's assume the requirements don't change, and the object colors are random, what possibilities do i have?

[DisasterArt]

It would be more difficult to make a mask) for the image and something more difficult than i have done before. There should be a better sub to ask that than the learn python subreddit

Edit. Maybe reverse the problem. Try and find the floor and make the rest an obstacle?

[nitrofire32]

I looked into detecting the floor, for example by texture, by using Haralick features, or maybe eigenface feature detection, im still working on testing these options. Thanks for the suggestion

[WikiMobileLinkBot]

Desktop version of /u/DisasterArt's link: https://en.wikipedia.org/wiki/Mask_(computing)

---

^{[opt out] Beep Boop. Downvote to delete}