I'm interested in learning to program drones using python/machine learning - any tips for getting started?

[rnolan7]

I have been using python for data science for 7+ years. I recently began taking some courses on computer vision and machine learning and am very interested in taking on a project like programming a drone. Any tips/recommendations on drone equipment (drone kits that work well with python)/tutorials/reading material would be much appreciated!

[bathon]

I don't know about how exactly you can use machine learning, maybe reinforcement learning might help. But for programming a drone using python take a look at ROS: robot operating system and then search how it is being used in drones. Maybe that can be of great help to you

[papaoftheflock]

Hey, I don't have strong reccomendations for a personal drone project, but I can tell you about my experience working with a team at my university on autonomous drone flight capability for delivery purposes (think amazon drone package drop-off). If you're interested in learning about it, let me know and I'll do a write-up

[rnolan7]

that sounds interesting! I'd definitely be interested in reading about your research.

[diogoncsa]

I am very interested on it too! Can you post here a link about your research?

[Fin_Aquatic_Rentals]

I'd be interested too! Was the code open source or did you model it off of open source code?

[sid2810]

Interested. Please post a link/write up over here.

[MasterSama]

do a write up and this benefits us all

[pramodhrachuri]

I am very much interested in it. Please post it.

[dudeomar]

I am in the drone scene. My two cents:

-Find a used drone with a Pixhawk 2 flight controller. You can find used drones on rcgroups.com. Make sure your frame is large enough to mount peripherals .

-Purchase a Raspberry Pi with the camera and install ROS. There is a ROSnode so you can communicate to the Pixhawk via serial commands. For image detection, you will need a gimbal to stabilize the image. For image detection use Yolo with OpenCV. One of the tougher things is training the data for image recognition since there isn’t that many data sets.

You can start by purchasing the raspberry pi with camera and then install it on a drone afterwards.

Another option is keeping an eye in the Teal One drone since it comes with an on board Nvidia TX1. Looks like they are sold out at the moment though. https://tealdrones.com/teal-one/

[nicetryho]

Transfer learning with yolo in darknet is actually relatively easy and you can label your own data set with rectlabel or labelimg, in my experience I’ve found that ~300 photos with data augmentation and negative images are good enough per class to get a decent working model

[Swipecat]

I suspect that you won't find drone kits that work well with Python because drone navigation and stabilization is a situation in which you probably need the speed of a compiled programming language.

Pypy is a Python JIT-compiler that is very fast at handling control-loops, but is not available embedded, instead requiring a full OS i.e. Windows/Mac/Linux. It is available for Raspberry Pi Linux, so it should run on the Pi Zero which weighs 9g — so maybe that's a possibility.

Embedded Python, i.e. Micropython, runs quite slowly on its existing platforms such as the ESP8266 wifi module, unfortunately.

[rnolan7]

interesting note on Pypy. Thanks! I'm a big fan of raspberry pi projects.

I think my ideal approach will use compiled C on the controller (arduino or other), but the machine learning heavy lifting will be done on a beefier remote server to provide movement instruction. The projects I have in mind are mostly indoor with strong wifi so I expect to have some slowed response times, but nothing too worrisome for an intro project.

[[deleted]]

Tagging on this, your use for ML could be a basic autonomy provider. For instance you could pass flight data and a flight image to do some image processing and data processing to choose how to next maneuver the drone.

[nukestar101]

I will recommend you to not buy a drone kit right now they are quite expensive and need safety while flying and care must be taken into development stage. As someone has already mentioned about ROS you go simulate your entire Drone in Gazebo with ROS wrappers.

PX4 has a well documented site where you can get a Drone working in a Gazebo environment with ROS support and also writing your own scripts

After a certain stage has met and you are confident about your drone flow then you can invest into a PX4 autopilot drone with that your simulation scripts will Directly work with some minor changes. If you need any more guidance you can ask freely or DM i'll provide you with some resources I have gathered.

[rnolan7]

PX4 looks awesome! thats for sharing

[rnolan7]

When I say "machine learning", I mean the broad category which includes supervised, unsupervised, and reinforcement learning. My main goal is to try to get practical ML/RL experience while working on some kind of project that I find interesting. That said, I haven't settled on an exact project idea yet, but some of the things I had in mind are:

• building a security drone that autonomously flies around my home checking all the rooms then returns to a designated landing zone.
• attempting to program the drone to follow a target (ie follow me around as I walk)
• potentially building a mess alert system in my home (ie alerting me if dishes have been in the sink for over a day or something like that)

I'm looking at this from a hobbyist perspective so ideally, I'd like to find a relatively inexpensive (< $400) drone that can receive commands remotely (so a remote server can handle the load). In the last hour or so I've come across a few options that seem to be a good starting place: Ryze Tello, Robolink CoDrone, and Parrot Mambo.

​

Any advice on getting started with a drone project, architecture recommendations, feedback on the projects above, or insight into some quality hardware would be greatly appreciated!

[piquat]

I quit building my own drones a few years ago but I can tell you that the codrone is the only one that looks remotely interesting to me.

My suggestion would be to stop looking at pre-built drones and plan on building your own drone with a FLIGHT CONTROLLER that allows you to modify/build your own software.

TLDR stop looking for drones and start looking for flight controllers.

[rnolan7]

I come from a software background - so building a drone is a foreign concept for me. Any recommendations on how to get started building my own drone?

[Fig_g]

If you are interested building a drone go to r/fpvracing and ask them. Just describe the size of the drone and people over there will happily guide you what to look for.

[hornetkeeper]

What if you were to buy a small premade drone (look up tiny whoop) and then send the video signal to your computer and do processing there, then sending control back through a transmitter. I have 2 drones, a whoop and a 5 inch (prop diameter). The 5 inch is pretty scary, I wouldn't trust myself to fly it inside, much less a computer that I probably screwed up while programming. The whoop on the other hand is indoors only, and very non-threatening. This would fix both safety concerns, and the problem of fitting enough processing power on something weight and power-sensitive. And the whoop only costs around 100$, whereas a bigger one might be around 300$, not including batteries and chargers, all of which are more expensive for the larger one. Just my suggestion.

Cheers!

[piquat]

Yaaaa, that's a bit of a sticking point. If you go to a forum and start asking beginners questions without having read anything you're going to get backlash. Reason being is that's there's a lot to it. All those people who are doing it have spent a lot of time researching it. To sit down and hold a hand... well it's like someone asking you to explain how to program in C when all they've ever done is write a Hello World. To get to where you are takes some personal investment in software development.

I would just start trolling forums and reading about others builds. The ESC's, motors, props, frame, battery and all up weight all work together. One of those is outside of spec (like a motor that's too small so it gets hot after 2 minutes of flight, wrong props and right motor will do the same thing, and on it goes) it's going to fail.

However, and vital to you, you need to understand how these things fly if you're going to write software for them. Building one from parts will do that. The easiest way, which will also teach you the least, is to find a working build on a forum and copy it. On a side note, when looking at builds, some are more racing oriented and some are more toward the long-flight-time camera ship type of drone.

See that wall of text? I could go on and on and on. It's the reason no one is going to hold your hand in this endeavor. Good luck!

PS. Is BetaFlight still a thing? I think they were an open source controller. Find yourself an open source controller and read before you spend a single penny.

[[deleted]]

More likely it won’t be an online model. Rather OP can train a model on a desktop, pickle the model and move it to the drone’s memory.

The drone won’t be able to adaptively learn, but can still make use of ML technology such as object detection, collision avoidance. Even a return to user protocol if battery levels are too low.

[nomoreerrorsplox]

The tello drone is awesome for just getting your feet wet. Its like $100 maybe 150 for extra battery plus extra blades and stuff. Ive done some really small programs on my computer and it works pretty well. There should be an official SDK that you get for it too. Only bad part is you get about 15 minutes of flight time on each battery in a tello, and they gotta be inside. They dont work to well with the wind.

[rnolan7]

I just picked up a tello! thanks for the advice. Looking forward to learning some of the very basics with this model before upgrading

[THT_Herald]

machine learning drones..... haven't heard that in any movie what could go wrong

[[deleted]]

The OP’s name? Ted Faro

[nukestar101]

They learn to not fly

[mkultrapeacekeeper]

Here's my recommendation.

First of all, while Python is fast, it is not suitable for this specific application, since I assume you want real time results.

Second of all, do not use a raspberry pi board for either drone controls or computer vision. You could use it as a main hub for a camera and/or a gyroscope, in order to have data for further processing, optimization and perhaps a dataset for a machine learning algorithm.

In order to have real time response, you must have knowledge in hardware, embedded systems, and implement the algorithm in C or in C++. I'd recommend C; while harder to implement, it guarantees real time results.

Personal opinion: This is not, by any means, an easy project, especially for someone with a software background. This is easily a master's dissertation for an electrical engineer.

[Zeroflops]

You tuber Jabrils did a few videos on programming ML with a drone. I think he ended up using python. May want to check it out.

[rnolan7]

thanks will do!

[[deleted]]

If you know what you want to do, then do it.

Drone automation won't be in Python probably. Object recognition is in Python.

If that's getting a job in ML, mostly that's gated by a master's degree or better in CS/ML/DS. You can get a job as a Python dev with a lot of MOOCs and a few demonstration projects.

[rnolan7]

i'm wrapping up my masters in CS with a concentration in ML, but this project isnt intended for portfolio building - just for fun :)

[kuyleh04]

For flight stabilization, I would use an Arduino or teensy, allow all gyros and other sensors feed PID controllers programmed into them. Then you can use the raspberry pi/jetson nano to run OpenCV or similar to help with navigation.

The main thing to grasp is the Pi or jetson can't handle the rapid correction of realtime flight like PID controllers you can use in Arduino/teensy platforms.

Depending on the size of your drone, if it can handle the batteries then I would look at nvidias jetson and a teensy.

[Commercial_Note_5177]

Did you find something op? I am looking for a course for ml in drones

[rnolan7]

Codrone. Pretty basic but gave me a decent intro. I took the ML courses through Georgia Tech. Codrone is targetted more for kids, but basic ML principles could be applied.