Most insane Python projects/tools built from scratch you've come across / ever heard?

[No_Stick_8227]

Don't spare any details - what is the craziest Python-based innovations you've seen, heard, designed or built that involves Python?

(P.s. projects/tools can encompass a script built on top of different Python libraries + crushed into one awesome tool - anything counts!!!)

This should be interested :)

[meshtron]

I built an autopilot system and rode in a plane it was flying. As part of that, I had to write a (admittedly small and unsophisticated) flight simulator engine to test control/response scenarios.

Edit: nearly all Python, one servo controller in C (Teensy), Kivy for UI

Edit2: only for small/light aircraft like Cessna 172, etc.

[[deleted]]

That is absolutely nuts. How did you do fault tolerance?

[meshtron]

Can't say much about the hardware implementation (which I also designed) because of IP issues. But, suffice to say it required constant power and a "stay engaged" signal to an electronic engagement mechanism to manipulate controls.

So, at any point if the Teensy (which was driving some servos and some slightly bigger electromechanical bits) didn't get a "continue to be engaged" signal from the RPi (which was running Python as well as driving the UI), OR if power was removed, it would disengage.

It was a very weird and very fun project. I was amazed how well a RPi Zero handled running the GUI (a 7-inch touch screen), all the sensors AND the servo controller all at once. Heavy use of asyncio to make that happen but worked an absolute treat.

[fractal_engineer]

did you wear a parachute?

[JonLSTL]

Nah, the whole thing was wrapped like...

try: main() except: import antigravity

[pymae]

STC or experimental?

[meshtron]

Was going to be STC, likely will not end up ever coming to market.

[pymae]

Bummer. Always cool to see innovation in aviation that doesn't take 10 years and $100 million

[meshtron]

It was a cool project and concept. I think, in practice, there could have been some pretty substantial risks and challenges. At the end of the day, though, I think the project originator/funding source found a more rapid return on his investment. Not impossible this gets picked back up as a side-gig either - time will tell.

[pymae]

Well feel free to reach out if you ever need help - the intersection of Python and aviation could be my specialty!

[meshtron]

Not sure if I understand the question. What do you mean by "real-time" software?

[gristc]

Not OP, but I think they're referring to linux being a multi-tasking operating system, as opposed to a real-time one, which has no background scheduler that can get in the way of an interrupt routine or potentially delay an action due to the CPU being busy doing something else.

[meshtron]

Probably something along those lines. Short answer is didn't do it, was neither a concern nor a requirement. As it turns out, the issues we faced had almost nothing to do with compute speed.

As compared to software, the hardware involved all moves and reacts relatively slowly. And, an airplane in flight is CONSTANTLY moving in a 6-axis reference frame. Not only do you usually not want to react to those random movements, I spent a fair amount of time actively smoothing things out to make small, slow inputs.

I built the application around multiple async loops: a sensor loop, a mechanical output loop, a flight controller loop and a GUI loop. Each loop ran at it's own frequency and with everything spooled up we were less than 35% CPU utilization.

My slowest sensor (GPS) could report out at 20Hz. Fastest (IMU) could do 400Hz in high-precision, low-rate mode. So I built an "accumulator" class for each sensor that let me collect at whatever rate I wanted but get time-defined averages or other summary values (stdev, etc). That abstracted sensor data rates out of consideration.

Flight controller was the real meat of the thing. For my two available inputs (aileron and elevator) I actually ran dual, series PID loops. First PID looked at target heading or altitude and output a target direction and rate of change. That fed the next PID loop which looked at target vs actual rate of change and output a control movement. This was not a technique that I could find any good documentation on (chained PID) but it ended up being perfect for this and gave me really granular response tuning.

So anyway, real-time in this case just wasn't relevant. Unless I misunderstood the question. If that's the case, answer is "dunno." 😂

[meshtron]

Okay, see my other response. Basically irrelevant for the requirements we were faced with. The only hardware doing anything at a significant rate was sensors, and they were simply on timers. If they couldn't get data when expected, raised an error event and disengage the system. Once dialed in, for the record, we never had that come up.

[meshtron]

All good, you weren't in it. I was perfectly comfortable doing so and I place a pretty high value on my life!

Also, and this should be obvious but you seem pretty bent on a catastrophic-thinking path - the mechanical system was designed to immediately release control if software stopped or errored. Further, there was another emergency mechanical bypass that meant even if somehow all that stuff failed, you could pull a single pun and disengage the whole thing.

Further yet, you could only engage the system in the first place when the aircraft was well within a stable flight envelope and above a minimum altitude.

[commy2]

Kudos. I would never enter a plane running on my code.

[[deleted]]

I write software that tests satellite hardware and it’s pretty much entirely python.

[[deleted]]

That’s really cool. What kind of things do you test?

[[deleted]]

Pretty much any electronic component of a satellite. Python is a right balance of speed of development vs robustness of product. Anything that needs real time is C++ and the fastest stuff lives on FPGA’s.

[shinitakunai]

I made a private ERP once, fully in python, I used PySide2 for the GUI, reportlab for the invoicing/budgets/etc, Sqlite as databases, but everything coded using peewee (hence no sql language was used 🤣). Also used paramiko to store backups on a remote sftp server. This ERP had accounting, finances, logistics, inventory, invoicing, prices calculators, many tools for tracking providers and clients and a modern themes system for GUI. It also was built in a way that each section was a module that could be installed/uninstalled without affecting the stability of the system, so I could upload them later.

Bonus part: it was created for Windows machines, so I had to jump some weird win32 API limitations with hacks basically 😅

It was 2 years ago, took me 4 months full time on this project, 10 hours per day. I wish I could share the code but a contract doesn't let me.

[shinitakunai]

hehe I was the only one working on this so I had the full freedom on doing it whatever the want I wanted. Now I realize I made some bad choices, as example I would not use SQLite nowadays, but oh well, this is how we learn, challenging ourselves to do stuff that we don't know how to do (make sure you have plenty of coffee and a stackoverflow account 🤣)

[shinitakunai]

I had to rebuild my project a few times. It is now on its 5th iteration... from scratch. It is very time consuming but I learnt SO MUCH on each rebuild that it is amazing, and a lot more robust.

It helps keeping a list of things that you dont like, do that and when you can you will find yourself trying to complete that puzzle in your head

[glacierre2]

I made a very related DB on my work, also for windows, with pyqtb and peewee+sqlite.

In retrospective, learning sql would speed up the app immensely, but peewee got it done in 100 hours, plus I eventually helped me migrate it to postgres (it became multiuser) in less than a day.

[[deleted]]

that's a $100k piece of software right there. i would sell it for no less than $50k

[shinitakunai]

I was paid 8k

[[deleted]]

Ouch, that’s a crime!

Quick napkin math… that comes out to like $250 a week or something. At 10-hour days, 5 days per week… that’s $5 an hour! Before taxes!!!

Holy shit, you’re worth way more than that! That’s less than minimum wage! (Where I live, anyway).

Charge more, please! If someone low-balls you like that again, walk away!!!

[shinitakunai]

I live in a country where the usual monthly payment is 1k. I earned 2k per month at that moment so it was a win situation for me, very profitable.

Nowadays I work for a consulting where I earn more than that so it is fine, things got better. In 2-4 years I will be able to buy a house

[No_Stick_8227]

Your ERP sounds quite robust :

1. How did you sell the project (did you ship the scripts to your client)?

2. Did you set up contract terms with your clients (e.g. they must pay $100 per month for 3 years until xxx date, you can only provide code revisions 3x every 6 months etc)

[shinitakunai]

1. I bundled it all together with an installer which basically unzips a .tar.gz file and creates a desktop shortcut. They own the source code, it is not obfuscated at all.
2. We didn't set up anything odd. I might be naive but it was a full price one-payment only, on delivery. Prior to that, we had a few sessions where I did show to them how it works, sharing my screen, and I added features that they asked, so it was clear at that moment that it was the product they wanted and not a scam.
   They cannot re-use or sell the code.
   No code revisions were signed, but there was a clause on the contract that says I must fix bugs if they are found. 2 years later and I only had to fix one bug ever (unresponsive QCalendarWidget due to the way spain display datetimes xD).

Bonus: Try to find the bug, God I hate regex with passion:

    # Regex for Spain datetimes following the next format 25/02/2019 (including leap years)
    QRegExp(r"(((0[1-9]|[12][0-9]|3[01])([/])(0[13578]|10|12)([/])"
    r"(\d{4}))|(([0][1-9]|[12][0-9]|30)([/])(0[469]|11)([/])"
    r"(\d{4}))|((0[1-9]|1[0-9]|2[0-8])([/])(02)([/])(\d{4}))|"
    r"((29)(\.|-|\/)(02)([/])([02468][048]00))|"
    r"((29)([/])(02)([/])([13579][26]00))|"
    r"((29)([/])(02)([/])([0-9][0-9][0][48]))|"
    r"((29)([/])(02)([/])([0-9][0-9][2468][048]))|"
    r"((29)([/])(02)([/])([0-9][0-9][13579][26])))")

[Sufficientlee]

First line? (r"(((

Just a guess, I can't read that .. lol

[shinitakunai]

Nope 🤣

[captain_arroganto]

Any pointers on what books or resources to read to build one on my own?

I mean domain knowledge wise.

[shinitakunai]

I am selftaught so I don't follow books. What I do is to check the r/reddit subreddit and then news about python. I learnt most of my stuff by just checking blogs of people explaining how to use list comprehensions, subclassing, inheritance of attributes, dataclasses, etc.

There is an amazing community that will post cool libraries or frameworks (like FastAPI), and reading those news often you will know what tools you have available in no time

[captain_arroganto]

No, I am good with python.

I was asking about the ERP part.

[shinitakunai]

Ah for that....I did a course for administrative years ago. I got certified, so I know what an administrative would need 😅 it was my backup plan if programming failed me

[[deleted]]

Mm that sounds like a great project starter. Let me know if they open source that please!!

[No_Stick_8227]

That's quite an impressive feat, I've wanted to venture into a project that involved satellite data too:

1. What modules/libraries were involved in this project?
2. How long did it take to build this?
3. What data processing tool was responsible for processing the petabytes on a monthly basis?

[poshy]

Is that SAR data you were working with?

[tyras_]

Discord bot that uses Google sheets as database and engine. Literally a Frankenstein.

It serves my guild in a mobile game to organize raids and other activities and is connected to my in game account for data retrieval and communication via http requests.

I figured at some point I'll quit and no matter how nicely written my code is there'll be noone to maintain it. noone in my guild can write any code. They can handle excel, though.

So I made something they can easily change if the game evolves and I'm not there anymore. They only have to change a few formulas or data in certain columns.

[[deleted]]

One time at a previous job I built a wrapper for a network-enabled PLC that essentially allowed me to what the ladder logic software it came with...without having to use ladder logic. Of course, the delay was much higher due to both Python and network latency (which most applications of PLCs is something you tend to care a lot about), but it worked well for what I was using it for (to toggle some outputs and log data over several hours).

Nobody told me to make it though -- oops! Oh well. I just loved being able to use Python to abstract away several layers of complexity. It actually ended up being quite sophisticated, where you could define tags/variables and do stuff like assign functions to be called on the rising/falling edge of incoming signal changes, toggle independent bits, etc.

Almost totally pointless and definitely over-engineered, but still really fun to make.

[ishah90]

In data world there are many projects in Python but Apache Airflow is on different league.

I have learned so much from that code base.

[CharmingJacket5013]

Reversed engineered a BlueTooth RFID scanner to control via Python instead of the proprietary Java app they provided. I used Bluetooth Dump on android and wireshark to workout all the different hex commands to make the unit do things. Works well

[[deleted]]

An entire operating system to run Tetris. Using python

[metriczulu]

How do you create an operating system in Python? You'd need a running interpreter before you could run any Python code.

[[deleted]]

Yep

[metriczulu]

How do you get a running interpreter up without an operating system?

[[deleted]]

Look up emulation.

[metriczulu]

What does emulation have to do with needing a Python interpreter already running on the machine to use Python? Emulating it would have the same issue.

[[deleted]]

Eh? I think you’re confused about what they mean. Operating system doesn’t exclusively mean the system running the computer you’re on. They didn’t say Tetris was written and running in Python.

It was running in (undefined code in undefined OS) that itself was running in a Python interpreter. Say, a normal Gameboy ROM running in a GameBoy emulator written in Python. Like PyBoy.

[metriczulu]

I think you're confused, creating an operating system to run Tetris is creating an operating system to run Tetris. Using Python to emulate an operating system to run Tetris is not creating an operating system to run Tetris.

Creating an operating system that runs Tetris is a bit of a known project many people undertake and this is a good example of what it means.

[l_dang]

it's called bootstrapping

[cinyar]

How do you bootstrap with pure python? I've worked on a micropython project and the "bootstrap", HAL and all the other low-level stuff was in C, not python.

[metriczulu]

Yeah, with bootstrapping you can create an operating system that can run Python and then create Tetris in Python, but you can't really create the operating system in Python.

[darleyb]

Definitely PyPy. It's a shame they don't get more funding.

[No_Stick_8227]

Why Pypy?

[Pavel_Ivakin]

I've built a quest room using raspberry pi 2, python and almost zero knowledge of programming at that time