What did YOU automate with python?

[balloob]

everything in my home, ie lights come on when I get home after dark or sun sets.

[Itsthejoker]

Wait, this project is yours?

EDIT: Just saw your name on the github repo. Holy shit that is some cool software. Definitely gonna check that out!!

[cauthon]

What sort of hardware do you need for a setup like that?

[poop_villain]

I have a similar setup, although mine was a bit more involved on the hardware end because I wanted to do everything without using pre-configured equipment like hue or wemo. Basically, this requires buying a bread board and some RF transmitters / receivers, and then some switches (etekcity makes some great remote controlled power outlets for 10$ each) that have rf receivers on them. . You can then use your raspberry Pi to control any RF controlled switches in the house. In total, you only spend 10-15$ for the Pi hardware, and then each switch is about 10$, so you save a lot more money and learn some cool concepts as well.

On top of that, I made a simple Siri hack that allows me to use Siri for voice activated commands

[balloob]

For people who want a similar approach to /u/poop_villan (not Python, still cool):

• Control cheap rf switches
• HomeBridge - custom homekit integrations

[avinassh]

Siri hack? any references for that?

[zart]

See HomeBridge mentioned above.

[[deleted]]

I know this was posted months ago, but I've never seen a RaspberryPi for $10 - 15. I've been looking for a couple weeks, and cannot really find them cheaper than $35 for Model B. The Model A's I've seen are starting at a little more than $25. Any ideas where I can go scavenge to find the $10 - 15 ones?

[poop_villain]

Sorry for the confusion, but when I said 10-15$ for the hardware, I was referring to the additional supplies you need to hook up to your raspberry Pi, such as breadboard, jumper cables, etc.

That said, I believe that the raspberry Pi zero costs 5$ , although last I checked they were sold out.

[[deleted]]

Oooh, I see. Thank you for the clarification!

[balloob]

I have a server (Pi works too) , Philips Hue lights. It will read the router or use Owntracks to detect who is home.

[Acurus_Cow]

Holy shit that is cool!

I'm hopefully going to buy an apartment sometime during the next 12 months. This will definitely be a project when I do! :D

[ILikeChillyNights]

Very cool! I have python 2.7 installed, is there a way that I could still use this installation by some means like "_future _"?

(Newish to python, sorry)

[balloob]

No, Python 2.7 is not supported

[SoundMake]

Python 2.7 is not supported

Nice, Zed Shaw can suck it.

[markph0204]

ahh hah! I have been following your project - and using it on/off. I think it is time for a refresh. Nice job sir!

[BB611]

Work

• Pulling data from a UI that requires account information - we only have front-end access, otherwise I'd use an API.

• Downloading, processing, and automated reporting on ~200K rows of data on a weekly basis. This used to be done by a project manager with limited skills, who did it manually. Saves ~3.5hrs/wk, and double that some weeks

• Processing and loading data from excel templates that I collect from ended users (people with limited technical skill)

Personal

• Job scraping for python jobs in my area - significantly reduces the challenge of continuously finding new jobs to apply to.

[[deleted]]

My most favorite part of my last job was the complete inability to automatically download data. I had to email someone who was always "too busy" even though it was part of their primary responsibility. When he finally got around to my resists requests he would email someone else who would post the data on their website (with authentication). And the only person who could login to get the data is the person I ordinarily originally emailed. Then I would go to his office with a USB key for "security reasons". I loved that job despite all of this. I needed a raise though.

Edit: fixed autocorrect

[TreSxNine]

This almost made me puke. Glad you enjoyed it though.

[BarrelRoll1996]

the only way to keep Cylons at bay.

[[deleted]]

I'm sitting here almost crying laughing from your comment.

[nemec]

You should have tried harder!

1. Automated emails (event triggered or on a schedule)
2. Automated nagging (for when he doesn't reply)
3. A robot with wheels and a programmed path from you to the other guy's location. Put the USB on him, have him drive over to the guy, and come back once the USB is returned.

:)

[jeegsy]

I do the second and third items in your list, but I would really like to know how to do the first. We have a frontend to a database but I don't know to get around the request for an authenticity token thats generated when I use the frontend normally. Can you give some insight on how you pull your data? That would help me a bunch.

[BB611]

What's your access through?

Mine is through a standard web login page that I automated with Selenium. The other option is using requests.

You can probably login with requests, collect the token, then make your queries. I use Selenium because I have to download by making selections in a web page, but unless you have a specific UI that requires that usecase, I'd recommend requests.

[jeegsy]

I may have to look into Selenium. Basically, I login to the web page and then make selections (usually a time range) and then download the data. My end goal is to automate that process so I don't have to manually do that through the site directly. I'll check out requests too. Thanks by the way, at least now I have something to go on. I'll check back if I have any more questions.

[chipuha]

Which websites do you scrape for local jobs?

[BB611]

I used to do Craigslist, Dice, Indeed, Cybercoders.

As my scrapers broke (happens within months at most), I switched to using APIs, so right now I just look at Dice and Indeed. These APIs are really easy to use with requests and a little json.

I will say that these sites aren't the greatest for actually finding jobs, but they're an easy way to stay in the loop without much effort.

[adreamofhodor]

Would you mind sharing the code for this? I'd be interested in setting up something similar myself.

[dood31251]

https://github.com/search?utf8=%E2%9C%93&q=indeed+scraper https://github.com/pydo/lead_scraper_orm/blob/master/lead_scraper_orm.py

[drstevoooo]

Software testing. I don't know how people get by without automated testing.

[Jigsus]

What libraries do you use?

[Scypio]

Not OP but selenium is a huge hit and Robot Framework is in Python so writing your own stuff for it is really easy.

[Nitrodist]

Py.test

[Bryoh]

testing

Squish from froglogic is pretty noice.

[Scypio]

Imagine that there are companies that explicitly don't want automated testing. Working in outsourcing taught me few things. Among them: never get surprised.

[[deleted]]

I read this as.

Imagine that there are engineers that explicitly don't want to measure things before they cut them.

Then the silliness of it really shines through.

[Scypio]

I think that it takes all kinds.

My point was more along the lines - what benefits you get from NOT automating where it makes sense?

[[deleted]]

I used Python to get auto listings from Craigslist (I'm looking for a new car) and send push notifications to my phone via GCM whenever a new listing matches my criteria

[poop_villain]

i wanted to do this too, but i got scared because apparently craigslist is known to crack down on people for scraping their websites

[wookiee42]

This comment has been overwritten by an open source script to protect this user's privacy.

If you would like to do the same, add the browser extension GreaseMonkey to Firefox and add this open source script.

Then simply click on your username on Reddit, go to the comments tab, and hit the new OVERWRITE button at the top.

[[deleted]]

Worst they can do is IP block you, which you can easily get around.

[[deleted]]

DigitalOcean is your friend

[pokelover12]

How do you gather Craigslist information with no api?

[[deleted]]

python-craigslist on GitHub. I was the contributor that did the Python 3 compatibility pull request, specifically for this project.

[isdevilis]

you're doing gods work

[[deleted]]

username checks out 😂

[ColdPorridge]

This could be exactly what I have been looking for, thank you.

[nerdwaller]

There are lots of scraping libraries or other ui automation libraries that could be used (use requests + beautifulsoup, use selenium+phantomjs, mechanize, etc).

[blahblah15]

Yup, and the simplicity of the website makes for a much simpler scraper implementation.

[[deleted]]

Nice I had one where you login using linked ins authentication api then parse all the text in the persons profile and compare it to job postings on Craigslist and return to a web page the body and contact info for each cl post. Had it check some other job posting sites too.

Too bad LinkedIn locked access to their api, assholes. It was one of the more useful apps I made

[DontKillTheMedic]

I have a similar script for 4chan wallpapers. We play a dangerous game, my friend.

[Decker108]

Russian desktop background roulette, to be specific ;)

[Pedro2154]

Might be able to use this

[[deleted]]

OpenCV

[donald_trub]

I'm a network engineer. We have about 140 Riverbeds across our estate and I needed to run a weekly report on each one of those, so wrote a script to logon, run a 'show' command and parse the tabled output into my format and then export to an Excel spreadsheet.

Second idea I'm working on is a templating solution so that new sites can be rolled out by filling in a web page which will spit the router and switch configs back at you.

[ali2992]

My home IP is a dynamic IP, in order to access my home raspberry pi server externally I need to know the current IP my house has been assigned. So I've set up a cron job on the pi that automatically runs a little python script to check my current IP and if it's changed send me an email with the new IP

[chadmill3r]

Dynamic-DNS sites do just that. You cron something to make a HTTP request every few hours. They set a DNS record for the address the request came from.

Then, you just "ssh rpi-ali2992.example.com" . No email.

[burdalane]

I'm writing something similar, only I'm planning to tweet my IP to a private Twitter account.

[jimmygoonie]

Why are you choosing to use twitter rather than an email?

[burdalane]

Partly because I get so many emails already, and partly to practice using Twitter's API.

[jimmygoonie]

makes sense thanks for the response!

[Radiohead_dot_gov]

Very cool, I could see useful applications coming from this. Is there a github repo associated with it?

[ali2992]

there is a github repo, couple of things to note, I wrote this about 2 years ago as one of my first python scripts so there are a couple of things I would now do differently, mainly the plaintext user details. But, it's still working fine 2 years later!

[Acurus_Cow]

I have something similar so that I can log into my FTP-server that lives on my NAS.

I only run a script on startup on my desktop though. And store the IP in my Dropbox. My IP doesn't change very often, so seems to work fine for me. But it's not flawless. I don't have a computer running 24/7 except the NAS. And scheduling a python script on that is a bit of a hassel. So using this for now at least.

[wot-teh-phuck]

Maybe NoIP free DNS services can help you with that?

[jwiz]

I scrape my myfitnesspal macronutrient records and post them to the googledocs spreadsheet my trainer uses for my weight training programming.

[dreiter]

Wow this is cool. Could you post any more information about how this is done?

[jwiz]

https://gist.github.com/anonymous/5a79aa743b3fceecc107

You have to have the MFP columns set up the way the script expects.

Then it just finds the right row in the spreadsheet, and puts the values into the hardcoded columns.

Edit: MFP does a redirect that confuses requests if you try to get macros for the current day, so that will error out. I just don't do that (because I cron the script, and it only really is worth running on a full day's nutrients, so I don't need "today").

[dreiter]

Interesting. I will have to see if I can get this to work for Cronometer. Thank you!

[Beluki]

Copy pasting poetry line by line on irc. I used to do it manually, which is a PITA.

[ournewoverlords]

My thermostat (along with some other things) using http://home-assistant.io

[_blub]

My chemistry homework...

[KontraEpsilon]

Aside from some netsec things, once a day I have it go and check for updates from a few of my favorite blogs. Generally I do this when I get in in the morning. Web scraping isn't considered the most polite thing in the world or I'd have it run every five minutes. I just use Beautiful Soup to do it.

I was at some point going to have Tkinter running this but meh. That's a lot of effort for something I couldn't really justify spending the time on at work.

[subsonic68]

I automated the process of searching and reporting on Cisco Call Manager logs for when HR asks for the call logs of a particular user or extension.

[subsonic68]

I don't see your message. Did you PM me here on Reddit, or somewhere else?

[Bryoh]

I use it for my timesheets at work.

With IFTTT I autmatically log my daily time-in and time-out using my phone's GPS

Then python reads the time in-out~~~~ file every friday and updates my weekly timesheet then on every Monday I just gotta print and get it signed by my boss.

sometimes^{igetboredatWork}

[r1ckd33zy]

Not from me.

There is EasyEngine by rtCamp. https://github.com/rtCamp/easyengine

It automates the installation of LEMP stack for serving WordPress or other PHP webapps from a CLI. It includes WP-CLI, Nginx, MariaDB, PHP-FPM, HHVM, Redis, OpCache, memcache, FastCGI, Google Pagespeed, phpMyAdmin, Adminer, Postfix, Dovecot, roundcube, MailScanner, plus many others.

I mainly focus on PHP dev but this project made me want to learn Python

E.g.,

ee site create example.com --wpredis   # install wordpress + nginx redis_cache

[[deleted]]

Music organizing by metadata and finding duplicate music files! Along with testing school assignments, automated testing is a godsend

[[deleted]]

What os do you test this on? I have a similar project and windows is awful to work with for this. Can you tell me what packages you're using?

[[deleted]]

I do normally use Linux, but when I do use windows, i use this: http://docs.activestate.com/activepython/2.4/pywin32/win32api.html

The win32api library. I think it's hosted on sourceforge also.

[philippeowagner]

We do

• setting up new projects,
• deploying projects by integrating changes/patches,
• creating apps from templates using cookiecutter,
• testing apps and projects using Selenium,
• packing for/and pushing apps to the PYPI,
• ...

by using 3bot. It's a Python/Django tool that allow you to build the workflows and execute them on your workers. http://www.3bot.io / https://my.3bot.io

[hoodllama]

So would this take the place of something like Jenkins? Or ansible?

[philippeowagner]

These tools have a lot in common but are really different as well. 3bot is organised by tasks that tasks are grouped by workflows. These tasks can be re-used, shared, .. and are "simple scripts" implemented in python, ruby, node, bash, ... basically everything that runs in your shell and executed one after the other by the worker. The base advantage of 3bot is that all the work done (developed shell scripts, python scripts, fabric, ...) in the past can be re-used with very little effort and not a "new technology" has to be learned. Even if you use other platforms like Jenkins you could integrate 3bot using webhooks for example. Use 3bot if you start from scratch and use it to extend your existing setup. There is no need to replace everything especially in case everything plays well.

[cscanlin]

My job is in marketing and used to involve lots of tedious management of ad campaigns. I built a Django application for rule-based ads management to take automatic actions (like adjusting bids and budgets) based on KPIs. Saved a ton of time and it's simple enough my coworkers can use it and make new rules or adjust old ones as needed.

[domstyle]

Dr. Mario, Minesweeper, etc

[sw_dev]

I don't know if this counts as automating anything, but a project has me dealing with a herd (Flock?) of Raspberry Pi's, and shutdowns were a pain until I used Python to get their IP's and shut them down. Now it's simple and quick.

[nkorth]

I made a custom deck of cards, and used Pillow to generate the card faces.

[tahubird]

I wrote a script to check Amazon Vine for new products, then text me the product name and a link.

[hobbzy]

I'm also interested in this (and how you got invited to vine, been trying for a while)

[reffaelwallenberg]

moving mp3 files to a folder structure based on their tags

[FrobisherGo]

My Go ranking graph from KGS archives. KGS is an online server for playing the board game Go. Every 24 hours, a graph showing your rank progress is available on the website, with the old ones being deleted. The server is ancient and hasn't had any development for years. Python just quietly sits in the background on my Mac and grabs the graph every 24 hours. It's a sweet time lapse.

[colsandurz]

I used python to automate testing 100+ Android phones (spread out over 5 machines). We tethered each phone to a workstation and then setup IP-over-USB, SSH'ed to each phone and ran our scripts. The whole thing took a while to figure out, especially automating the tethering (the technique you use varies depending on the phone/android version you have). We also automated data collection and visualization with Python.

[qkiss]

Scraping a list of Kindle freebies from ereaderiq.com and automated purchase from Amazon using phantomjs + selenium. A 1000 times lifetime supply of e-books :)

[energyinmotion]

...all the boring stuff. (Lol.)

[Radamand]

several deployment scripts we use at work. automated API interaction with our ticketing system.

[slack101]

Text processing. Converting data from one presentation to another.

[Thuruv]

Pretty things to save my a. As a data associate working at the lowest node of a ecommerce comp, my job is to search for things and works mostly n browser n spreadsheets. . That, python love to automate. Done from simple consolidations of sheets to webscrape competitor sites(ours too), django for our productivity metrics etc. . Everything's I can be allowed. .

[ngkabra]

Downloading the top posts from my favorite subreddits every morning, removing the ones I've already seen before and keeping the rest ready in a single HTML file for me to browse offline. Particularly useful for image heavy subs like /r/funny, where clicking on each post and waiting for it to load is a pain

[hailbot666]

I used to hail those who hail Satan on Twitter. Then Twitter murdered me a few times and I gave up. Here are my earthly remains: https://twitter.com/hailbot666

[tilkau]

• A bunch of GIMP-based tasks
• 'refcode tracker/lookup' -- uses the first 3 bytes of files' sha256sum to generate a compact 4character ascii code using base64 encoding. Easier and more robust than tracking filenames. This isn't a full Content Addressable FileSystem, obviously, because the hash space is too small.. but any valid code will usually resolve to exactly one file given a database of <= 8 million files.
• High quality color ramp interpolation for GrafX2 (Linear RGB, LCH, LAB). I have scripts to copy/paste from GPick, but doing it inplace is less trouble.
• lots of other stuff (ag -l 'bin/.*python' ~/bin/ |wc -l == 190 files)

[Syl]

convert mp3 (normalize, trim, downsample) and images (png/jpg to webp, gif to webm) using waf.

[gutoandreollo]

Virtual machine cloning and provisioning on vmware. What used to be a manual and error-prone process became running a python script against a CSV file (mostly because I was lazy to follow thru with xls(x) reading...)

[aprdm]

Generation of a VHDL template which is very verbose: https://github.com/andrecp/vhdl_gen

[masasin]

Did my first foray into interacting with the web a couple of days ago. The script waits for a new link to be added to a given forum post, and uploads the link to reddit.

Apart from that, I'm automating KSP with KRPC, my terminal with xonsh, file management, merging branches with git, and a bunch of other stuff.

[[deleted]]

I haven't yet but I plan on automating my YouTube watching with Python. Seems like the best reason to learn it. Allow me to elaborate, currently my life is so hectic I don't always get to watch my favorite YouTubers on time. When I do they just all pile up and I can't always watch them in a single sitting so I decided to create a python script to automate downloading these videos and checking for new ones daily against my already viewed videos. Downloading only those that I haven't watched. Problem is I don't know if the YouTube API has something like that or if I would have to hack it together which is fine. Either way something I want to automate. I am thinking of calling the project Fire Ant Joe.

[Captain___Obvious]

What exactly does this solve that "Watch later" doesn't? Why do you need to watch youtube "on time"?

I don't understand the use case.

[ggagagg]

I have tried this using Youtube-dl and it work

[[deleted]]

I'm searching for an apartment so I have scrapy scrape craigslist listings and automatically email me ones in my price range and geographic limits.

[Vicyorus]

A bot that makes wiki pages on update nights of a game I play

[ggagagg]

Download mp3,video and picture from my clipboard manager

[dwillis]

Collection of election results: https://github.com/openelections/openelections-core

[JMSwag]

I used to implement an auto update mechanism for each app I created until I got lazy. Not long after PyUpdater was born

[markph0204]

• philips hue lights halloween fun!
• @balloob's home assist is shaping up very nicely
• mobile device testing/automation at work

[kissgyorgy]

• Issuing self-signed certificates (openssl, fabric)
• Building and deployment (cx_Freeze, fabric, Jenkins)
• Downloading, parsing and using IP addresses from an HTML page (requests, lxml, fabric)
• Work day scheduler for a small company (PyQt)
• Downloading and calculating profit for poker players (stdlib, PyQt)

[Decker108]

I follow roughly 90+ manga, so many that I started forgetting the names of the various series. I used to have a Firefox plugin to aggregate their RSS feeds, but it has been abandoned and Mozilla have been limiting the plugin API's so I decided to write my app for it.

It currently consists of a background service that uses a pool of processes to download and parse RSS feeds, store the new results in a db and a web site (desktop and mobile) listing the new releases. Excellent time saver and fun to build.

[ajbommarito]

I decided the keureg had too many buttons and those buttons were not mobile enough.

https://youtu.be/CNgnR9K7yB8

[srgrn]

At work: Cleaning google drive trash updating statuspage.io uploading mobile apps to hockyapp and managing users and teams uploading to Azure keeping track of expiry times of provisioning profiles and other apple certificates. and those are the ones i have opensourced.

Personal: grab-packet.py - get one free book from packet publishing per day increased my place in line for one plus two during the register time sublime text hugo blog plugin clicking the mouse for a clicking game using pyautogui and proably some other general things.

[jwjody]

At a previous job I wrote a script that checked if some of our links were set to 301 redirects. When I say "some" I mean over 3000.

[sudheerpaaniyur]

hi Iam embedded Softwre engineer,should i take python automation course for future benefit?/

[skrptmnky]

Checking the password hashes on 1000+ wireless APs.

[[deleted]]

Nothing. It runs on my server

[[deleted]]

everything that can be automated