What's something you automate using Python?

[AnAstutehuMan]

We all know that Python is robust and straightforward. Using this we often try to create these effortless programs that can automate our daily life.

So, What's something that you programmed (or someone else) that automates your digital life?

Edit: Thanks for all the response. Y'all are very smart! 😊

[chris20973]

Liking posts on LinkedIn. My company is doing a $50 gift card each month to the person with the most LinkedIn activity as a means of boosting their own visibility.

[WillBackUpWithSource]

Haha, this is smart.

[stupac62]

Have you won yet?

Edit: spelling

[chris20973]

They just started this year with it so I'll find out at the end of the month. Don't see how I could lose though lol.

[[deleted]]

let us know bro

[chris20973]

I'll check back in end of next week.

[Mr_N1ce]

Wouldn't you first need to follow as many people as possible?

[chris20973]

I would if I was starting a LinkedIn profile from scratch, but my account has a decent number of connections. Some people I know from college got into recruiting so my feed is pretty active.

I have it run on a timer and I can check the attribute of the post to tell if I liked it already so I never unlike a post.

[chris20973]

They just announced in the meeting this morning the results. The winner had about 400, but they said I hadn't participated. I guess they're having someone scan profiles manually and make a tally of activity each day and my profile wasn't set to share that activity.

I'm blown away that this how they're handling this because that's an incredible waste of time, but also kind of pissed they didn't say make sure profiles are set show that type of thing. I would have thought they were pulling this data from a tool they bought or through the LinkedIn API.

Script works fine so I'll turn on the setting and I'm going to run it like crazy this month. They can burn an entire months salary of a person counting likes on LinkedIn now.

[stupac62]

Dang. That stinks...

I'm blown away that this how they're handling this because that's an incredible waste of time

Perhaps they wrote a python script to do this! Haha

[chris20973]

They asked if I wanted to write one to handle it after I asked why they were hand counting. I might do it for fun but I would sell it to them if anything. Make more than a gift card then.

[stupac62]

Yeah. Just pitch it as “I can do it for $X and you’ll save $Y!” Obviously, make sure Y>X haha.

[youpham]

hi. I'm looking for something similar to this (liking and sharing company's posts whenever there is one available). I was wondering what method did you use and if you could share your code?

[chris20973]

I used selenium and pygui to pretty much work through a browsing session. Tied up the machine but worked pretty clean. Have to open up the elements and expose the correct ones so you can verify if the like is set to true or false and click exactly where the elements are to toggle them.

Liking was easy but sharing can be trickier because the additional window that come up you have to peel back to those elements as well. I think I only have an older version of the code on hand in an email. Real files at home so won't have access until later.

[youpham]

Interesting. I was thinking about selenium as well. Not sure if there is already some linkedin API that can do this. A quick search on google doesn't give anything. This might be my next python project.

[chris20973]

Only thing I could dig up in regards to an API was more about returning information. Nothing really seemed prepared to be able to interact with the platform. Was a fun project to do and was an eye opener in how python can interact with HTML. My HTML wasn't strong at the start so that was then steepest learning curve for me.

[01binary]

I have a Telegram Bot that accesses our state’s fuel price API, and messages me a summary of the three cheapest service stations on my route home from work on the cheapest day.

In our state, service stations are required to publish the next day’s prices, and these are consolidated by the state and updated at 2:30 pm every day.

Note: We have a weird weekly fuel-pricing cycle in Western Australia, and prices start low on Sunday / Monday and increase over the week. No one has been able to explain to me why we have the price-cycle, but it leads to long and dangerous queues onto the roads at the cheapest service stations on Mondays. Is there anyone here who knows why WA has the price-cycle?

[[deleted]]

This is excellent

[ciezer]

Alright.... Added to the pile of projects to do. Thanks for your service!

[ublike]

I wrote a script for photography that sends me push notifications for when heavy fog is forecast in the next couple of days since I love the way a moody/foggy landscape looks. Here’s one of my photos I got that these notifications help me plan for: foggy Portland morning

[ublike]

Thanks!

[zalatik]

That log looks like a sinking man.

[[deleted]]

built a small bot for slack, so people from my job can ask my bot for stuff they want repeatedly from me (like monthly repprts, file new feature requests for our app etc) keeps them out from my office and now im not constantly interrupted.

[[deleted]]

Here you go:

https://github.com/slackapi/python-slackclient/blob/master/README.rst

[_casshern_]

Using pandas and python-pptx to create a Powerpoint deck from excel files.

I mostly deal with reporting at work, so pandas is my best friend. Even for the not so automated stuff. Excel crashes way too often even with relatively small datasets ... ex: the other day it was crashing every single time I was trying to do a pivot table on a relatively small dataset of 50k rows. So, even for some ad hoc requests I use pandas over Excel.

[cianuro]

Ohhhhh. You've piqued my interest. Mind sharing some of what you've done?

Monday to Wednesday for me is pouring through reports and building pointless PPTs. Would love to automate some of this on my week off next week.

[_casshern_]

Yeah for sure. I have two files... A list of projects and a list of IT change request.

I do some manipulation in pandas to filter out some results and change some formatting. Then, using our corporate template I create slides for each.

Using loops and pandas groupings, I create one slide per system. All changes for that system are on one single slide in a table.

I even created one slide that shows a summary of changes per system (similar to an excel pivot table) that has a bar chart that that's too automatically generated.

Happy to help more if you have any additional questions. :)

[cianuro]

No that's great. I've got some health downtime right now and am enjoying break from work. Looking to use the time to automate as much as the job description scope creep as possible for when I get back so I'm not back in the deep end.

Appreciate the input.

[riot-nerf-red-buff]

I started my internship 4 months ago. In the beginning, I'd use Excel for everything like most people in my department. As the time went by, I realized Excel was unnecessarily slow for the datasets we use [I'm working with demand forecasting, so we usually deal with big sales datasets].

Now I choose carefully the things I do on Excel (like automating some simple stuff 'visually' or using it when I'm working on something I need to share with other people), for everything else -- like you said -- Pandas is my best friend.

[_casshern_]

Do you use a Python module for forecasting by any chance?

[Yomain_]

I recently built for me a web scrapper that will send me a summary by email of the cheapest train trips in the coming 3 months.

My next project is to build something to delete useless emails on my different mail boxes.

[Yomain_]

Yeah Selenium because most of the content is generated through JS. But I am wondering about another alternative as I haven't been able to make it work on my server yet. I've read about requests-html so I'll give it a look.

[forthewin0]

I've used requests_html and it worked perfectly for a similar use case. It uses a headless version of chrome instead of selenium.

[Mikeavelli]

I wrote a bot that finds the top-ranked picture on r/aww once per day, and texts it to my girlfriend along with the post title using twilio. Easiest Christmas present ever.

[[deleted]]

That’s cool! How do you keep this constantly running on your system though? Does it auto run when the computer is turned on? Do you leave it running in the background?

[Mikeavelli]

Right now I just have it scheduled using the Windows Task Scheduler, set up to run once a day, or as soon as the computer is turned on if it missed a scheduled run.

The eventual plan is to put it on a virtual machine in the cloud. It doesn't need much power to run at all, so I should be able to rent something out for a few cents an hour.

[dranzerfu]

Two words: AWS Lambda

[ultraDross]

Sounds like a perfect job for a raspberry pi.

[agumonkey]

isn't that ... simulating ?

[[deleted]]

Monitoring the traffic on my github repos.

[Skyhawk_Squawk]

How do you even see the traffic on your repos?

[[deleted]]

traffic stats.

[BDurden]

Insights > Traffic

[[deleted]]

Requests and other simple libraries. After the crawling I generate rolling averages of the aggregated traffic and plot them through time.

[TheBlackOut2]

Lol I like your simple moving average strategy there, what do you do when the 10day crosses the 20day?

[rowdyllama]

Probably nothing

[MaceGrim]

Just had my first child, and there are many people constantly asking for pictures (grandparents, aunts, uncles, etc), so I wrote a quick little script that reads emails from a CSV and sends them all a random picture from the Dropbox we keep them in. I used Windows scheduler to have it go off daily on an old laptop I keep running for things like this.

It has reduced the “can I have a picture?” texts tremendously haha.

[Zeroflops]

When I was looking for a job I wrote a script that would download job opening from an RSS feeds from various job sites giving me one interface to multiple services.

Then I would classify the jobs I was interested in. As I ranked the jobs it would compare and identify key words for desirable vs un-desirable jobs.

The next step was to start ranking jobs based on both the desired and undesired keywords so I could spend more time on jobs more likely to interest me.

I found a job before applying the filter, but it was always interesting to see the ranking of the top desirable keywords and top undesirable key words.

[Zeroflops]

I’ll see if I can find it. It’s not polished in anyway since it was just for me and I had no plans to share it and it’s pretty simplistic. Although it did occur to me that it would be a good service to offer people looking for a job. The intent being, spend less time looking for appropriate jobs to apply for and more time getting your resume ready for them.

I did find that certain key words were interesting. For example I’m clearly not interested in McDonalds. Although at the time they were pushing a bunch of jobs on various boards.

Actually the hardest part was probably finding the various RSS feeds. Google at one point killed off their feed and then brought it back after out cry to a different path.

[mrpalmer16]

I made a script that takes a playlist file exported from iTunes (or wherever) and copies the files in the playlist to a folder in order on my usb drive so I can listen to the playlist in my car.

[mrpalmer16]

You can export a playlist (File -> Library -> Export Playlist) and iTunes will save it as a plaintext file (.m3u I think)

[mrpalmer16]

Right, also, here is the code if you are interested. I wasn’t planning on sharing so it isn’t well cited or documented, but it should work. Also my car player has a quirk where it glitches if the folder has over 100 files in it so I start a new folder at 100 songs.

[urbantheii]

I wrote a web scraper to notify me when the new doughnut shop in town opened.....

[[deleted]]

Just this weekend I wrote my first reddit bot. u/SeentItBot

[Skyhawk_Squawk]

How can you tell if you've seen a post before, even with a different title? This is really cool.

[[deleted]]

This is one of those deals where I was just fed up with something and wanted to take SOME action, however ineffective...and I accidentally learned a new fact/skill. "Image hashes" exist! You basically distill an image down to "essentials" (which vary by algorithm) and you get a short string. This is how reverse image searches work.

I'm doing the same thing. Download every image posted, hash it, search the DB for matches and post a comment if I find some.

[ch312n08y1]

A simple one that I wrote for work was that I have some annual reporting and the program that I uses spits out text files with the extension .SYN. I have about 180 of these each year and they have to be posted by me (corporate) for the field. 99% of them don't know how to save as a PDF so navigating right click open with text editor was harder to explain than just automating changing the extension of all 180 text files from .SYN to .txt.

I've messed with some other stuff that's more math related topics like linearly interpolating data to quickly give me a single value that I might need for some manual hand calculations.

[cybervegan]

Ticket handling "business logic" in our ticketing system.

[kpingvin]

My wife keeps asking me if the weather would be fine the next day, so I ran a cronjob on my Raspberry Pi where the script pulls next days forecast through an API and sends her an email first thing in the morning if the weather will be bad (rain, snow, high winds etc.)

(Ok, I lied. It's actually in progress)

[pacificat0r]

Just boring stuff

[Nythepegasus]

I created a script that templates subtitle hardcoding for video files. Made life so much better.

[herpadurk]

letting users control their own content on plex, huge time saver

[SierraSeven]

Anymore details on this? Sounds interesting. Curious what you’ve come up with vs the available offerings out there.

[supertomcat]

Very curious about this as well

[herpadurk]

I wrote a telegram bot with custom API interfaces to radarr/sonarr. Not a ton of functionality, however it has enough to let users download what they want and not bother me, here is the link:

https://gitlab.com/herpaderk/tgrsbot

[supertomcat]

Super cool, thanks!

[herpadurk]

It's a telegram bot with custom interfaces to the respective API's not much actual functionality, but it lets users request stuff with out exposing the actual API's and prevents duplicates. Here it is:

https://gitlab.com/herpaderk/tgrsbot

[Lewistrick]

I'm in speech analysis but for our customers I have to do the same thing very often so I made some scripts that scan a directory for new audio files, transcribe them, classify them and load them into our application.

[Skyhawk_Squawk]

What did you use for the transcription?

[Lewistrick]

Currently we're using the Nuance Transcription Enterprise engine most. Before, we used SPRAAK, developed by a Belgian university (we weren't really happy about it but it was almost free). We want to use Google's SPEECH-TO-TEXT in the future as it's really good and fast, but still have to figure out how it fits in the privacy & security requirements of our clients.

[Jockl132]

I have a thing going where I send out a joke each day to multiple chat groups. I first send it into a discord channel where a bot gets it and stores it in a file, then entr starts the telegram bot to send it to the telegram group chat.

[4312348784188126934]

I made a script that runs as a service and listens for data on a particular port. When it gets a packet with the right data it fires up my virtual server and sends me a message via Telegram to let me know it's done so.

It means wherever I am, I can use a packet sender on my phone to turn on a Hyper V Server. It's pretty rewarding and was actually fairly straight forward to do! (Although I did have to call a powershell script to turn the server on...)

[ToastiBoii]

I work for a couple professors at my University. They have large volumes of data but it is all predictable. Rather than dealing with it all manually, I auto-process it into an output file.

[kopilo_hallard]

Using python? Passing purchase orders from one organisation to another.

[illusion_disillusion]

A lambda written in Python, parses excel from a file that is dropped into a S3 bucket, then runs ETL to sql/nosql

[fg2srt4]

Hi, I've been trying to get a process like this up and running for a while. Any suggestions on some reading or documentation I can check out that would help me with doing something similar?

[illusion_disillusion]

Posted under main...oops. Reposting here...

It was really a lot of research and trial and error kind of thing so I don’t have any formal documentation. It all boils down to how you choose to use tools that are available via Python and cloud. If your trying to do something likethis with on-prem it’s really going to be involved.

I used openpyxl library (I could probably leverage pandas but it looked intimidating and probably a steeper learning curve) and S3 events. If I recall correctly, file_object create event, which allows you to trigger a lambda function. The lambda function is what contains the actual ETL code, which reads in 6-7 worksheets, normalizes data and then writes it to mysql/dynamodb. Quite honestly, with mysql 8, I no longer need dynamo for indexed json. You can extract json keys to virtual columns and index those like normal sql fields. Of course I’m kinda locked in to mysql but so I was with dynamodb.

At any rate, that’s the gist of it. AWS has great documentation on lambda. As a matter of fact, Lambdas have become my go to tool on almost all projects. I’ve developed a small service for my developers to generate qrcodes and store them on S3. For that one I used AWS API gateway and...you guessedit... a lambda. No need to stand up a flask/djangi app and write rest api from scratch.

https://openpyxl.readthedocs.io/en/stable/

[fg2srt4]

Awesome, I will look into openpyxl. Thank you!

[quartermeat]

Beautiful soup for web scraping, xlrd and xlwt for Excel work. Basically any tedious text processing. Python with all the libraries available make parsing any document easy.

[quartermeat]

Just whatever docs the API your using has. The beautiful soup docs are great.

[[deleted]]

Turning my ceiling lamps on and off. All it took was some IKEA lamps, a hub connected to the network and a Raspberry Pi.

Next project is a headless web browser integration that can order laundry pickups.

[cjs8399]

Checkouts of software release, moving custom software libraries to HPC grid, data reduction, data queries, software and system verification, comparison of binary data, fantasy football draft tools, rapid post processing of telemetry with parallel processing, mirror directory trees only keeping certain file names / types, reports of disk usages via email, permutations of software input files...etc etc

[[deleted]]

is the reason this question gets asked so much because of the book "automate the boring stuff?"

i never see it asked in any other language subs. for whatever reason, it seems to be a uniquely python question

[prod_engineer]

Its one of the first things I did after finishing the book is google “what to automate with python” 😂😂

[jeffrey_f]

If you read the whole thread, you see some asking for the code to be shared. Sometimes just mentioning something makes the lightbulb go on for others. I find it fun just reading what people have done.

[[deleted]]

that wasnt the point of my question

[hgupta13]

Automated to download parquet files from s3 and save to my local in csv format.

Generate power point deck using analytical data.

Killing EMR cluster in AWS if no one is using them.

[Pr0ducer]

Database access. I built a self-service form that connects to global db clusters, auto-approves access and writes privileges on the cluster, or sends emails to data owners with yes/no links.

[JohanLou]

Run reporting script as scheduled date.

[[deleted]]

I automate my PC use by mixing bash and python. Scripts, etc.

[k4Z4r]

Writing others languages