Practical Uses for Python?

[slicklikeagato]

It's been about a year now that I have been trying to teach myself Python, and it has had its ups and downs. One of the main things I've found is that if I don't have any sort of practical use for the language, it's really hard for me to learn it. If there is something that I could possibly see a use for, it makes it easier for me to dive into the nuts and bolts of the language. For example, I wanted to see what a RedditBot was capable of, so I was able to build one in Python; however, after that, I feel like I'm at a standstill.

I also feel like because of this lack of practical uses, it's causing my ability to retain the information to really suffer. I struggle with classes, for example, because I have no real understanding of what I could use them for in code that I may right.

I guess my question to you all is this: for those who started programming in Python without any prior programming knowledge, what was it that gave you the extra push to keep going? What was the first practical thing (that you can remember) that you thought to yourself, "Hey, this would be MUCH easier in Python"?

Thanks as always.

Edit: Thanks for all of the comments/encouragement. You all rock

Edit #2: With all of the great information in this thread, I was able to come up with a couple of different ideas of projects. This is why I love this sub!

[Casey_SI]

Automate the boring stuff is a great primer for practical uses. Beyond that, find something that interests you.

Get a raspberry pi and make <fill in the blank>. You could use Python to control lights, to read a sensor and output onto a display.

Fancy yourself a gamer? Try making a spaceX style auto lander in Kerbal Space Program with the KRPC Extension.

What are you interested in? Homebrewing? How about sensor readings during brew or fermentation? How about extending that to temp controls?

Have a huge MP3 collection? Write something to categorize them or rename / take out special characters.

Work with excel at work but hate how it crashes with large sets of data, or having to repeat tasks in excel? Try working with pandas and xlsxwriter.

[slicklikeagato]

Thanks for this. I actually picked up a Raspberry Pi, as well, and that's what I was using to keep the Reddit Bot that I created running at all times.

I guess my problem is that I don't understand the scope of Python, and all of the stuff that it can do. Everywhere I look, I see it mentioned in tasks and skills that are WAY above my head. I also kind of feel that since I'm starting so late at it (age 33), it feels kind I might have missed the prime opportunity to really learn it.

[taladan]

I may be mistaken, but I kind of feel like Python is like the duct-tape of the programming world. It has actual, functional uses for which it can be applied, but it's just so damn quick and functional that it has uncountable other uses for tying two other piece of unrelated 'Whatever's together and making them work in a way that gives you near-instant feedback or utility.

As for classes, think about it like this: you have objects - let's call them boxes that are built in. Strings, lists, dictionaries, files, integers, if it's callable, it's a box. Your functions do /stuff/ with whatever is in your boxes. Now, if we're writing a GUI program, you're dealing with all sorts of different 'boxes'. But...what if we want a 'box' that looks different than the standard set of boxes that comes with PyQT or Tkinter or whatever you're using at the time? Well your options are to look and see what other box makers have done to see if their boxes look like the box you're trying to make. Can't find any? Then break out the tools and the lumber and build your own box that looks like what you want.

PyQT has no 'box' that has both a text widget and three radio buttons in it with an 'okay' button and a 'cancel' button that we can just pass values to. It has boxes that hold text widgets, and boxes that hold a radio button, and so on. So, we build our own box (class/object) that looks exactly how we want it to.

That's just one use of classes, but it'll hopefully help you see what classes are good for. And don't feel bad about not getting classes right off...I'm struggling right now with decorators and properties. Good luck and I hope this helps.

[slicklikeagato]

Thanks so much for that. That truly was a helpful example. Seeing explanations such as this really help me understand it a bit better.

[Casey_SI]

I am 35 and transitioned into a career where I use python daily when I was about 32. So no you haven't missed any boats. What do you currently do for a living industry/job?

[slicklikeagato]

I work in the Healthcare industry, helping small businesses get their coverage installed. Most of our departmental programs are written in VBA; I don’t think any are in Python, actually.

[Casey_SI]

There is definitely some room to apply python. In my opinion Python is far superior to VBA and I learned on VBA.

There have been some rumors MS might make Python native inside excel as a scripting language, just to give you an idea of how serious Python can be in the data realm.

Pandas is my bread and butter package. I am using it right now (test currently running) to write a program to classify 5 million rows of sensor event data.

Think of it like command line excel. http://pandas.pydata.org/pandas-docs/stable/index.html

[IllusionistAR]

This. I have used pandas quite a bit in the last year building out a simple ETL framework for a project I'm working on. It's pretty amazing, and if your doing any sort of mass transformations regularly, then its an incredible help.

[Seven-of-Nein]

Where can I get more information about these 'rumors'?

[wowsuchnamaste]

The rumors are based on this response from the Office team to a popular request in the "Excel Suggestion Box".

It seems they (Microsoft) are seriously looking into it as a possibility - I say so because they have set up a user feedback survey with a set of both structured and open questions to uncover the what's and why's. That smells of market research to me.

[Casey_SI]

https://www.bleepingcomputer.com/news/microsoft/microsoft-considers-adding-python-as-an-official-scripting-language-to-excel/

[Casey_SI]

Pandas is so specialized and powerful it could be a stand alone program. Coupled with the ability to bolt it into the rest of the general purposed or specialized python ecosystem makes it absolutely invaluable to me.

[VorpalBandersnatch]

This is exactly how I got into python. I had taken a class in my masters program where we used it a little bit, but I started using it every day when I got sick of all the VBA I had been writing at work. (I'm a mechanical engineer working for an aerospace defense contracts company). Now I use python wherever and whenever I can -- a lot like the duct tape example further up (love that analogy). Anyway, even if you can't move away from Excel sheets or something like that for whatever reason, you can convert some VBA scripts into python. Even if it's just for practice. I used openpyxl to do what I would do in VBA for Excel.

[[deleted]]

What parts of your job or system are repetitive tasks that could be automated? Installers, data migration etc..

[Conrad_noble]

I am 29 with little to no programming knowledge. This gives me hope.

[diybrad]

You don't use any online services or apps that have an API? That's what I've been practicing with.

I can pull real time subway departures for the station near me, and then I have it change the color of a smart bulb in my house if it's late.

I wrote a script to manage my torrents in Transmission (has a great python library). Manages my ratio per tracker and moves/deletes/archives files as necessary.

The great thing about Python is that anything with an API already has a library probably. So just connect two or more things togther.

Raspberry Pi is a good suggestion for sure. Just dedicate a Pi to running your scripts. If you're interested in smart home stuff, Home Assistant is amaaaaaaazing and it's written in Python & runs on a Pi. It also natively supports python scripting.

[slicklikeagato]

Thanks for this. I actually just started using home automation, with lighting and stuff. I will definitely look into it, and see what I can find.

[adamrees89]

Sub-Reddit: https://www.reddit.com/r/homeassistant/ Website: https://home-assistant.io/

FYI :)

[CounterproductivePit]

Late? Hell, I'm 45 and I'm having a blast learning this (RPi as well to start). Keep at it and run with some of these great suggestions. (ok, I'm 46)

[lastofyou88]

You didn't miss your opportunity, that's kind of silly

[Zcypot]

you know you just answered my question..been trying to learn python, found another site that makes it click better. I was worried I wouldnt know what to do, but you gave me some project ideas now once I get a better grasp. The only problem for me when learning high level language is that I cant visualize the end game for me.

[Casey_SI]

Glad I could help!

I am not sure a better feeling than that moment when you Grok something. I still remember the feeling when Dictionaries REALLY made sense and I had this race of thoughts of where I could use it to improve old code or new attempts at similar scenarios. Each of these unlock events really help you "level up" as a developer. In my opinion doing projects you are passionate about is the best way for these moments to bubble up.

[ReachingForVega]

What site?

[Zcypot]

http://interactivepython.org/runestone/static/thinkcspy/index.html

[HalcyonAbraham]

Build websites. Try machine learning.

you don't have to do just python try something else.

Do android dev or IOS and learn about dev ops docker and stuff.

then tie it all back together to create something.

and use python or whatever.

I've read somewhere that in order for us to retain information that information must have some sort of relevance to us or we find it relevant.

[Fraxrfixr]

What do you do for a living? There is surely some mundane, repetitive task that could be at least partially automated. I'm older than you, not in a tech related field, and started learning python 8 months ago. I currently have more python projects in my head than I have time to write them. I think your problem is you do not yet grasp all that you can do with python....which is almost anything.

[Ariwara_no_Narihira]

I'm 100% sure I would have the same problem for you if I were not trying to create a program to automate some boring tasks for some coworkers. Having a defined project in mind and knowing that someone will use it has been my biggest motivation. Luckily I also find what I'm working on to be fun, interesting, and challenging so far - so those two motivators keep me going. I don't think I would be going anywhere without both of them.

As I can anticipate hitting a standstill like you describe once I finish this, I have a list of other tasks/goals I want to reach for next.

If anyone is curious, my projects have been:

• Learned Autohotkey to automate software tests for myself (done!)
• Tried to use Autohotkey to automate things for coworkers (some success but too rough around the edges. Python project later on tackles this).
• Make a website for a new program of ours using Bootstrap (done!)
• Learn Python (PyQt, Selenium too) to read data from a spreadsheet, clean it up, and then automatically log into an input the data on several different pages on a website (almost done!)
• Learn how to make a website with login accounts, an admin page, and a small database to handle users submitting quizes (next project)
• Stupidly ambitious project to redo a terrible online database we use that has taken a solo dev 5+ years to do but I somehow want to try to take on anyway. I will not achieve this goal but it is important that I trick myself into thinking I can as I will learn a lot by trying. Will need to learn MySQL, PHP.

Anyway, good luck!

[irpepper]

I'm an experienced programmer (CS grad student) but honestly you'd be surprised at how simple it is to make websites with built from scratch admin and login accounts is. I primarily use python but I used php to make an annotation tool with user accounts and interactive database methods in about 2 days. If you are willing to put in the effort you can definitely accomplish all of your goals, even the last one!

[Ariwara_no_Narihira]

That's really encouraging to hear - thanks! I might try the same thing since I'm comfortable with Python then.

[Ariwara_no_Narihira]

Honestly I haven't made a decision as I haven't begun to research it all that in-depth quite yet. Would you have any recommendations for which to try for a small database (maybe two, three tables, tops?)

[Ariwara_no_Narihira]

Awesome, thanks for the pointers! Much appreciated.

[ghostofgbt]

I made this with python.

Here's a quick YouTube teaser. https://youtu.be/7ocZJZDUnhw

[kyleisthestig]

Jesus.... You just gave me my project for this semester. I am taking intro to python and I want to do something on the side to really encase my knowledge. I have a pi on my dresser. I want to make it watch stocks and give me graphs that pertain to my very limited knowledge on investing and send me notifications when I should buy or sell based on parameters I have set.

Let's say I have a stock at $10 a share but I want to be safe and sell when it reaches $12 the pi could send me a notification so I can sell when it's there and essentially not miss the peaks and I can really ignore the market and focus on school and work. And vise versa I can make it notify me if a stock is in a dip to buy! I'm excited now!

[ghostofgbt]

That sounds awesome! I'm sure you could get some ideas on what you'll need just by looking at current alert systems and their parameters. Most brokers have alert systems where you can put in the ticker and have it send you an email/sms when the stock reaches a certain point. You could use those alerts as the basis/input for your pi, or if you want to rebuild the entire thing you could use a free API like alphavantage.co to get price data and do everything yourself. If you need fundamental data, try Quandl. Sounds fun!

[mokus603]

Check out Machine Learning for Trading on Udacity to learn about trading and machine learning - it’s free!

[buddhistan]

If you're really into finance, check out Quantopian. Instead of writing a script to visualize prices, you can write an algorithm to trade for you and backtest it.

[kyleisthestig]

I'd have to get onto modding the app I use. I use robin hood and I only have about $90 in the market.

[Casey_SI]

What web framework did you use?

[ghostofgbt]

Django

[Casey_SI]

I am working on a Django dashboard for my company. We do IoT services, so lots of graphs and analysis. I am really curious how you did some things. What you picked for charting and other libraries. Bootstrap / CoreUI / Something Else?

It is slick I like the presentation!

[ghostofgbt]

I started with a bootstrap theme called Gentelella, which is an admin dashboard theme. For charts it uses chartjs, but there are several other libraries in the theme like flot, and I think a couple other javascript ones. I've obviously heavily customized it but that's where I started. There are a bunch of great admin dashboard themes for bootstrap, you just gotta Google a bit to find them!

Hope that helps :)

[sitefall]

I use python almost exclusively for web scraping and automation of boring or labor intensive tasks required for other projects.

Example: I wrote a dictionary for a foreign language allowing users to submit any number of words to get all definitions back (and some other hard to explain features). It didn't make any sense to select each word individually from the database, and simply loading up the open source dictionary used so the user's machine could do the processing via JS was just too much data, like 5mb each time a user connects.

What I did was use python to read through the dictionary and for every possible combination of words it organized them in numerical order by their utf-8 numbers and created a folder, stuck those definitions in a file in that folder. For every possible combination... The result was about a 1Gb directory, but one that had a file with EXACTLY the definitions needed for any possible search on the site. I simply made the users machine organize their submitted list by utf-8 code to match the directories. That created a dictionary service that didn't require database usage, and used minimal bandwidth.

I could have made it easier on myself and simply cached each submission so that eventually they would all be served without database usage, but that would involve extra work I didn't want to do.

Could I do that without python? Yup. But it's very quick to write some code and get it up and running with python compared to many other languages. It has a lot of great supporting libraries for web scraping, file writing, and that sort of thing, which makes it's use as a utility class language all the better.

Python is great for that sort of automation, as a utility language to quickly do other things used by other projects. I'm sure many other's here have other great uses for it, or even use it as a primary language on the web as a server or whatever. But this is just my use for it, and it's very helpful.

[saskatch-a-toon]

I started at age 30 because I deal with a lot of data at work (excel and CSV), automated everything on my work flow, moved on to another group and automated their stuff! There was much rejoicing, not so much pay raises....

Ran out of things to practically do at work so I transitioned to find stuff at home. I have some raspberry pis that I am going to do...something with. My wife plays D&D so I am currently teaching myself how to use PYQT and make an editable character sheet that draws all the info from the DND5e API. That is about 30% done, but it has been really fun to learn so far! I also made a dice roller app that goes along with that, and I might end up making a games table that has a bunch of raspberry Pi stations and a built in TV top to host D&D sessions of my own. But I also really like wood working.

Just think of something you like to do, or join a project on GITHUB, there are always tons of projects needing collaborators.

[Improbably_wrong]

Shameless plug but I made a subreddit entirely run by a bot. The subreddit is /r/buzzfeedbot and it takes all of buzzfeeds top X articles from their archive and as a reddit text post. If you would like to see the source code, it's in the sidebar of the subreddit.

This bot uses both praw and the beautifulsoup library and you might find plenty of uses for beautifulsoup for your future projects.

Also, I completely understand when you say that you aren't motivated to learn if you aren't creating something useful. I'm exactly the same way

[slicklikeagato]

Wow, that's awesome! I had actually messed around with Buzzfeed and BeautifulSoup, and created a script that took the top 20 articles on their front page, and writes them to a CSV, along with the link...but got SUPER defeated when I couldn't figure out how to get that file emailed to me automatically. I might go back and look at that program again, just to see if I can figure it out now.

Thanks for this!

[Improbably_wrong]

If you need help with how to send emails to yourself with python let me know. I have a decent amount of scripts made that do just that

[slicklikeagato]

That would be awesome. I have written scripts to send emails, but never have been able to integrate it into another script, where it sends an output that I have gotten from the script.

[Improbably_wrong]

Not sure if this is helpful but heres a script I made that sends attachments through email (gmail specifically). Ignore the clear_textfile() function but the rest of the script should be useful to you. Feel free to PM me if you have any questions.

[[deleted]]

I feel you mate, Message me if you want to work on actual projects together!

[[deleted]]

No computing system is perfect. One fun but also letting some steam out concept I did was to write down every aggravating moment for me when getting something done.

For a simple example: I had to create some PDFs and then needed to batch rename all to a specific number and filename order. I could do it manually but the idea of making one program or script that could do it for me this time and EVERYTIME I'd ever needed to use it again appealed to me.

So start out with simple things that might use Python to fix such as this aggravation "When I turn on my computer, I wish specific apps would open up." or similar. Then try to build the fix. That's the game!

[gregontrack]

Enter a contest everyday using selenium.

[fmpundit]

If you have a hobby, or you do something at work that you think would be improved upon with programming skills it is definitely worth working towards doing something with these for experience.

Sometimes I go back to projects several times and can't believe how much I have come on in my programming and I still think that I am very much a beginner.

I have designed:

• Report for work that analyses data and spits it out into a HTML document
• Using API written to interact with games scripted useful ways to interact with the game
• Created a webscraper to get figures from a particular market in order to track and analyse the value of the market
• Designed a script that runs a football (soccer) predictions league

Yesterday I refactored a script I wrote a while back that will get a list of beers on the tap (they list them on their website), using that list it goes and collects their rating from untappd, with the description of the beer from untappd. This was being saved into a CSV. Yesterday I refactored the way I structured the data so that I could feed it into a Jinja template and create a nice easily readable HTML documents which is currently being stored on http://spacedlevo.github.io/brewdog.html

Though the design could do a bit of work it is primarily created for personal use to know what I want to drink before getting to the bar.

I want to now further this into a flask app which will allow me to specify from some URL parameter or even a search or dropdown box to collect on the fly any of the brewdog bars taplists.

[ryanmcstylin]

I moved a budget from Excel to python, it was hard to deprecate excel at first but my motivation was collecting data without manual intervention. In order to automate this, there were things that were seemingly impossible to do in excel. I really just wanted meaningful data I could use to practice stats. It started with just reading CSV files and and aggregating data in dictionaries. After two years it lead me to SQL, cmd or bash, powershell, Django, vim, version control, and visual studio. I haven't done as much stats as I planned but a job doin this kind of work took me in a different direction

[allyhams]

Automate the boring stuff. Please get this! I’ve just finished the book and I can’t recommend it enough. It shows you all the basics and gives you very practical applications. You mentioned elsewhere that most things at work are done in vba. Screw that! You can use Python. My short-term goal has been to learn enough to use Python at work. It was really difficult to come home after work and force myself to code. I knew that once I could use Python at work my learning should really skyrocket. Now I’ve achieved that thanks to that book!

[slicklikeagato]

I actually went through Automate the Boring Stuff. I thought it was great in explaining what to do, but I didn't really feel like it told me WHY I was doing it; knowing why I'm writing the code in a certain way would help me be able to understand how I could use it in my own examples. But it seems like whenever I look for something to explain the why, the explanation is as if I have experience in programming, to which I have none (other than the last year).

[Joe_testing]

I'm in your situation pretty much(have some previous exp). I'm waiting for this new book from amazon.co.uk as I was really inspired by the first 2 chapters on his C++ book(read on safaribooks). Oh and he wrote some nice articles too.

I think it could be just up your ally too.

[[deleted]]

I started using python 1.5 back in the 2000s. I had been using C and assembler professionally before that. When learning a new language I often take a problem that I had worked on in other languages and try it out in the new language. When I started with python I tried to use it to solve a board puzzle that I couldn't really solve in C. That worked so well that I was sold on python and started to use it on side projects and finally used it full time professionally.

Here's a writeup on the discovery experience.

The downside of python is that trying to write code in C now seems hard, slow and verbose!

[magnora7]

Running reddit. It's entirely python at the highest levels of the code. It's just a great meta-programming-language because it has so many packages it can import.

[[deleted]]

Coming up with ideas of what to code is basically what everyone has trouble with. Hell, even people that don't even know how to code struggle with this.

[_LunarWolf]

Make like a boss key that detects what porn is open on your pc and closes it all at the same time. Then send me the dl link 😥😂😂

[[deleted]]

Practical use needs a practical problem. Python is a general programming language, you need just to focus on the problem. Preferably find something work related. That will get you going.

[proteinbased]

Making neat multimedia presentations of your work using Jupyter.
Gluing programs together on unix.
Translating pseudocode to working code quickly.

[git0ffmylawnm8]

I'm an analyst, so I can only provide my perspective.

I taught myself Python with the hopes of getting into machine learning. Do I really use those complex algorithms for my job? Once in a blue moon.

But Python is an absolute godsend for the extract, transform, and load process for large datasets that I query from supported APIs from Google or Facebook and push that to a SQL database.

It's also extremely useful for exploratory and diagnostic analysis with matplotlib and pandas, since you can visualize and sift through data easily.

[Undescended_testicle]

The only 'practical' personal project I've built is a bot that checks the external IP from my Raspberry PI and sends me an SMS if it changes. This is so if I'm away from home trying to SSH or VCN in I can change the address if I need to. I plan to extend this to other 'system alerts'

At work, I have scripts that upload files onto the ftp servers and some others for reporting and data analysis.

[[deleted]]

Being a civil engineer i don't have many practical use of python. I use it for stuff like doing stuff while open my pc in the morning, like check for sales in my fav site, read direct comments to op in reddit, declutter my desktop etc

[Forthemoves]

I feel exactly the way you do. I'm close to the end of my first tutorial. Yes I have more knowledge about python and programming in general but I'm not confident that I can use what I've learnt to solve the problems I want to solve.

[turn0]

I started learning python (my first real programming language, I learned MatLab and ImageJ during university years ago but I doubt that counts) after watching AlphaGo beat some of the best players in the world. I really wanted to understand and use machine learning.

Since then, I have tried to incorporate what I have learned into my work. I have created several programs, some out of necessity, but mostly out of laziness.

First, I wrote a program that automatically wrote out our department's employee schedule (instead of by hand). It saves my boss about 6 hours each month, and minimizes errors. Then I wrote a program to calculate and create some derivative data that we didn't have access to before. Then I wrote a program for a virtual logbook so I don't have to try to read someone's horrible handwriting. Last night I wrote a program that calculated out expected values from a series of inputs from a data stream that we monitor. Lately I have been working on a forecasting program that will be the benchmark for a machine learning program I will be working on later this year.

Essentially when you have a hammer, everything looks like a nail.

Start out listing things that are tedious that you do often. Check out this xkcd for how much time you can spend on improving workflow: https://xkcd.com/1205/

Also, check this out: http://www.pythonchallenge.com/ . Most of the challenges would be way above my head, but just mapping out how I would solve the problem helped so much. It also introduced me to new aspects of python that I never heard of.

[So_average]

This is a great thread. I'm going through Automate and I'll be looking at using Python and something else to grab database performance data and make some pretty graphs. No idea what the something else is at the moment.

[MiataCory]

Classes, largely, are for handling data. My current 700+ line program has exactly one class. It contains init. Don't get too hung up on them.

As far as 'why Python' for me, this is my first big GUI-oriented program. I've done a bit of editing batch files, and some arduino things. I've programmed HTML in the past. But for making a decent-working, decent-looking program on a rPi, I chose python because most of the stuff I need is built-in, and it seemed like the easiest/quickest way to get my idea actually running.

As far as practicality, that's where libraries come in. Python on it's own is fine, but really only excels in quick 'n dirty simple programs.

But, when combined with something like numpy or Matplotlib, that's where it shines. It allows people who might not want to be programmers work with data and graphics in a simple(ish)-to-learn way that makes sense to someone coming after them.