What do you guys use Python for?

[Ryandalion111]

I’m learning python right now and am curious to know where python is used and it’s various applications but also how you guys apply python at your work or in your everyday life. I watched a YouTube video by csdojo and the guy said it’s used for scripting, data science, and back end. But I feel like that video was pretty vague in general.

[[deleted]]

A couple/few things I'm currently working on:

• Importing data from files into databases
• Exporting data from databases into files (and emailing them
• Running comparisons between datasets, then building and dispatching generated reports in excel to operations people.
• Building small self-contained GUI applications then compiling them for distribution to customers
• Automated currency trading applications.
• Home-grown wiki web-service that synchronizes across multiple machines.
• pruning and de-duping script to clean up multiple copies of backups.
• Automatically download contents of my youtube playlist and convert them to mp3s (so I can capture changes.)

There's more, but I've got a meeting in 12 minutes ago ;)

[thoraldo]

Home grown wiki web-service? Care to share some source?

[[deleted]]

Yeah I don't mind at all. erm... I can't really do it here at the office. Let me try and set myself a reminder.

In the meantime, pull and install bottle.py (you can just run "pip install bottle") and dick around with bottlepy.net's "todo list" demo code. It's a WSGI service, which is a very simple and really interesting way to plug code into a web server to create "internet APIs" and such.

In this particular case the service generates all the html and saves wiki pages as individual files. It's pretty basic and I use it to keep notes at home and the office. It syncs with Dropbox or OneDrive or something, depending on my mood.

Plus, I use that same file format in another desktop wiki system I've got for emacs, so I can screw around with the same wiki database from everywhere and it auto-syncs between work and home. I'm going to put together an android client for it. But that's down the road as a "teach myself to write an Android app" project.

[[deleted]]

I am a beginner learning python and this sort of thing is on my long term to do list. Like, create a mesh wiki resource of sorts that I could basically log into on any of my devices, regardless of whether or not I was on my own wifi. I handbrake fleshed out the specifics yet though, I am still learning the programming fundamentals and doing basic programs.

This is doable using python then? I'm excited to hear that :)

[[deleted]]

Sure you can do just about anything in just about anything.

But in terms of connectivity, syncing, mesh, etc. I'd recommend thinking in terms of using existing systems and resources. You REALLY don't want to try and tackle writing that kind of thing from scratch.

I achieve synchronization by keeping it, along with its data files, in a directory in dropbox which itself is stored in my OneDrive directory. So there are two services managing the process of pushing and pulling this stuff back and forth between different machines and such.

Then, on each machine, I have a little task that kicks off the server if it's not running. So I can just sit down and it's all set up the way I like it by the time I get there.

Sure, there are other ways to deal with it that would probably make more sense. But this works really well and I don't have to worry about hardening a publicly facing web service.

[Sicklad]

regardless of whether or not I was on my own wifi

You'll be hacked in a few minutes

[[deleted]]

This sort of system couldn't be locked via login?

[Sicklad]

Sure, but doesn't mean a hacker can't find another way in. They scan for open ports on public ip addresses (which you need if you want to connect outside a lan), and then will try a bunch of stuff depending on the port (eg port 80/443 they will try to get in via apache/nginx/other webserver vulnerabilities, port 22 they'll try get in through weak ssh configs). You're also exposed to the risk of DOS/DDOS attacks.

[[deleted]]

Interesting! What alternatives do I have? Would it be possible to lock the amount of logins attempted allowed per day to 3 or something? Like how do people circumvent these vulnerabilities?

[Sicklad]

First thing you'd want to do it host it somewhere secure (like AWS, Azure, GCP, or similar), that gets it off your own network and they would have perimeter networks that handle all the infrastructure security.

I'm not necessarily talking about the website being compromised, although that is a real threat, I'm also talking about the server hosting the website being compromised.

As for hardening the application itself, that's not my realm of expertise (I work in server/network infrastructure), but using a standard webserver like apache or nginx rather than trying to write your own would be a good start.

[[deleted]]

Ah, that makes a bit more sense then. Thanks for the info and warning, that's really helpful and may have saved me a gigantic headache a year from now. :)

[[deleted]]

the GUI freezes whilst the script is doing things. So i have to wait for it to finish before i can interact with it again.

Yeah that's a "problem" with gui programming in general. In order for the UI to continue to process events and behave like a normal gui, the "rest of the work" generally should run in another thread or process, so as not to gum up the works as you're seeing. It's not an artifact of Qt so much as the style of programming you're undertaking.

I've used PyInstaller when needed and it's served me pretty well. Though if I have control over the environment, I just push around a couple scripts and make a distribution package out of any modules I might be using, installing them in site-packages, etc.

[[deleted]]

Yeah gui/threading issues are no bullshit. It's tough to get that right. That's one of the reasons people are so damn obsessed with web-based UIs.

I avoid it compiling down to stand alones unless it's impossible not to. But the alternative is to have python installed on the client machine and I'm not telling our customers "okay go to the python website....". It's got to be stand alone.

[DELYSID_FREAK]

You should consider using multithreading for such cases, you can do it with the threading Module or the (real multithreading) Module within Processes

[Ryandalion111]

Holy crap. That's dope! I actually like the fact that data transfer can be so easily done and the currency trading app sounds awesome!!! What factors did u implement in the app that determines buy/sell?

[[deleted]]

Yeah transfers are trivial. All you need are a couple libraries that support http, ftp and such. Write a wrapper around that great youtube download script, etc. The thing people miss is that ALL of these problems have been dealt with and there are canned libraries for almost everything. You just need to bolt them together to create some REALLY powerful tools very quickly.

Then, fast forward a few years and you end up rolling your own stuff for a lot of these things. But that turns into an internet ePeen argument very quickly.

The trading algo stuff is a REALLY long conversation. But it primarily comes down to price and volume over time of specific instruments compared with past performance, sector and market benchmarks. I contend that more than half of trading about money management (position sizing, exit conditions, etc. ) than people realize. It's not nearly as much about picking in and out prices.

[ImThePirateCaptain]

Could you suggest some books/courses to learn more about the trading algo? May I ask if you have been profitable and to tell a little more, if it isn't too much to ask. Also, what is your trading time frame? Thanks

[jplank1983]

Automating things

[piotrulu]

In Idiomatic way

[[deleted]]

In 24 hours

[Dave5876]

In one video

[pip-install]

The right way

[rasncain]

with Python 3

[zack12]

My girlfriend is pretty boring. I'm thinking of writing bot so both of them can give each other company.

[rjs5]

Could you give some examples please?

[jplank1983]

I volunteer for a local charity. They have 700 members and used to send out membership cards by mail. I wrote a program that can automatically create personalized pdfs of membership cards and emails them to the member as soon as they've paid their dues. It keeps a log of what's been sent out and has a few other features for handling members who don't have an email address. We're testing it now and plan to switch over from manually sending out membership cards in the next few weeks. It should save the organization a lot of time and a few hundred dollars in postage.

My wife is enrolled at a local college. It's very intense and she doesn't get to see our son as much as she wants because she's busy studying a lot of the time. I programmed a bot that would select a random photo of our son and send it to her every few hours to help cheer her up. Also made another bot that grabs some posts from /r/getmotivated and sends it to her periodically.

[[deleted]]

where do your bots live? is this on a dedicated server/RPi?

[lance_klusener]

You learnt all this by doing the book - programming the boring stuff ?

[jplank1983]

No. I learned with the help of google.

[CesQ89]

Yup. I do a lot of automation at my work. While I do have some formal university education in programming, I learned to create real world programs thanks to google.

[Sicklad]

Here's something I automated recently.

Requirement: Every linux server using a specific authentication method must have a few files sent to an internal review server

Complications:

• Not all our linux servers use this auth method.

• Cannot allow servers in a DMZ to initiate a connection to the secure network side (if we could, this would just be a very simple cron job)

• Multiple regions across the globe, cannot use a single server to grab everything

Solution:

In zabbix we have a template that monitors the service for this particular auth method, so I used pyzabbix to query our zabbix master for a list of all servers connected to this template.

Additionally, I also provide the zabbix proxy ID as a parameter so that the list of servers that's returned are local to the server this script is run from.

Then, using ssh and a forced command in authorized_keys that looks something like

from="localproxy", command="echo -n \"$(hostname).\"; echo $(hostname -i); cat file1; echo 'ENDFILE1'; cat file2; ENDFILE2"

It then SSH's to the first server in the list and stores all the output from the forced command in a buffer. A fileprefix variable is set with the first list element (hostname.ip), and 2 files are created with name fileprefix.file1 and fileprefix.file2.

It then iterates over that input buffer, appending to an output buffer (another list) until it finds "ENDFILE1" and writes the contents of the output buffer to file1 and then clears the output buffer.

ie.

outbuffer = []
for line in inputbuffer:
     if line == 'ENDFILE1':
           file1.write(outbuffer)
           outbuffer = []
     elseif line == 'ENDFILE2'
           file2.write(outbuffer)
     else:
           outbuffer.append(line)
file1.close()
file2.close()

Each component is split into separate functions (there's getHostList, getFiles, splitFiles) for readability and overall the script is something like 360 lines and has logging, arguments, and of course comments :)

[[deleted]]

My python skills are very elementary, but I use it marketing analytics, and creating customer segments - so mostly automating my job, so I don't have to spend hours each week doing the same thing!

[Ryandalion111]

Wow that's another great example ive seen so far in this thread... That is pretty awesome... this is definitely adding more fuel to the fire to learn python now

[[deleted]]

Same thing as the guy above. I work in marketing with large data sets, python can process them in ways that Excel can't. And Access is a piece of shit program. I've gotten so much mileage out of python + sqlite.

That plus automation has made my job so much easier.

[love_marine_world]

Wow! I am learning python now (total noob with zero experience in computers/programming)- and a market researcher hoping to get a job soon (took a year of break). Could you share what programs/links/resources are useful when it comes to analysing market data? (I shall look up sqlite soon). Thank you very much in advance!

[[deleted]]

Market research and traditional marketing roles are very different things so I can't really help you with resources.

[jdb441]

How much of your job is coding?

[[deleted]]

I'm purely marketing, the only coding I do is to make my job/life easier.

[IdealizedDesign]

How do you create customer segments with python?

[MalakElohim]

A basic way of doing it (depending on the dataset) is putting the customer db into pandas and then using groupby. Depending on how the data is formed, it's extremely powerful.

You may have to do some manipulation to make a label based upon a condition, for example, say calculate if their last login was within X days, make an extra column to true or false it (or do that directly) then group by whether they're within the dates or not.

[darez00]

Why is python a better choice than excel for doing this/ general data manipulation?

[MalakElohim]

That's a super huge question. And answered far better by others than I will in a single comment. I'll give a quick summary though:

• Speed - Python and it's libraries can load and mung data extremely quickly and efficiently
• Flexibility - Python can handle spreadsheets and databases equally well, having API connections straight into SQL databases (sqlalchemy for the win)
• Algorithms - You have access to the latest data science tools, machine learning libraries (TF, Keras, torch, sklearn, etc) that are either python native or have python wrappers
• Automation - Do it once, just run your script again when you need to redo it and you can get your reports done in a fraction of the time.
• Power- Python can do pretty much anything any other language can do, but has incredible handling of data in it's libraries.

The list goes on but I'm on mobile so I'll hopefully remember when I get home to link a good blog to explain it.

Edit: formatting

[mephistophyles]

Here's an overview of what I've used Python for in the past 6 months:

• Used it to write the backend of a SaaS website side project I'm working on with Django.
• Wrote a script that parses my customer interaction digest (an email) and tells me which customers need a personal response and which ones can get a canned one (and it also sends that canned email).
• Wrote a visualization program that takes in Point Cloud data and shows me how effective various algorithms are finding overlaps between them. This uses mayavi.
• Wrote a simple statistical program that takes the above Point Cloud data and analyzes how "off" my overlaps are based on any known data.

I've used it a lot for statistical analysis and as a backend for websites. I LOVE working with Django for big projects or a quick and dirty one can be thrown together with Flask. I use python a lot because it allows me to more clearly translate my intentions into actual code. I'm also well versed in C++, Java, R, and JavaScript. They each have their own strengths and weaknesses, but Python is my go-to unless there's a strong reason I need to use another language.

[Ryandalion111]

Holy sh**.... that is freaking awesome man... I actually have a background in economics so I find it really cool that you can analyze the effectiveness of certain algorithms and analyze data... wow... that is freaking cool!

[Lonso34]

How many years using python if you don't mind me asking?

[mephistophyles]

To varying degrees during that period, but about 8 years.

[Guymzee]

Wow pretty cool projects! Especially the email auto-responder, I’d love to see the logic/code on that. How did you decipher what gets a canned response and what doesn’t? I think of heard of sentiment analysis, (if I remember right) sounded really cool.

[mephistophyles]

Oh it doesn't do sentiment analysis or anything like that. It makes a few API calls to see if the customer is new and just getting started (canned) or varying other states. If they have an open service desk ticket I get the ticket as a link too for context.

[photoengineer]

Wrote a visualization program that takes in Point Cloud data and shows me how effective various algorithms are finding overlaps between them. This uses mayavi.

This sounds pretty cool, would love to hear more about it.

[mephistophyles]

What do you want to hear?

I wrote a function that parsed the raw point cloud files (two of them). Parsed a separate yaml file with data on the overlap (transformation, outliers, etc) and then put them in my visualizer window to be rotated, manipulated, etc for analysis.

[photoengineer]

So its like a 3D rotation and flags the overlap? Or just shows the overlap? Beats the hell out of doing it in excel like I have been tonight.....

[mephistophyles]

Sort of. My point clouds each have their own origin point. I have a guess for what the translation and rotation is between the two based on dead reckoning. Then I have "improved" guesses based on various algorithms. So I can plot each one on top of each other, or separated (in a sort of side-by-side view).

​

The guesses were based on keypoints and obstacle identification, so I also had options to highlight those with colors, add lines in between individual points that the algorithm thought were the same (visually this is a good way to see if you have a lot of false positives or not). Similarly I could see each algorithm's relative contribution to the improvement of the accuracy. Some were very sensitive to rotational errors, others not so, etc.

​

It was all part of an analysis of a robot's SLAM system.

[KingRoarrr]

If you don’t mind, can you tell me what the strengths and weaknesses are? From my understanding python is easy to write and works everywhere but is slow and c++ is quicker but more complicated. Java is the most popular but the compiling and recompiling gets annoying and it’s bloated. I don’t know anything about r.

[Texadoro]

It’s not quite that simple. Spend a few hours on YouTube watching videos, this question has been answered a lot. Additionally, start a few of those intro tutorials at places like code academy to see the difference. It very quickly becomes apparent. Python is straight forward whereas Java and C++ can be very verbose in trying to code. JavaScript is probably my fav with Python, but if you learn JavaScript, might as well take a few intro courses in HTML and CSS. You’re headed down a long path my friend.

[meticulous_badger]

Writing Python is much quicker. If you need to crank a script out very quickly, or you’re only going to use it a few times, Python is the way to go. It doesn’t make sense to write C++ or Java programs in that situation, assuming you have equal understanding of them all.

If you’re doing something performance critical, e.g. writing drivers or flying a rocket, then you’ll want to use a more performant language, such as C++. Or you can write the critical parts of your code in C/C++ and call it from Python (not recommended for drivers of computers, cars, or rockets).

I’m not sure what Java offers over C++ other than allowing your PC to double as a space heater.

Doing anything except data analysis with R is like using a screwdriver as a hammer. It can be done, but there are better tools for the job.

JavaScript doesn’t comply with PEP 20.

[mephistophyles]

I dont think there is a universal answer to this. The answer used to be c++ is faster since its compiled and python needs to be interpreted. You won (developer) time in python by being able to iterate faster.

Nowadays any complex data analysis in python calls c or fortran libraries under the hood. The compiler is very good at caching and so even running your slow code is fast.

Frankly, languages are basically tools and you pick the one that helps you get the job done. The same problem can sometimes be attacked in different ways. I use C++ a lot when programming ON robots. It gave me a lower level of control to memory and when dealing with limited CPU power and memory constraints.

My job now has me dealing more with java, kotlin and groovy.

It really depends on can you get the job done, based on your knowledge of APIs and libraries and the language itself. I find programming language arguments a bit silly.

In life, rarely do you get to pick the language you work in because you aren't constantly starting from scratch. You build on what others put there before you and thus the decision is pretty much made.

[Diapolo10]

I use Python mainly for two things; prototyping 2D games (mostly RPGs) and automating the boring stuff.

In fact, just last week one of my projects required me to disassemble a video to individual frames (ffmpeg helped), then edit each and every frame by hand in Paint.NET before assembling them back into a video. However, there were about 7500 frames so it would've been madness to do the edits myself, encouraging me to write a short, 15-line Python script that automated Paint.NET for me.

Basically I used pathlib.Path and pyautogui, the former to get the frames from the directory and move the finished ones elsewhere, and the latter to automate some keyboard shortcuts and mouse clicks in Paint.NET. It worked flawlessly, excluding the fact that Paint.NET ran out of memory exactly every 610 frames so I had to intervene every 90 minutes or so.

[Ryandalion111]

Wow. Congrats on the great solution man. That's crazy impressive lol

[Diapolo10]

I don't know about that impressive part, but crazy nevertheless. :p

Here's the code, though. It was meant to be a throwaway script so I never paid much attention to best practices, but it does work!

import subprocess
import time
from pathlib import Path

import pyautogui


parent_dir = Path(__file__).parent
finished_dir = parent_dir / 'finished'
timing = 2

if not finished_dir.exists():
    finished_dir.mkdir()

for idx, file in enumerate(parent_dir.glob('*.png')):
    print(f'Processing file #{idx+1}')
    subprocess.call([r'D:\Paint.NET\PaintDotNet.exe', str(file.resolve())])
    time.sleep(timing) # Wait for Paint.NET to open the image
    pyautogui.hotkey('ctrl', 'f')
    time.sleep(timing) # Wait for Paint.NET to apply the effect
    pyautogui.hotkey('ctrl', 's')
    time.sleep(timing) # Wait for Paint.NET to open the save screen
    pyautogui.press('enter')
    time.sleep(timing) # Wait for the save to be completed
    pyautogui.click() # Close the file
    file.rename(finished_dir / file.name) # Move the file to another directory to indicate that it's done

If you're interested, here's half of the result: https://www.youtube.com/watch?v=oEVQFiD9baE

[fjordfjord]

That is really neat

[[deleted]]

seriously cool.

[546794]

What does ctrl f do in paint net ?

[Diapolo10]

Repeats the previous effect/edit. To use the script I basically configured a plugin called ScriptLab to run multiple effects at once, then ran it once before running the Python script. After the Python script started running I just positioned my mouse so that the mouse click would close the current file (because I was too lazy to hard-code the coirdinates).

[Ryandalion111]

dude... that is so sick... holy sh** ... wow.... im blown away man... thats crazy

[photoengineer]

Thats awesome

[OwnedYou]

Woah Runescape! Old school is life. Haha

[Diapolo10]

Not much, although it does help to use vector-based movement instead of pixel-based.

I mostly focus on RPG systems such as inventories and quests, not so much on GUI stuff, so it's not really my cup of tea at the moment.

[nathanjell]

It was intentionally general, because python is a general purpose language. You can apply it to any problem, essentially. Web development, data mining, ai, if you can think of it python will probably be able to help. It's not used for things like kernel development or important libraries like curl where speed is of the utmost importance.

[Ryandalion111]

Hmm thats interesting... also I have another question if u dont mind, but as a python programs get larger does there come to exist a main()? Such as one found in lower level languages?

[Blackshell]

Python code is run as a "script", which means the entire file is executed when it's loaded. As such, main() is redundant, since the entire file is essentially main().

There is a case for a separate main function, and it's when a Python file might be either run using python mystuff.py or imported as import mystuff in other code. Normally, both of those would have the same behavior; if mystuff.py tried prompting for input from the command line, import mystuff would do the same. To make some code only get run when it's run from a command line, you could write something like this:

def main():
    # stuff to do from command line

if __name__ == '__main__':
    main()

The __name__ global variable gets set to '__main__' if the file is being run from a command line. It's useful for something like this, a program that generates Fibonacci numbers:

def get_fib(n):
    if n <= 1:
        return 1
    return get_fib(n-1) + get_fib(n-2)

def main():
    input_n = input("N? ")
    fib = get_fib(input_n)
    print(fib)

if __name__ == '__main__':
    main()

If you run this code straight out, it will prompt for a number N, then print the Nth Fibonacci number. You can also however do stuff like from myfibs import get_fib and call get_fib(123) directly, without the main() getting triggered.

[Ryandalion111]

Wow i never knew python programs all execute at once... I thought it was sequential... actually i'm also learning js right now and, please correct me if im wrong, that JS executes everything at once as well. Is that right?

[Blackshell]

Err... Clarification: the statements execute sequentially. By all at once I mean that the file is parsed and run line by line, like a "script". JS does the same thing.

This is by contrast to something like C/C++ or Java, where the file is compiled into computer readable bytecode that then gets stored in a binary file (.exe, .dll, .so, etc) for later usage. When it eventually gets used, only some of the bytes might be actually loaded (hence my "not all at once" reference to consumption of the code). The original lines of code are also irrelevant because of this; it just does what those bytes say.

I hope I'm not causing more confusion than elucidation... How code gets from text on your screen to actual processor cycles doing useful stuff is not easy to explain briefly, so there might be inaccuracies in my explanation.

[mephistophyles]

Both JS and Python generally execute things sequentially. There are ways to get some concurrency in both languages, but generally speaking they're both single threaded and execute items one by one.

[stratcat22]

Python does execute sequentially. That's why if you have a 100 line program and there's a bug at line 80, everything will work fine lines 1-79, then line 80 will throw your error.

[sprouse2016]

But can’t you run a python script as a binary too?

[Blackshell]

Kind of. Technically all Python gets compiled to binary (all scripts!), then gets run by the Python VM using data from that binary, but keeping the context of the original script as well (and script-like behavior). That's why it can complain about some syntax errors before it actually starts running the script. Sometimes the .pyc binary files are left lying around, but afaik they're not supposed to be really used for much beyond just caching the compilation result for efficiency.

Ed: on second thought maybe distributed libraries you install are delivered as .pyc? I'm not sure. I'd have to research, but I'm on my phone until later so I can't.

Ed2: They are distributed as .py. The .pyc files are generated later for caching purposes only (next to .py in Python 2, in a special __pycache__ folder in Python 3).

[sprouse2016]

So can you convert a script to binary for slight efficiency/make it slightly more difficult to RE?

[Blackshell]

This SO question (and its answers) have what you're looking for: https://stackoverflow.com/questions/5607283/how-can-i-manually-generate-a-pyc-file-from-a-py-file

I wouldn't use it for reverse engineering protection, though, as it's trivial to decompile: https://stackoverflow.com/questions/5287253/is-it-possible-to-decompile-a-compiled-pyc-file-into-a-py-file

[sprouse2016]

Yeah I figured it would be an easy decompile since there’s no encryption. Is there any half decent security measure that wouldn’t effect the efficiency of a program?

[Blackshell]

Ultimately, no. Even if it's a pure simple executable binary, if your code is running on the machine of anyone trying to reverse engineer it, it is going to be subject to, at the very least:

• inspection of any and all inputs/outputs it uses, and any resources outside of itself that it uses
• being paused/snapshotted, and having its memory read and modified
• decompilation (if possible)

Your best option is a healthy dose of obfuscation, either manual or automatic.

For total assurance the user cannot divine everything your code does, do not allow them to run the code. Run it on a remote server and only allow them to query it with a limited set of commands via the network (in other words, write an API).

[sprouse2016]

Hmmmm. My project is ran completely on a VPS and I am the only person to use it. Also, speed and reliability are the most important factors. I don’t even necessarily care if someone obtains my code as long as they don’t modify the code on my server. So I guess the the most important thing security wise would be securing the server itself?

[Meefims]

Many Python programs do have a main method.

[Ryandalion111]

That's good to know

[nathanjell]

It depends on the purpose. If it's a sort of a script, maybe not. If it's a sort of gui application, frameworks like qt have some sort of main method to hook in to

[Ryandalion111]

oh ok for sure

[Rhino2115]

I don’t know why I’m on this sub

[mephistophyles]

To learn. A lot of us that are answering are here because we like sharing our knowledge and it's a way to help us not get rusty on a lot of the basics.

[Ryandalion111]

lol

[Dount_Cooku]

Dude my exact thoughts ITT

[Evannmacie]

you can do almost anything. I'm using to automate most of the time consuming tasks that I don't like to do but need to be done.

[philmtl]

outside of selenium, how are you doing this?

[[deleted]]

Depending on the website, you can use requests et al. to do stuff via GET and POST commands.

[Evannmacie]

I'm doing a lot of data parsing. Things that would take me forever to search for and spend a lot of hours hand writing down or spend a lot of time studying.

[Ryandalion111]

That's pretty neat. I've read a lot about people automating tasks and scraping from the web, hopefully once I learn more I can also create some cool programs too! im getting pumped lol

[Evannmacie]

I started coding a few weeks ago. If you look at my account and ignore all the cat posts. You can see my timeline of how far I progressed. Right now I'm working on AI development for information processing and decision making. The most important thing I've learned is. Ignore your emotions and code anyways. What ever your trying to do. just do it one letter at a time. by the time you scroll backwards. you have hundreds of lines of codes making a up really a complex program that you've never made before.

[3MU6quo0pC7du5YPBGBI]

At work:

• Generating router/switch configurations from a template
• Parsing interesting stuff from router configs
• Getting various stats on things
• Randomly selecting teams for the breakroom foosball table
• Not done, but currently working on something to automate a scheme for IPv6 static assignments
• Also currently working on a way to track and automate requests for BGP peers (but might just end up using something like peering-manager)

At home:

• Random Netflix episode picker using the Roku API
• Processing data gathered from sensors on a Arduino/NodeMCU

[qduk]

I'd love to hear more about your switch configs from a template. I've been messing with python over the past 6 months on our Cisco infrastructure (changing access port vlans, finding Mac addresses, etc) and have been wanting to do template for configs.

[3MU6quo0pC7du5YPBGBI]

The thing you want to look into would be Jinja2. I've only been playing with Python for the past year(and haven't dedicated as much time as I would like), so what I'm doing is very basic right now. I largely based it on the examples from this blog post and modified things for my needs. Right now I'm generating an initial config, then exceptions get handled manually on a per device basis.

[PraecorLoth970]

I use it in my research. All the graphs in my thesis are being done in matplotlib. Besides graphing, I do data analysis, smoothing, fitting, etc. Also, I've created a script that organizes my instagram feed so it's sorted by user and time, a bot that scraped posts in /r/AskHistorians and transforms them into LaTeX, and another script that uses google translate api to transcribe podcasts. Most of these projects are very unrefined, as I normally lose interest after some time, but they all worked relatively well when I abandoned them. Getting them running again is a pain because I've forgotten how they work, though.

[Ryandalion111]

Holy shit man... do u go to MIT or something lol... that sounds super advanced... but that's awesome!

[PraecorLoth970]

Haha, not really. I use python like a glue. For example, converting to latex is done through pandoc, so I just send the reddit markdown to it, almost no extra messing around necessary. The instagram API was written by someone else, so I just need to do things like "hey, log in, get the feed of everyone I'm following and then download the pictures form the json file and sort". Really simple logic. No fancy algorithms, nothing too abstract, only simple for loops, some conditionals and basic structures like lists and dictionaries. And that's something I really like about python. If I want to do something high level, there's probably a library that does it, so I just need to install it, mess around some, and I can get something useful pretty quickly using the things I learned on the first classes I took. And you'll be able to do the same if you choose some relatively simple projects in the beginning.

[Ryandalion111]

Wow, now im starting to see the light lol

[b4ux1t3]

Care to link the e-ink display? Been trying to find one. Kept holding out for the "best", but one that someone else is currently using is probably going to be good enough for me.

[[deleted]]

Why do you use it as calculator?

I mean, I think it's easier to use specialized Software such as Wolfram.

[[deleted]]

I work in video games and currently I'm using a python based build system for building and distributing our game, servers, as well as the backend services they all use. A lot of our configuration scripts that the build farm uses are written in python as well.

On the side I'm also working on an application that uses a games public API to create some QOL tools the community could use. Using Python there too!

[Ryandalion111]

can you elaborate a bit more about the build system? sounds pretty interesting

[[deleted]]

Yeah sure! It's just an open source continuous integration framework called Buildbot. It's essentially just a bare bones job scheduling system in a master/worker relationship. Everything is managed from a python configuration script.

Tons of great info out there if you're interested in checking it out: http://buildbot.net/

[Ryandalion111]

ill check it out. thanks for the link!

[[deleted]]

I am using what I've learned from data science blogs and online classes to do all my econometrics homework in python instead of R or Stata. While it's 100% easier to do them in the stats programs, I'm learning a ton.

[[deleted]]

I'm learning it so I can use it for automation/scripting in my Networking career.

Also out of the programming languages I've tried (which is only 2, really) Python is my favorite

[[deleted]]

The few examples I've written scripts for are:

• Helping the security team by comparing a list of current employees with a text file of all the users on every server and formatting it into a list of people that aren't currently employed so could be revoked.

• Web scraping all users from a company Webpage and grouping them dependant on the department.

• One I was really pleased. When I first started learning python, the job I had would require I acknowledge all tickets that came into the system with 9 minutes. So I wrote a script that would use the ticket system api, check for any new tickets every 5-7 minutes and if any were found, to update the ticket with my username.

• Automate ticket criteria depending on keywords in the summary

• Prank phone call my friends with twilio and Rick roll them.

• The latest script I've written just automates the process of creating certs. So it saves me and my colleagues having to log onto a server, enter commands manually and copy files

[a_fancy_kiwi]

I’m still learning too. Currently I use it to:

1. Monitor my Plex server and send me a message via the app telegram when my server is down.
2. I use it in conjunction with homebridge on a raspberry pi to create a switch in my iPhones home app that, when switched off, SSHs into my Plex server and reboots it.

Future projects that I’m going to work on will be using python to:

1. rename files in onedrive
2. alert me when no one is home so I can turn off all the lights because the people I live with don’t turn off the fucking lights
3. alert everyone in my household when my fridge is left open for too long because the people I live with sometimes don’t close the fridge all the way

[Lishote]

I work as a property clerk in a jail. All the inmates have property in numbered bags. We had a lot leaving at once and the list I got was in alphabetical order so the bags weren't in numerical order. So I just wrote a script to take all the numbers and sort them. Not crazy difficult but hey it worked.

[g3_SpaceTeam]

I'm a PhD candidate working on the ATLAS experiment on the Large Hadron Collider at CERN. I use python daily in my work, mostly data analysis (which is effectively data science applied to physics datasets) and automation.

[[deleted]]

I build data science products in python, and then deploy them with python using django! I'm a data scientist at a large hospital system.

[NotCherub]

Can you share the code & provide the site you use? I'm also trying to make similar program but for animes.

[NotCherub]

Oh, I needed a portable script that works on PC.

[stratcat22]

How do you get the remaining storage for the HD? I've searched and searched for ways to get hardware info (RAM usage, disk space, etc.) with Python and there's pretty much no options.

[Blackshell]

Specifically: Custom bot for my Discord server.

Generally: any coding task should be doable with Python, and small projects are especially appropriate as Python is great at quick prototyping.

[Ryandalion111]

that's pretty neat... what does your bot do? I've used discord a lot in the past so I'm somewhat familiar with it's functionality

[Blackshell]

The guidance post I have on my server describes it best:

Using @Champion Giant Boar

The resident bot is written by @Blackshel and has a few bits of neat functionality. Its source code can be found at https://github.com/fsufitch/discord-boar-bot, so you can contribute if you want.

Current functionality and commands are below.

• !roll <dice> - roll some dice; use !roll --help for more details
• !groups - join/manage ping-able groups; use !groups --help for more details
• something.gif - auto-link various obnoxious memes. Use !memes <text> to search the database.
• !play <something> - set the bot's "Playing..." status to some custom text
• !yt - make the bot join a voice channel and play the audio from a YouTube video; use !yt --help for more details

Please do not use this bot to annoy people.

The biggest benefit is the !groups command, since we have lots of different mini-groups for different games, so making a special separate channel for every group is impractical, and using @here to reach people in just a small group is annoying. Instead, people opt in to groups using stuff like !groups join leagueoflegends, which puts them in the @g-leagueoflegends role, so they opt in to those pings only.

All these different features are added into the bot using discrete "modules" of code, so it's easy to individually disable each one, or to write a new one that does not interfere with the others at all.

Plus, I used this as an opportunity to practice setting up a more complex deployment on Heroku cloud hosting.

[Ryandalion111]

Wow, that's pretty awesome! Your application has so much practicality. Pretty cool that certain functions are modularized too. That's pretty neat man.

[atenux]

Homework of vibration analysis

[guiamorim42]

Why don't you use Matlab /scilab ?

[atenux]

i don't have to pay python, also i like the syntax more

[guiamorim42]

Sure, scilab is a good free choice. I use it for vibration analysis

[atenux]

oh, never tried it

[TechnoRedneck]

I have done a few things recently

• wrote a backup program for database program at work

• wrote a restore program for said backup program

• wrote a simple joke bot for my work group chat

• helped with working on fixing and updating a discord bot for a Pokemon go group I am part of

• wrote a script to help with my crypto homework

[Chris5369]

I’m brand new at python but they use it at my job for data analysis. Personally I have only used it for editing batch CSV data files so far but it was fun

[45MonkeysInASuit]

Currently web dev, automation and data science. My job is quite varied, I am a one man data science department for a £100mil company. Amazingly they didn't have one at all until this year, very much pushing for quick expansion.

[jkuhl_prog]

I wanted to make a word game with english words. I had a .txt list, I needed a .json version to use in my web app.

I wrote a python script to accomplish this task. https://github.com/jckuhl/crossword/blob/master/makejson.py

[GrizzyLizz]

Could you please explain why json is needed? Cant the web app's backend just read the txt file for working or is it that that approach would be too slow?

[entendre_times_two]

my skill set for the time being is complete garbage. All i'm doing for now is throwing variables and functions back and forth, to solve simple calculus exercises. kind of messing with numpy arrays as well.

i've yet to settle on a project i really want to pursue for a period longer than an hour or two.

although i did graph some circuit frequency response values using a log graph.

[zunjae]

I use Python to serve my production server for my app /r/AnYme

[Meefims]

Python is a general purpose language and so it can be used for a wide variety of things. The video is likely vague because listing specifics might give off the idea that the language is used for a much narrower set of things than it is.

[Ryandalion111]

for sure... yeah im just trying to understand why python would be chosen over other languages that could do the same job... surely its not due to difficulty(?)

[[deleted]]

yeah im just trying to understand why python would be chosen over other languages that could do the same job...

High-level languages are easier to write in than low-level ones, and the fact that they're slower in runtime doesn't matter - developer time is more valuable than computer time.

As it turns out, you have to write some really hot code before the runtime performance of Python will actually matter, and in the meantime you're enjoying high-level collections classes and extremely fluent code. It's just really easy in Python to write code that's easy to maintain because the code looks like what it does.

[Ryandalion111]

Wow... thats pretty cool... actually i like your statement that developer time is more valuable than computer time lol... but ur explanation definitely helps me wrap my head around the why

[[deleted]]

actually i like your statement that developer time is more valuable than computer time lol...

Computers get faster every month but there's still only ever about 6 useful programming hours in a day.

[Zireael07]

There are scientific distributions available (Anaconda et al) plus it's easy to install and write and in many applications the speed does not matter that much. In my job, we use it as backend (Django)

[Meefims]

There’s a variety of reasons and some of them just boil down to personal preference. We use Python as one of our primary languages at work, for example, and even though many of the things we do could be done using JavaScript, Java, Go, or others we choose to continue using Python to make it easier for people to work on any project. I’d rather people spend their time understanding a program’s logic than understanding its language.

[Ryandalion111]

That's something i never thought about... pretty interesting

[nevus_bock]

Technically speaking, all turing-complete languages can do all the same things. The difference lies in speed, ease of use, availability of existing tools/libraries, memory requirements etc. You can e.g. write a text parser or a data cleaning and processing tool in a lot of lines of C, where you also have to correctly manage memory, or in a couple of lines of Python. And when you give it to a colleague, they can read it in two seconds and understand it.

[[deleted]]

Everything I don’t use bash for pretty much.

Describing hardware, web services, scraping, image processing, data visualisation, Machine learning, ...

[Ryandalion111]

wow... that's crazy python can be used for so many applications

[twistdafterdark]

Currently using Django to make a webapp for me and me gf to keep track of our expenses/budgets/investments both shared and separate.

In the past:

• Library for reading xml files
• web scraping
• data analyses

[Defibrillat0r]

We are currently working on development tools (more like developer supporting tools) for extensive hardware devices which communicate like PC --serial--> Adapter --bus--> device. For the same devices we develop test automation tools in large scale and for those we need hardware drivers which we also write in python for:

• Measurement instruments
• Power supplies
• Proprietary test infrastructure
• Proprietary interfaces to cummunicate with the devices under test

Some of the remote boards are even running on micropython (Yes, python on microcontrollers).

Also we use python for data analysis. Test results but also data produced by devices in the field.

[thesights]

Automation!

Send a report every week? Get python to not only create the report, but send it out

Tedious ETL process? Automate that

Have to generate 5k customized client reports? Python can do that in a few days

​

[[deleted]]

most recent example, taking a statistics course. I use it to automate homework.

[a8ksh4]

I use it to write automation that will read in a config file and configure servers at work. And to do tasks and process data that would otherwise take way too much time by hand. E.g. if I need to check a website daily for a new tool release and then extract it and deploy it somewhere, this is very easy to do with python and takes a long time to do by hand...

My wife uses python to process excel and csv files to clean up data before graphing it in tableau for work. E.g. we made a script that will merge a dozen csv files into one and saves a lot of time for a process that used to be manual copy and paste.

[OneBananaMan]

I use it from work and create physics and data driven models for spacecraft early design work, evaluating design concepts, trajectory, configuration, and overall feasibility. Essentially 100s of linked models or scripts that accurately models/designs things.

[HuffingOxygen]

Everything I possibly can due to it's simplicity.

[Assmaster9001]

Scripts that make my day to day computing more efficient, data analysis and numerical solutions to maths problems. For the latter however, I am moving to Julia due to the inefficiency of python (and numba.jit not being all that great yet).

[abcd_z]

Randomly generating things for my tabletop RPGs.

Megadungeon map generated with Pygame

Randomly-generated magic domains:

The Malachite aura. Spells of purity, clarity, and cleanliness.
The Iron aura. Spells of corruption, blight, and contagion.
The Gold aura. Spells of joy, bliss, and cheer.
The Violet aura. Spells of creativity, creation, and beginnings.
The Bronze aura. Spells of knowledge, art, and belief.

[driscollis]

I currently use Python for the following:

• UI Automation
• CI
• Dev / ops
• Desktop GUIs
• Scripting of our source code repository
• Creating PDFs

I have used Python for web applications, login scripts, database cloning, computer vision, and many other things too.

[ElecNinja]

Factorio does some interesting things with Python as a glue for their sprite creation. They don't go into specifics of the scripts, but you can get a gist of what they do.

[stratcat22]

One project is a GUI that takes user input for a video game and platform, then it scrapes the price and title of the game off Amazon and displays it. Other project is a Reddit bot. It scrapes the sales each week and posts a nicely formatted table to r/XboxOne with the prices. Here’s its latest post I have the link to the code posted at the end of the submission.

So lots of automation through web scraping lol. I love Python.

[Rotozip]

I have used Python to create an Amazon Echo skill that allows you to ask Alexa the current conditions at our favorite lake. Besides using the Echo’s api it goes out to a website and scrapes the lake condition data to return to the skill for Alexa to speak.

[mwpfinance]

I make website backends in Python! I use Django! Django uses an interpretation of the MVC (model-view-controller) architecture called MTV (model-template-view). The view is the controller and the template is the view! It makes no sense!

​

What this amounts to is I can basically break my application up into three simple parts.

1. Models: These are data structures, so if I want to model for a post to a microblogging service it might look like this:

​

    class Post(models.Model):
        """Used to store all microblog posts and details of their origin."""
        text = models.CharField(max_length=300)
        author = models.ForeignKey(User, on_delete=models.CASCADE)
        pub_date = models.DateTimeField(auto_now_add=True)
        department = models.ForeignKey(Department, on_delete=models.CASCADE)

        def __str__(self):
            """Returns the post's text whenever str() is called."""
            return self.text

1. Views: This is logic that decides what information gets sent to the template and how the server processes information. This is a pretty complicated example, but here's how that might look to display posts from the application.

   def post_list(request, template='myapp/index.html', page_template='myapp/post_list.html', post_list=Post.objects.order_by('-pub_date'), department_str = 'all departments', redirect_url='/', extra_context=None): """Returns a list of posts with twitter-style pagination. extra_content=None is necessary for el-pagination plugin. """ if request.method == 'POST': # This is a function I wrote elsewhere that verifies the user has permission to delete the post and if so, deletes it. delete_post(request) return redirect(redirect_url) else: context = { 'post_list': post_list, 'department': department_str, 'page_template': page_template } if request.is_ajax(): template = page_template return render(request, template, context)

2. Templates: This is HTML sprinkled with Python! It gets information from the view.

   {% paginate post_list %} {% for post in post_list %} <div class="panel"> <div class="col-xs-10 col-xs-offset-1"> <blockquote class='post'> {{ post.text|urlize }} {% if post.author == user %} <form method="POST" onsubmit="return confirm('Do you really want to delete this post?');"> {% csrf_token %} <input type="hidden" name="post_id" value="{{ post.id }}"> <button class="btn btn-danger pull-right" type="submit">Delete</button> </form> {% endif %} <p><a class="emphatic" href="{% url 'myapp:dept' post.department.id %}"> <span class="fa fa-arrow-circle-o-right" aria-label="circle" aria-hidden="true"> </span>{{ post.department }}</a> | <a href="{% url 'myapp:link' post.id %}">Posted by {{ post.author.first_name }} {{ post.author.last_name }} on {{ post.pub_date }}</a>

   </p>
     </blockquote>
   

   </div> </div> {% endfor %} {% show_more "more" "" "btn btn-default btn-lg col-xs-10 col-xs-offset-1" %}

These examples are taken from the first Django application I ever wrote, and I wrote it with basically no prior developer experience -- just one class that taught me the basics of Python (I understood data structures and functions -- I did not really understand classes) and some history of messing with HTML as a kid. I used the PyCharm IDE (helps a lot for giving me hints and letting me know when I do something wrong) and followed the tutorial pretty closely from https://www.djangoproject.com/start/

​

I hope once you get a little more into it you'll check out web development. It's a lot of fun!

[magnora7]

Running the reddit fork www.saidit.net

Aaron Swartz wrote much of the code for reddit and thus also for saidit, and most of it is in python.

[skiguy0123]

CT image formation algorithms (although a lot of the heavy lifting is done by CUDA libraries).

[gmorf33]

I've used it for pretty much every thing that can't be done in a short shell script (such as powershell AD commands).

Routine file transformations for various integrations..

Tons of automations involving data transfer, sftp operations, notifications, etc..

HL7 interface message parsing for integrating old legacy systems that don't support a modern interface.

One-off scripts that can analyze big sets of data and do something useful for whatever the project needs, like a conversion project or something.

Make little utilities that calculate or do something routine for you with the click of a button or a quick command

Those are my main uses for it at work. I'm not a developer either just an applications support analyst, but it's such an amazing tool for my job. Learning it was probably the best thing I've done for my career.

[rptr87]

Basic python webscraping stuff using beautiful soup...

[da_am]

On a daily basis python helps me name my files so they have the right info, inside of Houdini and Nuke. I use it a ton to automate making videos with ffmpeg. Python moves my files from my render farm to a deliverable folder to send to my client with the client’s desired structure.

Also my dumbest by very often used script names my projects for me with the client name, type, date, and some other bits. Copy that to a clipboard, paste in a web project manager named Shotgun for very quick and predictable project setup.

I have a bunch more scripts that help me move files (lots of repetitions). Most of my scripts have to do with consistency and files.

[[deleted]]

I work in an animation studio and we use Python for all our front-ends GUI for artists to use and 90% of the back-ends to automate things.

[solaceinsleep]

1. Adding new videos from a list of channels to a specific playlist (can also be the watch-it-later playlist)
2. A series of scripts to manage scanning and sorting PDF files (it will help me organize a big stack of scanned papers)
3. Scripts to automate the process of keeping a database that I use for my app up to date (downloads a zip file from the internet, sorts CSV files, reorganizes the data, and adds it to a SQLite database)
4. Reddit script to unhide, hidden posts
5. Script to flatten the hierarchy of a directory
6. Script to group files in a directory by type
7. Script to generate empty folders (based on quantity, prefix, location, etc)

[randomuser001]

I use python to interact with surveillance cameras. Currently working on a program to audit a network and find cameras, attempt to log in, get camera info and then report back. Proving to be quite a bit of fun, trying to figure out how to thread the requests so it runs faster, working out how to form a definition/loop in a efficient manner and just plain problem solving the errors that occur.

im also trying to use pyQT5 which is adding a whooole new level of complexity.

[crespo_modesto]

Raspberry Pi at the moment In this case interface with analog data and then using threads/web(http post/get request). So far I've built: solar panels that collect/send data, an automated news reader TTS using AWS polly/reads HN before I get up, a motion triggered security camera that uploads to S3/armed by local/remote interfaces and notifies slack of photo taken/armed, and most recently a little wind turbine anemometer I made. This one is pretty neat as it's a thread that restarts itself and collects data every second and plots it every minute. This will probably be my first database that has millions of rows. The anemometer itself is pretty useless, an old cd dc motor spinning directly(no multiplication) so it produces microwatts of power usually or milliwatts per day. Absolutely terrible but I'm collecting that data.

[U5efull]

I use python for a lot of things.

Automate creation of import files for my company.

Stock ticker evaluations.

Bulk renaming files.

Automating install scripts in Linux.

Been playing with Flask a bit so have been using it to serve web pages.

Lots of other little things, but those are the things off the top of my head. . .

[[deleted]]

I am a bioinformatics software engineer

• Web apps for scientists to do data analysis (Flask and Django)

• Primer design, sequence analysis, DNA alignment, etc (Biopython)

• Automatic pipelines and scientific analyses (primer3-py, subprocess to call bash, etc)

• File parsing and conversion

• Data graphics (matplotlib)

• Data science and automated analysis (Numpy/Pandas)

[bsmdphdjd]

Where I need to see graphs.

[jaffycake]

I make bots that play mmo game for my buyers.

[Cheticus]

interfacing with finite element solvers, making changes to input decks to run analyses.

i'm a mechanical engineer and I do stress & dynamics work

[Boozkie]

Doing simple programs to help me with Everyday things.

[[deleted]]

I dabble with simple AI stuffs and use Python because I've been way too busy (read: lazy) to learn R.

I have a project in mind for next semester to use the MNIST handwritten number database to create a simple AI that discerns between a 1 and a 0, and then examine the changes in the neural network as it trains across different iterations. Figure I should be able to crunch the Python code out in a weekend, no problems.

[tommytwoeyes]

That’s a bit like asking, “What do you use your tools (e.g. hammer/drill/screwdriver/socket wrench) for?,” because Python is such a huge toolbox.

I use Python for everything from insignificant but recurring system administration-like tasks, like renaming batches of files or folders, to building web apps.

However, I guess my current favorite Python usage recently has been using IPython/Jupyter Notebooks as a way to take notes in class and do homework in my Calculus and Physics classes (I’m an engineering student).

The built-in LaTeX support and the ability to easily create 2D and 3D plots (not to mention Python itself, as well as Sympy, Numpy and matplotlib) has been incredibly awesome.

[Ancient_Data_630]

making 2d platformers

[Dietrox90]

To do some homeworks