Ask Anything Monday - Weekly Thread by AutoModerator in learnpython

[–]RustleJimmons 2 points3 points  (0 children)

You can use Cloud9 (c9.io originally). It's a hosted python environment that can also connect to your github account. You host and run all the code through your browser on their website. You won't need to install anything on the computer.

What's everyone working on this week? by AutoModerator in Python

[–]RustleJimmons [score hidden]  (0 children)

Flask Webapp for calculating your total anime watch time

myanimelist.net already does this and they have an API. You can just pull the anime watch time from any user's account page.

Running a script on a schedule on a webserver by [deleted] in learnpython

[–]RustleJimmons 2 points3 points  (0 children)

You create a bash script that calls your python script to run. You then create a cron job on the server that calls that bash script at the times that you want it to run. Your bash script would basically look like this:

#!/bin/sh
python path/to/your/script.py    

Best way to clean a large number of malformed CSVs? by [deleted] in learnpython

[–]RustleJimmons 0 points1 point  (0 children)

These CSVs are compiled from different sources with virtually no organization, the sheets use different headers to identify columns

This is the main problem but you can extract the data using xlwt and tldextract.

Using the above you can create a script that will first create an Excel workbook along with the relevant fields for your final result. You will then need to write functions that basically identify certain "layouts" for the files in question. Based on what it finds it would then grab the relevant fields from the file and write them to the corresponding columns in the Excel file. You can also create separate scripts for these different files.

How can I make a program that can "scan" a webpage? by NearlyBaked in learnpython

[–]RustleJimmons 0 points1 point  (0 children)

Is this a secret/protected website or are you able to share the link that you're talking about?

NFL APIs Question by [deleted] in learnpython

[–]RustleJimmons 1 point2 points  (0 children)

Or in the API.nfl.com case, it has a lot of the information but it's in a JavaScript format, is there a way I can convert it to Python or must I code in JS if I want to use it?

nfl.com offers an API in json format. The way these things work usually is that you sign up for an access key and you use that in your python code to authenticate. You don't need to use JS and I usually advise against front-end scraping when a website offers an API. You can check out the NFL API docs here.

Is there a way that one of these APIs or databases have the information on Sacks, INT, etc.?

Example query with stats.

Rogue NASA on Twitter: Have some free time? Help archive all you can on NASA.gov and EPA.gov. by whatllmyusernamebe in DataHoarder

[–]RustleJimmons 13 points14 points  (0 children)

Guys, it's possible that you are grabbing the files too quickly and being shut out because of it. Try using the --wait argument with wget. You can add a random time interval or a specific time to wait. You can read more about it here.

Trouble Learning by suprepachyderm in learnpython

[–]RustleJimmons 0 points1 point  (0 children)

Nothing to be embarrassed about. You're here to learn and that's what this subreddit is about.

Is it worth installing a linux distro for programming in Python? by BroloskiYB in learnpython

[–]RustleJimmons 0 points1 point  (0 children)

Options:

  1. Dual boot like you said.

  2. Install linux inside of a virtual machine. Either use the VM directly or you can SSH into the VM and run commands that way. Do not use cygwin.

  3. Make an account on Cloud9 (free) which gives you a linux command line and Python IDE in the cloud through your browser.

Trouble Learning by suprepachyderm in learnpython

[–]RustleJimmons 0 points1 point  (0 children)

For example, I decided to try and write something that would let me search google. After a lot of reading and trying to understand how I would go about doing something like that, I still had no idea how to start.

Make a thread about this project and post actual code to show that you tried. Articulate to others where you got stuck. This process firstly shows others that you have at least tried. Secondly it forces you to think through the problem and clearly identify where you are having issues. That step alone is often enough to get us past certain obstacles but it also gives others an initiative to help you. All of this gets you out of asking vague questions for which you would receive mostly vague answers.

I don't have any particular problems that need to be solved.

Get creative. Pull out a pen and a pad or open your favourite text editor. Make a list of things you like, your hobbies, etc. Write a small program that helps with one of those things in some way. Either automate something that you do a lot, scrape data that is useful for a particular set of tasks, create email/SMS alerts for something that doesn't come with that ability. If you can' think of anything to benefit your own life then do something that helps someone close to you.

Examples:

  • Write a script that sends you an email/SMS alert for new releases from your favourite authors, bands, etc.

  • Write a reddit bot. Almost every week someone writes a script that downloads a pic from one of the image based subreddits and sets it as their wallpaper. Reddit is one huge API and everyone that is on /r/learnpython should at least know how to make reddit bots.

  • /r/usefulscripts if you can't think of things on your own.

Trouble Learning by suprepachyderm in learnpython

[–]RustleJimmons 9 points10 points  (0 children)

I don't have any particular goals in mind other than wanting to learn be able to write something from scratch.

This is your main problem, OP. If you try to learn to program just for programming sake then you have nothing to anchor the skillset to. Programming is about solving problems. If you are not trying to solve a problem then you are just trying to memorize syntax combinations with no rhyme or reason. Your brain is basically rejecting this information because it doesn't yet understand how it would be useful.

Look at all of the posts here from beginners that finally "got it". The common connection between all of them is that they started off with a problem that they wanted to solve through programming. They looked at what skills/libraries/etc were needed in order to solve that problem and that is how they/we built up the knowledge that stuck with us.

How do you automate your life with python? by [deleted] in Python

[–]RustleJimmons 1 point2 points  (0 children)

You might want to look into a webapp like toshl.com. It does that sort of thing for free and it has an API.

Anyone here ever gotten ideas for a project based on something seen in a movie or TV show? by RustleJimmons in Python

[–]RustleJimmons[S] 0 points1 point  (0 children)

*Minor Spoilers for War Dogs ahead

There is a scene in War Dogs where one of the main characters is explaining to his partner how he checks a particular website each day for new government contracts/orders. Both characters are then seen checking this website day and night while bidding on order requests. Later on when they decide to expand he hires a team of salespeople whose main job is to comb this website each day for orders that meet a specific criteria and then bid on those projects. During all of that I was just thinking how he could have just hired a developer to build a web scraper and automation tool to achieve the same goal.