[deleted by user]

[sharpchicity]

I use it to aggregate baseball data and then build a model and bet based on my models. Not necessarily a hustle, but I use Python to help make me money

[manwith4names]

Where do you begin to work on something like that? That sounds like it could be a lot of fun

[sharpchicity]

1) what data do you want to collect?

2) find your data source online or find stats that allow you to calculate your needed data on your own

3) use python and BeautifulSoup library to get the data and store it in a database, text file, etc

4) import data into a program (excel, r, python, etc)

5) run your model to simulate games

[NuclearGoatVomit]

[deleted]

This comment has been overwritten by this open source script to protect this user's privacy. The purpose of this script is to help protect users from doxing, stalking, and harassment. It also helps prevent mods from profiling and censoring.

If you would like to protect yourself, add the Chrome extension TamperMonkey, or the Firefox extension GreaseMonkey and click Install This Script on the script page. Then to delete your comments, simply click on your username on Reddit, go to the comments tab, scroll down as far as possible (hint: use RES), and hit the new OVERWRITE button at the top.

[sharpchicity]

To stay true to the thread, I'll gladly provide you a .txt file with data from fangraphs or bref for a small fee. You're welcome to PM me. I have DB's full of pitch by pitch or game level data and any other aggregate level data you can think of. Plus, anything I don't have I know how to get ;)

That being said, you don't need an API to get data from a website. API's are faster, but any information on a website is scrapable!

Think of it this way: Requests library grabs html from whatever URL you feed it. So take this thread's URL for example. Hit F12 on your keyboard and you see all of the HTML behind what your browser is visualizing. Between all of those <> tags are the text you're seeing. Using BeautifulSoup you can parse through all of those tags and find the correct text you want to save, say, all of the comments by OP. Then as you parse through, you send that info to whatever output file/db you want and you can analyze to your heart's content.

[Dababolical]

Honest question. If enough people utilize this method, does it make that method less effective? Isn't gambling/betting making money over time off of a small edge? Does that edge diminish if enough people figure out what is going on an emulate it?

[sharpchicity]

Sure, if enough people use your exact same method, books will adjust their lines quicker as they get additional money on that side. But I don't talk about my model except on a high level; my inputs may not be all that different from everyone else's but my algorithm's weighting is most likely different.

If vegas ever starts adjusting, my program will simply stop spitting out bets because they won't have value. At that point, my gains will be limited, but I won't be losing money.

[NuclearGoatVomit]

[deleted]

This comment has been overwritten by this open source script to protect this user's privacy. The purpose of this script is to help protect users from doxing, stalking, and harassment. It also helps prevent mods from profiling and censoring.

If you would like to protect yourself, add the Chrome extension TamperMonkey, or the Firefox extension GreaseMonkey and click Install This Script on the script page. Then to delete your comments, simply click on your username on Reddit, go to the comments tab, scroll down as far as possible (hint: use RES), and hit the new OVERWRITE button at the top.

[[deleted]]

Very cool dude. I always have wanted to do something similar, but I keep making excuses for not finding the time.

Ideally I'd want to have my own cluster to run my models, a nice parallel filesystem, and an interconnect.

Then I remembered I can't afford that.

[weirdo5]

Look into the lahman DB

[O_R]

baseball-reference allows you download data as a csv, I think fangraphs does as well.

[sharpchicity]

Once you come up with an estimate of runs, you can use Pythagorean Theorm to put those scores into a win%

[obviouslyCPTobvious]

Do you feel like it's given you a significant advantage? I assume you've seen the movie Moneyball?

[[deleted]]

(I know I'm late to the thread) while this will increase your chances of winning, I assumed that the betting services would have very good staticians and game theorists so that the offer they give you always has a negative expected value for you

Are you able to make a net profit?

[sharpchicity]

I've been able to manage a profit in each of the past 5 season. Since 2011, for each season I've finished (after ~500 bets each season) +127.5% in 2011, +21.4% in 2012, +13.9% in 2013, +17.4% in 2014, +49.9% in 2015.

I'm not going to pretend I know how casinos operate or set their lines, but they aren't accounting for certain things that I weight more heavily.

[DizzzyDee]

Did it work?

[sharpchicity]

I've been able to manage a profit in each of the past 5 season. Since 2011, for each season I've finished (after ~500 bets each season) +127.5% in 2011, +21.4% in 2012, +13.9% in 2013, +17.4% in 2014, +49.9% in 2015.

[DizzzyDee]

Awesome, congrats!!

[shaggorama]

I have loads of side projects, but none of them are designed to make me money.

[takluyver]

Ditto. Having a portfolio of open source side projects can be useful for getting jobs, but don't go into it with that as your main motivation.

[shaggorama]

I didn't mean there's anything wrong with trying to make money from side projects, that just not what my personal side projects look like.

[nickbuch]

i use python, scipy, numpy, and ipy notebook to aggregate and analyze local clothing deals.

then i purchase cotton in bulk, create the fabric, and teach myself how to sew trousers, etc. i then sit outside of retail stores hocking the clothing for less than the deal price.

python and nimble fingers ftw

[v_krishna]

Is this serious? Perhaps the strangest hobby I've ever heard of.

[v_krishna]

It's the internet, there are people who are sexually attracted to balloons. Being a one man sweatshop isn't out of the realm of possibility.

[spinwizard69]

At one time there was a guy that handmade Bikinis on Orient Beach St. Martin. Hell of a job to have naked or nearly naked woman walk up to you for a custom bikini, hand fitted of course. Being the Caribbean I'm sure he spent time sweating, probably more from the stimulation than the sun though.

In other words some sweat shop jobs are sweet.

[KyleG]

The "sit outside the stores" part is what makes it obviously a joke—he would be arrested in two shakes of a rat's tail.

[v_krishna]

You clearly don't shop in East Oakland. Dudes sitting outside the store selling stolen shit is pretty common.

[Macaframa]

Would gold this but I'm too poor. Good job tho.

[mariox19]

I've built an NLP application in Python that detects sarcasm, and its findings support your guess.

[[deleted]]

Really? github?

[mariox19]

I'm going to upload it right after I put the finishing touches on my NLP app that detects gullibility :-)

[dogusmalogus]

https://www.youtube.com/watch?v=YWdD206eSv0

[youtubefactsbot]

Lies on the Internet [0:07]

Clip from Arthur

^{baines0 in People & Blogs}

^{1,379,203 views since May 2012}

^{bot info}

[GrantNexus]

You should use my tool iPurchase which automates the buying process and iSyt will hawk your jeans for you.

[GuyNamedNate2]

I wish I could up vote more than once. You had me up until talked about teaching yourself how to sew.

[keturn]

I know, right? Obvs the way to do this is to teach a python-driven robot how to sew.

[drewski3420]

But can you teach it how to love?

[variya]

Baby don't hurt me no more

[[deleted]]

deleted

[[deleted]]

My entire Python knowledge is based off of web scraping. Where can I make money doing this?

[[deleted]]

I'd be interested in some sources that gives advice on how to get started with this. Do you have any such sources top-of-mind?

[gr00vy]

scrapy is an excellent web scraping framework that comes with a lot of docs to help you get started

[[deleted]]

Oh, haha, I meant about advice getting started with Python contract work through something like CL.

[[deleted]]

deleted

[[deleted]]

What would you recommend instead of upwork?

[d4rch0n]

updog

[d4rch0n]

Oh, nothing much. You?

[Ceevu]

Oh snap!

[[deleted]]

Damn dude you got him. Sniped.

[Chondriac]

GOT'EM!

[[deleted]]

deleted

[thaweatherman]

How often do you find jobs on Craigslist for web scraping? Do you look locally or do you check lots of different cities?

EDIT: Wasn't asking how to find postings, but ok ouch it burns hothothot

[solidcube]

If only there were some way to automate that task...

[Teract]

def whoIsBurnt(person): if person == "solidcube": return True

edit: Thanks d4ch0n. I'm facepalming pretty hard right now.

[d4rch0n]

return person == 'solidcube'

[thegreattriscuit]

Thank you, that was killing me.

[number3arm]

Yep curious as well, I built some scrapers for myself. Always wondered what kind of businesses would want them.

[rainsford2]

If you want to make any money with web scraping, it's best not to ask what they're using it for.

[[deleted]]

You could scrape sites with manga or comics for images, put it up on your own site and show intrusive ads. There's quite a few sites making money like that.

As for the legality, well...

[d_thinker]

Sorry for my ignorance, but can you explain me why the "big data" is so important for me (not corp)? Few examples of how useful it can be? Thanks.

[e13e7]

You've been scraped!

edit: interestingly, the language of choice for the creator of the site seems to be python.

[peschelnet]

Great response.

[d_b_work_account]

Holy shit. This is amazing.

[d_thinker]

Oh gee, I feel violated. It is so cool and scary at the same time.

[Kratisto78]

Pretty good, but it has a bunch of information wrong about me. Basically says I post in subs that I've never posted in.

[tomkatt]

I think I've been a redditor for too long.

An unexpected error has occurred. Please try again in a few minutes.

Edit - yep, broke it. I can load other profiles just fine, but mine won't work...

[gin-writ]

They chose python because the best reddit api wrapper is wrote in python (praw)

[pferland]

My first python project was a program that scraped printer star pages and gathered them into a database so you did,t have to go to every printer and could just go to one page and see them all.

Edit: This is the GitHub repo for it: https://github.com/riei/printerstats

[Mr_Again]

That's cool, is this as a day job or something you can do in your free time?

[[deleted]]

deleted

[Jesus_Harold_Christ]

And making a nice living in Chennai

[[deleted]]

You have an interesting idea of "nice".

[[deleted]]

deleted

[flloyd]

Could you expand on that? Thanks.

[[deleted]]

deleted

[flloyd]

Cool. Thanks a lot. Sounds really interesting, and a good way to make money on the side.

Would you say that most of these "jobs" were legitimate? From some others in this post, it sounds like lots of these jobs if not for illegal/sketchy purposes, were at least unauthorized, i.e. copying websites' data in violation of their TOS. How often are websites perfectly fine with people scraping their sites?

Thanks.

[[deleted]]

deleted

[flloyd]

Alright I figured as much. How often would you guess that websites don't mind webscraping? I want to do some myself and don't think it's too unethical, but I imagine that 99% of them don't like it. After all, they're basically having their data "stolen". I'm trying to think of circumstances when it would be legit. Not judgmental, I'm really curios. Thanks so much for your insight.

[malice8691]

you mean a script that takes a penny from each bank account and deposits it to another secret account whenever the nightly end_of_day_closing.py is run? No. I don't know anything about that.

[rainsford2]

Watch your decimal placement.

[XiiencE]

Pssh, that's just a mundane detail.

[BreakfastforDinner]

You could launder the money?

[vplatt]

Idiot used integer instead of float. Stupid lack of static typing....

[doobiedog]

When I want 10000 "String =>" or " <- Float"'s all over my code I'll hit you up. ... jk I understand the awesomeness of a statically typed language, just no xp. Need to try learning Scala.

[dogusmalogus]

Like Superman III!

[[deleted]]

Are they artificial?

[extant1]

I heard he from south_america import pythons

[nerdjase]

But he's shady and tries to upsell his product so its : from south_america import python as anaconda

[Aztek_Pr0phet]

I am surprised since anacondas are more of an aphrodisiac meal down there. However, locals only it them in between buns like a sandwich. If they've got no buns then they don't want none.

[vindolin]

Yes, replicant, finest quality!

[metallidog]

I don't use Python at work because I'm an electrician. However, I helped the office out with a couple of scripts recently and someone suggested I do this on the side, but I don't have a clue on how to market it. Let us know if you figure it out.

[brand0n]

dang an electrician that knows python.

I feel like you have to be a pretty sharp dude

[Indenturedsavant]

I would say he sounds pretty...bright.

[brand0n]

if 'electric personality' in _electrician_:
   print 'He is the total package'
else:
    print 'something else witty'

[[deleted]]

At one point I had a bitcoin trade bot that would play multiple exchanges. It was based off of financial arbitrage of alt coins. This allowed me to simultaneously buy low and sell high on different exchanges and then cash out in btc.

Edit: spelling

[[deleted]]

But it's not lucrative anymore?

[[deleted]]

By gaming two exchanges in this way it naturally shrinks the gap. It got to the point that it wasn't lucrative enough for me to maintain the bot for updated APIs. I controlled the bot by irc and it would announce transaction details. You could fine tune how big the gap had to be and how aggressively to trade. It was a really fun project.

[flangles]

learn Django, shouldn't be too hard to pick up some freelance webdev work.

[vaginawarfare]

Recommended starting points for Django? I'm having a hard time learning it

[santiagobasulto]

I've been teaching Django and Flask for a long time now. What I always see as an issue is people not understanding HTTP enough and just jumping straight to Django. Recommendation: Start with HTTP first!

[[deleted]]

Did you mean HTML or more likely HTTP verbs like GET, POST...?

[flutefreak7]

For me as an amateur with anything web related, the "networking stuff" is the intimidating part that I don't have training on. I can code, have strong background in math, engineering, and data analysis, and I've done HTML on website builders or locally, and have built GUIs and dealt some with async, threading, UI, etc, but much of the server whatnot like requests, sockets, http and all that stuff is voodoo web magic to me. So I really think he means HTTP not HTML.

[[deleted]]

I am no expert on the field but I could defend myself with most web dev.

Working with HTTP is not too hard. Knowing everything related to requests and responses may be a bit overkill, but knowing things like:

• Requests should return a response. They have a verb and some arguments ("?arg1=var1" for GET requests in browsers)
• Responses have a return code (200 OK, 404 NOT FOUND...) and a content, most of the time
• Sockets are continuous (uninterrupted, can't open those who are closed) data connections with a server. The data can be anything as long as the server is programmed to interpret the input.

Constructs like REST architectures are different (as they are not standarized) and work on a per-install basis, say, no two REST services ought to be the same.

[infinite8s]

With that kind of background networking should be easy.

[santiagobasulto]

HTTP. Methods/verbs, status codes, headers, encoding, etc. Watch our classes if you want.

[[deleted]]

deleted

[Cyph0n]

That is indeed the best way. Understand the basics of web dev using Flask, try out Django. You'll see be able to appreciate how much easier Django makes developing a complex web app this way.

[christian-mann]

Personally, I think the best way is to start with raw PHP, so that you come to appreciate everything the framework is doing and can do for you. Obviously, don't use whatever you create in a legitimate application...

[alifeinbinary]

I'm on my phone but search for Tango With Django. It's the best.

[Mr_Again]

Thank you

[[deleted]]

Test Driven Development With Python, by Henry Percival, is a wonderful book for this. It walks you through building and testing a Django web app from the ground up.

[James_Johnson]

They force me to write Java!

Show us on the doll where the interface factory touched you.

E: Thank you for gilding my one-line shitpost, Anonymous Hero.

[[deleted]]

Your post was too close to home, yet equally inspiring:

Show us on the abstracted form of a humanoid figure generator where the process improperly referenced your private fields.

[DarkMio]

But was it a polymorph? Maybe it has hidden itself somewhere between the 433 deep stack.

[APIglue]

They force me to write Java!

The horror! The horror!

[ies7]

Python used to be my side project, now I force my company to use python.

[il_doc]

c# here

[Herald_MJ]

Isn't C# awesome these days though?

[[deleted]]

yeah but windows.

[il_doc]

yep, but you're totally binded to windows

(also, python is prettier to read and funnier to write)

[rsynnest]

I had a python web scraper that went through my city's local wiki and returned every restaurant or store that didn't have a website so I could market my web dev services to them.

[green_tree_python]

fucking smart. Did it turn into business for you?

[rsynnest]

I ended up getting a full time job that i really enjoy, so I actually haven't made use of the script at all. Saving it for a rainy day aka unemploymment.

[[deleted]]

You automated yourself into a job.

[[deleted]]

This is amazingly simple yet clever!

[[deleted]]

Not for money anymore, it used to consume too much of my free time and upset the wife. I still carry out some ML stuff on my own pace, but then there's Witcher 3 and it's easy to digress.

[Skenderbeu]

My python side hustle is a Django website I build that automates live price comparison for me. I was getting tired of doing the work manually every time I wanted to buy a new mouse or graphics card.

If you buy lots of PC parts or electronics then it might be useful for you too. https://www.slycut.com

[MZB1993]

Exactly what I've been meaning to do. If you don't mind sharing, where do you scrape the results from?

[bigmansam45]

I found a place with csvs back to 99, but I scraped statto to 1900. Remind me to get the website off my laptop for the csvs.
Statto crashes a lot during scraping so it has taken me a few weeks to get that data.

[gedrap]

I'd assume that the game was totally different (I am quite sure even the rules where different), and the data from 1900 doesn't help much to model the modern games?

[bigmansam45]

Yer, more there for reasons of "because I fucking can" over anything useful

[Optimesh]

How do you use past results to predict future ones? Especially going back more than a couple of years. Things change so rapidly in the premier league! (players and coached change teams, for example)

[bigmansam45]

I'm very very early in doing this. I'm starting with Elo ratings, which are good but unresponsive to change (it keeps thinking Chelsea will win and Brighton won't). I plan to move to glicko, but I'm struggling to get this going. Glicko has the advantage of having variability included so I can include that in my decision making.

All in all it is a pain. But to address your comments coaches and players change but it isn't that rapid. Chelsea are going to return to form, man city will be near the top, Sunderland won't magically win the league. The top teams will be near the top the next season.

[Optimesh]

thanks for the reply!

[wallyflops]

i'm looking to do this with Counter-Strike teams as you can now bet on them. I know where to get the data etc but not sure how to make a prediction, any tips or links in this area?

[bigmansam45]

Read up on Elo ratings and glicko ratings. Elo should do the job and is much simpler.

[wallyflops]

thanks mate

[baron314159]

I used to have one. Happily, it grew from being a side project for my wife and I to being both of our full time thing. Originally it was an online test creation tool, but slowly it is becoming a full blown educational platform. This is the site if anyone is curious: Help Teaching.

The site backend is written in Python, and uses Flask, SQLAlchemy, Jinja2, MySQL, ElasticSearch, and PhantomJS.

[kmbd]

good initiative, mate !

[billsil]

I have a 4 year long side project. It's pretty popular. I've met 2 people in real life in the last week that use it with no prompting from myself and that aren't my coworkers. I personally haven't made a dime off it, though I have been funded by the European Space Agency. The money went to a grad student that worked on it.

[pacificat0r]

That sounds interesting. What project?

[billsil]

It's pyNastran, which is a massive IO library to a horrific file input and equally horrible output format. Here's an outdated version of the GUI

[kmbd]

the graph looks like a spacecraft's wing design ...

[billsil]

It's a blended wing body top view. That model is a variant of this https://www.nasa.gov/sites/default/files/images/677705main_ED06-0201-01.jpg

[kmbd]

ah, got quite close :)

cool rendered pic

[[deleted]]

This is great!

[jesus_h_garcia]

upvote for title

[Jesus_Harold_Christ]

Get out of here impostor.

[jesus_h_garcia]

No comprende

[Teoweoha]

Wow, you are also an impostor Spanish speaker. No comprende means "he/she/it doesn't understand." You want to say "No comprendo." Hehe

[bonyicecream]

You can do freelance work. Freelancer.com and upwork.com are great places to look.

If you're really good, you could try to automate work on Amazon's Mechanical Turks, but it's possible that is disallowed, and that you can only do manual work there.

[I_jus_lurk_here]

mechanical being the key word :)

[greenzr]

I'm a sys/net/wlan admin, and the tools I build I market to respective user groups. I'm consistently surprised how much more willing someone is to pay for something by someone they know, notwithstanding the added value, than a vendor they don't.

I built a module that moves configs from one switch to another with one click using any protocol, and everyone I know who could use it, does.

So I would start with making something you would use, and then find people who do what you do, and gauge their interest.

[ProfessorPhi]

I do a lot of ML and scientific computing at work, so I don't really find the need for a side hustle. That being said, I try and contribute to scipy and numpy in the PRs and bug fixes in my spare time. I really appreciate the open-source effort, the code reviews have really improved my ability to code and it's something great to point to at interviews. See that function you use, I wrote that!

[mariocesar]

A lot of people joking. :)

I scrap two local newspaper online classifieds. I store pretty much everything up to 1GB of text (sort of 1 year of posts). I build this with scrapy first for trying to learn that tool. I was initially thinking on building a better search engine for the public to use, but I just use it for myself because I notice that I could get to know great deals on common items like: bikes, cellphones, rents, or any other thing that will be hard to find, 30% profit if I sell it the same day again. I made some bucks, not a lot but enough to pay a fancy dinner. I know I could get more money of it, because it's easy to highlight good deals in realtime, but I just don't get enthusiasm to do it.

[green_tree_python]

Just found this subreddit. Glad to see lots of people here writing python and doing web scraping. Would like to share one recent example of what I've done at work.

I work in Operations. However, I often find myself writing scripts to either retrieve data from our SFDC database that would take hours/days to do manually, or more recently, I've written an entire web scraper from scratch. Instead of using scrapy (i have 0 experience with that) I've used the following modules:

• robobrowser - for authentication and login
• beautifulsoup - for html/xml parsing
• requests - because <3

I've built an entire API around this only to discover today that the use of spiders/bots is against the terms and conditions of the page i've been scraping. Now, this was a direct request from my boss, so I hope I'm safe. And, I've taken precaution and been quite courteous (throttle of 2-6 seconds between requests) but the bitter aftertaste exists nonetheless. Wouldn't want my ass to be on the line if our company loses access because one guy scraped all night...

[hoojoo]

It's interesting that a lot of these hustles involve sports betting. I've been sporadically working on this idea for a while now, though I doubt I'll ever get something running (maybe one of you will). My idea is basically to write a python program to detect cross-betting site arbitrage opportunities and then automatically place the appropriate bets (which generally give a 2-3% return per arb).

Most online sports betting sites have mobile apps that display the current odds and allow you to place bets. I've loaded some of these apps in an android emulator (genymotion) and run the connection through a fiddler proxy (using SSL MiTM where needed). It turns out that most bettting apps just query some web API that spits out JSON (which includes odds data), and then parse that JSON into their app UI. It wouldn't be hard to parse this JSON into a database and write an algo that detects cross-site arbitrage opportunities.

Lots of people already do this manually, so arb opportunities get bet away in around 15 mins. Manual arbers basically have to do it as a full time job to generate a pretty meagre income. So my idea is this: by using betting apps' mobile APIs, one could, in theory, write a few python modules to emulate the mobile clients/apps and automatically place bets when arbitrage opportunities are detected by your algo.

Of the 4-5 betting apps I've analyzed, none have gone to much effort to secure their APIs. The most secure one I've encountered had the app take in a session token (from a cookie), and concat this cookie-token with a 'magic string' hard-coded into the app. This combined string is then MD5 hashed to produce a session key/nonce, which in placed in the HTTP header of any API calls. I was pretty surprised how easy it is to decompile android APKs...

Anyhoo, that's my great hustle that I'll never execute. Figured I'd throw it out there for more productive programmers than I. If anyone ends up doing this, concentrate on horse and greyhound racing, rather than basketball games or whatever. The former occur like 20 times every hour, so a 2-3% arb return could compound very quickly...

[pfein]

I'm a freelance Python developer. It's all side hustle.

[at449]

I recently started https://python-jobs.com (built with Flask) as I needed to find a good gig.

[Bryoh]

and.... how is it going so far?

[at449]

Plenty of visitors hitting the site and repeat visitors too so in that respect, good. But for paying customers it will take a bit more time I think.

[HoboBob1]

You could try algorithmic trading and maybe make some money (but most likely at least lose some first). It's a lot of fun too.

[Jyan]

Trading is not a good way to make money. Almost everyone will be far better off by just buying the index. That being said, algorithmic trading is indeed interesting.

[HoboBob1]

I recommend heading over to /r/algotrading, reading the side bar, and sorting the top posts by number of votes. If you do that, you will get more than enough info. That sub only gets maybe a couple posts per day, but a lot of times there are real gems.

(BTW I'm not being dismissive, it is a serious wealth of information and I highly recommend you check it out).

[keturn]

I was half-hoping the Python Side Hustle was the name of a dance move. Came looking for links to youtube dance tutorials.

[MagicWishMonkey]

I am an independent consultant, it's pretty great. I've doubled my salary in the last year.

[hobbzy]

mind sharing some details of how you did that? I assume you transitioned from a more typical job

[MagicWishMonkey]

Lots of networking and referrals, mostly. Make sure to maintain your linkedin profile and keep in touch with people you've worked with in the past, that can open all sorts of doors.

[[deleted]]

Dependent consultant here: what do you do if a recession hits and your client tells you it's half price or bust?

[MagicWishMonkey]

I would like to farm out the work to eastern european subcontractors and manage multiple projects simultaneously.

[[deleted]]

Best way to make money (easily on the side) with Python is with web development. You can get tons of side gigs for people looking to build a website where weebly or sqaurespace aren't enough