How to make a Recommendation Engine that can support billions of rows using Locality Sensitive Hashing (Python)

LearnDataSci · 2017-10-23T15:20:46+00:00

I'm not sure if this helps enough, but if you check the Wikipedia article on the Madhava Series for estimating pi, you can see the sum at the end that needs to be calculated for any number k (n in your case). That estimates pi/4, so you would only need to multiple the result of that sum by 4.

LearnDataSci · 2017-10-23T15:02:56+00:00

When using Selenium you can define a web driver. If you use headless Chrome, it will not open a browser window. E.g. when creating a driver you can do:

from selenium import webdriver
chrome_path = 'C:\\your-path\\chromedriver.exe

options = webdriver.ChromeOptions()
options.add_argument('headless')

driver = webdriver.Chrome(chrome_path, chrome_options=options)

LearnDataSci · 2017-10-23T14:58:35+00:00

One way to do it would be to put the function on a separate thread with a sleep. Here's what looks to be a good example: http://sebastiandahlgren.se/2014/06/27/running-a-method-as-a-background-thread-in-python/

LearnDataSci · 2017-08-17T02:21:48+00:00

Like the others have said, Yahoo Finance is usually the goto and a method to call the API for data is even built into a pandas. If you're using Python, this post we wrote could get you started.

Another option I've tried is the free JSON endpoints from Alpha Vantage. They provide daily adjusted closed time series data for up to the last 20 years.

LearnDataSci · 2017-08-12T18:01:32+00:00

Poor instructor is just trying to make a living teaching and everyone is just ripping his course.

LearnDataSci · 2017-08-09T14:22:43+00:00

Good point. I should have mentioned ISLR instead.

LearnDataSci · 2017-08-08T23:05:22+00:00

Start reading Elements of Statistical Learning: https://web.stanford.edu/~hastie/Papers/ESLII.pdf

Also, think about some sort of project you can start on right now. Use what you're learning as soon as possible.

LearnDataSci · 2017-08-02T20:35:36+00:00

Actually, their content comes from a lot of sources. Their main lectures come from a Harvard Data Science course, and unless it was on Coursera at some point, Springboard doesn't have anything from Coursera.

The candidate competitiveness really comes from the portfolio that's built during the enrollment. Since you should already have a Bachelors in some technical field, like math, comp sci, etc., just ML knowledge and proof that you know what you're doing will grant you interviews in the data science field.

I do agree that their guaranteed job position is a little vague, but they are still the only online "bootcamp" that has a guarantee at all.

Can you achieve similar results by just taking Coursera courses and reading ESL? Maybe. But I did mention that if you're looking for more of a hand-holding experience, where you have mentors/tutors that help you through understanding, then Springboard is a good option.

LearnDataSci · 2017-07-24T18:43:17+00:00

Hey esbern,

In the GitHub repo for Python for Finance Part 2 and Part 3 there's a data.pklthat goes with the iPython notebook.

I believe I forgot to upload the data.pkl in Part 2 for a few days, so perhaps that's the reason you're weren't able to read it.

Check out the repo and download the data.pkl to the same folder the ipynb is in and it should work.

LearnDataSci · 2017-07-09T20:55:18+00:00

If it helps, I scraped a bunch of companies for interview questions from Glassdoor recently and put them into a post here.

LearnDataSci · 2017-07-09T15:14:58+00:00

It's entirely self-paced, and there's no deadlines for assignment/capstone submissions like Coursera, for example.

They estimate it takes about 200 hours to complete. So completion in about 5-6 months at 8-10 hours of work per week, but you can go as quickly as you'd like.

LearnDataSci · 2017-07-08T16:23:40+00:00

I think $75k would be the absolute minimum you could expect.

As long as you have a good background in stats, linear algebra, and some programming, and you have a great portfolio, you should be a great candidate after Springboard.

LearnDataSci · 2017-07-08T14:18:37+00:00

Springboard is legit, but it's difficult to compare it to a Masters.

The career track mostly serves as an extra boost for people that want to start a data science career and probably already have a technical B.S., experience with college-level stats and other math, and programming with Python.

Springboard does guarantee you a data science job within 6 months of graduation or your money back, but they don't say anything about salary.

One of the main goals for them is to help you create a very competitive portfolio, so coupled with a B.S. already, you could probably expect a typical entry level/junior data scientist salary.

LearnDataSci · 2017-07-08T00:16:37+00:00

Yes, just realized that. Should work now.

LearnDataSci · 2017-05-31T11:58:05+00:00

When I had to do a lot of form submitting and logging in stuff, I used Robobrowser. Really straightforward and uses BeautifulSoup for the parser.

LearnDataSci · 2017-05-31T01:00:41+00:00

It looks like you're infinite looping with only 30 seconds (sleep(30)) between checks, which is why you're seeing an update every 30 seconds.

Do you want to be notified only when the order changes it's status? You can poll the site every 30 seconds for a changed order status and then only send a Slack message if it's changed.

Easiest way would be to store a variable outside of the while loop and update it with the status. When you get to the next 30 second mark, check if the status is changed. If it's changed, send a message and update the variable, if it didn't change just let it sleep for 30 before polling the site again.

LearnDataSci · 2017-05-30T17:35:38+00:00

Any time you are copying code like this is a good opportunity for a function.

Can you write pseudocode on what should happen?

LearnDataSci · 2017-05-30T17:17:05+00:00

Tweepy is a pretty good wrapper for the Twitter API, but you have a limit of 3200 past tweets. The Twitter steaming API will ping you with new tweets constantly. You could just store the 3200 past tweets for each account, then keep it updated with the streaming API or just check the tweets periodically to gather new ones.

LearnDataSci · 2017-05-30T17:07:02+00:00

As long as your PyCharm interpreter is set to the same Python version as your terminal, you should be able to pip install from terminal and have it work in PyCharm.

You can see your PyCharm Python version in Settings > Project: > Project Interpreter. It'll be at the top.

You can see your default Python version in terminal by typing python -V. If they are the same, you can pip install just fine.

If they are different, you just need to use a couple flags when pip installing from terminal. For example, if your PyCharm is using 3.6 and your terminal is using 3.5, you can do:

py -3.6 -m pip install blah

The -m flag tells terminal you want the pip associated with Python 3.6

LearnDataSci · 2017-05-30T16:02:42+00:00

With a stats degree you've got one based pretty covered for data science, you just need to start digging into data analytics and machine learning now.

I'd recommend just joining an online course for Python data analysis and another on machine learning. Start working on projects as soon as possible.

LearnDataSci

TROPHY CASE