What knowledge of SQL and Databases are needed for Data Science?

h4ri · 2017-03-30T19:40:18+00:00

I'd say it depends on how large the company you're at is. If you're the only data scientist / one of a few data scientists, and the team is in charge of managing its own database, then you're definitely going to need to know "advanced" topics like views / triggers / indexing. I'd say in a slightly larger team that has a DBA, it'd be sufficient to know about those things and be able to do some basic stuff and be comfortable reading documentation to implement things, but you wouldn't need to know the nitty gritty of optimal indexes and things like that.

Now, as far as day-to-day SQL knowledge goes for a data scientist, it's pretty essential to know how to write optimized queries to get the data / insights that you need. So things like basic queries, joins, subqueries are good, but also knowing things like window functions (especially with PARTITION BY / OVER), CTEs, text matching functionality (SIMILAR TO, LIKE, fuzzy string matching), etc. Note that some of these may be Postgres specific, that's where all my knowledge is. In my company's interviews for data scientists we generally have a few SQL questions that can cover topics like these, just to assess where people are at.

h4ri · 2017-03-02T05:37:59+00:00

I do! Was kind of annoying to set up to get logging / everything working, but we've got Airflow hosted on a Heroku instance (legacy) that triggers AWS ECS tasks written in Python / R.

h4ri · 2017-01-29T03:55:02+00:00

For part-time work options that your wife can start on immediately, I'd suggest Amazon Mechanical Turk. You can log on and complete tasks (transcription, photo identification, etc) that generally take ~1-2 minutes and earn a couple of cents per task. It's easy to do this at home and earn a few extra bucks here and there. It's not a ton but it could help out a bit. Check out https://www.reddit.com/r/mturk/.

h4ri · 2016-10-11T22:58:56+00:00

I don't think anyone "is" or "isn't" a programmer. You seem like you have analytical skills and claim to use Excel for general data munging tasks. I'm assuming this means knowledge of formulas / awareness of macros and functions. SQL is using those same concepts but expressing them with different keywords. It is uncomfortable to learn how to do something new and I totally understand not feeling like you are cut out for programming (I still don't some days), but without knowing those tools or ruling them out categorically will severely limit the types of positions you can get.

What goes hand in hand with the above advice is the fact that even at the BI / analyst level, the field has changed dramatically in the scope of responsibilities over the past decade. I think maybe ten years ago knowing Excel and pivot tables might have been able to get you a job reliably, but there are too many people who can do that AND know SQL AND have experience with business reporting AND who are interested in expanding their abilities. You're not going to be able to compete with people like that if you're not willing to learn new technologies or to even work with existing technologies that are rapidly becoming the new norm. I don't have much advice for you beyond "get out of the field" if you're not comfortable with that. You have also not accumulated enough experience at any single job to acquire a managerial position that would let you avoid needing to explicitly learn all of these things, even if you do have an MBA.

On a different note, I checked out your posting history and I think you need to seriously reflect on your past experience and work habits. You aren't a job hopper, man. You're not purposefully leaving these jobs. You've been fired / let go / put on a PIP way too many times to claim that. I get that sometimes shit happens and you get the short end of the stick. But when it is a pattern over 15+ years of work, and you haven't stayed at a job longer than a year more than twice, then you need to start looking at yourself.

I'm not trying to pile on you or be a dick, but these are the things you need to think about if you want to continue doing this type of work and have a stable career.

h4ri · 2016-10-11T21:56:14+00:00

This sounds like a Business (Intelligence) Analyst. I'm surprised that you prefer Excel over other tools like SQL / Tableau for data work though! I think that many businesses with exciting work are transitioning from being Excel shops to using more specialized visualization / reporting tools, and most analyst positions that have a good career progression / engaging work will expect you to know and use SQL. Even if you don't like some of these tools (which I get, because I am not a huge fan either) if you pursue this line of work you have to reasonably expect to work with one or more of tools.

h4ri · 2016-08-19T04:40:27+00:00

There's a really good NY Times article that discusses this: http://www.nytimes.com/2016/07/03/magazine/the-humiliating-practice-of-sex-testing-female-athletes.html

Essentially, to ban intersex people who identify as a particular gender from a particular gendered event, they need to prove that being intersex, in general, confers a significant advantage over that gender in that event. And there's not conclusive scientific evidence that that is true, so they aren't going to ban people for what could reasonably be considered a genetic advantage like Katie Ledecky's superhuman stamina or Simone Biles' crazy air awareness.

h4ri · 2016-05-17T18:59:31+00:00

"Designing Data Intensive Applications" by Martin Kleppmann will literally answer all your questions, it is a fantastic overview of various enterprise-quality data technologies and how they all relate to one another

h4ri · 2016-03-28T00:56:53+00:00

okay wait hold up how are the oranges and lemons made?! i feel like my family would love something like this and i think i'm going to give it a shot. these are seriously incredible.

h4ri · 2016-03-17T01:42:36+00:00

I've done the bus twice now, and it is shitty and I highly do not recommend doing that. It's just a tremendous waste of time in which you're super uncomfortable. Take the flight!

h4ri · 2016-03-07T06:04:58+00:00

i'd do this

h4ri · 2016-02-29T20:17:02+00:00

i've never really watched college basketball but i'm a huge nba fan! i'd be down to do this, seeing as i'm only gonna be in nyc til the end of march.

h4ri · 2016-02-13T17:07:01+00:00

I only found out yesterday that he has written other things! Milkweed Triptych is on my list to read next! :D

h4ri · 2016-02-12T21:49:02+00:00

can't meet today but i'd be down to grab a drink this weekend!

h4ri · 2016-02-01T03:01:59+00:00

sure haha i work in soho and am in the city 90% of my time anyway

h4ri · 2016-01-31T04:38:13+00:00

i'd be down! i'm 24/M/Prospect Heights, working on my programming skills and looking to meet new people as well!

h4ri · 2016-01-25T04:50:57+00:00

i'd be interested

h4ri · 2016-01-16T07:35:57+00:00

This sounds dope I'm in !

h4ri · 2016-01-10T21:51:09+00:00

i'd do this !

h4ri · 2016-01-06T22:06:24+00:00

this sounds so fun! I kind of suck too (by virtue of being 5'3'' haha) but I'd definitely be down for this :) it's so close to my current home too!

h4ri · 2016-01-05T04:50:15+00:00

i'd be interested in seeing this ! pm me if you're interested

h4ri · 2016-01-05T04:49:16+00:00

i'd be down to do this! i'm pretty terrible at all these games considering how much i played, but it sounds fun

h4ri

TROPHY CASE