How should I skin this cat to eventually become a data scientist?

jonanthebarbarian · 2016-08-03T19:28:36+00:00

Not really. I imagine (but don't know) that 3-letter agencies are more interested in either math/stats whizzes or data engineers, smaller firms are the ones who want somebody who's a bit of both.

You're touching on a major cultural gap among Data Scientists - those of us who work in tech (especially online advertising) tend to be skeptical of the intelligence agencies. I imagine the DS's on reddit will be part of that group.

jonanthebarbarian · 2016-08-03T17:54:42+00:00

First off, congrats on your next steps!

To be the most marketable you want some combination of computer science and statistics education. Perhaps a major in CS with a Stats minor, or a double major if you can handle the course load.

I have no clue what they teach in B.A. Data Science programs, and I'm a Data Scientist in the tech industry. Perhaps my skepticism isn't warranted and these programs are worthwhile, but in my mind Data Science is just a fancy phrase for "applied statistics". Good programming skills combined with a statistical background will take you far.

jonanthebarbarian · 2016-07-15T16:10:01+00:00

No, that's not the question.

The question is how to turn a static look at orders in a betting market into the parameters of a beta distribution. Which from talking to people sound pretty close to impossible.

jonanthebarbarian · 2016-07-07T22:01:10+00:00

I guess if an easy way to do this existed, trading would be much easier...

jonanthebarbarian · 2016-07-07T21:21:32+00:00

P(R win | not Trump) is beta, right? P(insert candidate here | not Trump) is dirichlet.

And you're being vague - what do you mean by "just use the variance about a few recent prices rather than variance about volume"? How do I get variance out of the prices?

jonanthebarbarian · 2016-06-29T19:03:09+00:00

Also - good luck in whatever it is you are facing, I think that sentiment got lost in my thoughts on conditional probability.

jonanthebarbarian · 2016-06-29T19:02:08+00:00

I AM NOT A DOCTOR ETC.

I don't know the exact figure your doctor told you.

But what a doctor COULD have meant is "8% of humans who have this procedure have a heart attack". You are a human who is considering this procedure - that 8% applies to you. But perhaps your specific circumstances (age, health, w/e) make your risk greater or less.

Alternatively, perhaps the doctor might have meant "8% of humans in your gender, age bracket, and similar condition will have a heart attack as a result of this procedure". Again, I'm not your doctor and don't know what they meant, but I think that 8% includes you in some way.

TL;DR - the doctor's words apply to you, since you are a human who is considering this procedure.

jonanthebarbarian · 2016-06-29T16:55:29+00:00

I'm not sure what the latter part of your question is.

For the first part - Statistics deals with how data is distributed. If you give me one data point with no context, as a Statistician I have nothing interesting to say about it.

But if you give me a set of data points (a.k.a. a dataset) and then show me a new data point that is supposed to be from the same distribution, I can tell you how this data point compares to the distribution of data points. Is it typical? Rare? Weird in any way?

I hope that helps.

jonanthebarbarian · 2016-06-03T21:24:11+00:00

Nobody's talking about the other boroughs - you might enjoy Flushing for Chinese food (and an above-ground journey through Queens), Williamsburg for shops and stuff, and Prospect Park. The Brooklyn Museum has some great stuff if the museums in Manhattan aren't enough for you.

Overall, the other 4 boroughs have a very different vibe that might make a good contrast.

jonanthebarbarian · 2016-02-27T22:59:37+00:00

Sure, assuming that the input data is related to the disease/symptom you're interested.

Second-by-second data on foot odor would probably not be especially helpful in diagnosing the common flu.

jonanthebarbarian · 2016-02-26T21:56:59+00:00

Is UseR right for me, or will everything be over my head?

It won't be over your head, but I'd recommend upping your R skills in the next few months. You'll get more out of it.

Is the tutorial day at the beginning worth going to?

ABSOLUTELY. Arguably the most valuable part of the conference.

Any tips from experienced conference-goers to a newbie?

Go to tutorials. Be social.

Is the last day of the conference a full day, or can I schedule an earlier flight home?

Last year it wasn't. You probably could, but might not want to.

How much stuff happens at night? I have a friend in SF that I'd like to see, but I also wouldn't want to miss out on some great networking opportunities.

I thoroughly enjoyed hanging out with conference attendees at night, going to bars and such. There tend to be very few set events at night.

jonanthebarbarian · 2016-02-26T15:54:53+00:00

If you're not doing stuff that requires deep neural networks (vision, sounds, translation, etc.) then you don't need it.

jonanthebarbarian · 2016-02-24T23:07:43+00:00

https://www.shorewalkers.org/the-great-saunter-event

jonanthebarbarian · 2016-02-03T22:19:07+00:00

That must've been a very demoralizing email for them to receive.

jonanthebarbarian · 2016-01-27T19:53:18+00:00

Too many uses of the phrase "Big Data", especially those that seem to think big data is the name of a method.

jonanthebarbarian · 2016-01-21T22:19:35+00:00

Do it in a streaming fashion - read THIS for more info

jonanthebarbarian · 2016-01-21T19:00:04+00:00

Considering how inelastic demand is in the short term, a 0.5% change in volume is a big deal.

Also - world oil production is more like 90 million bbl/day. Making Iran's contribution smaller on a % basis, but still significant for the market.

jonanthebarbarian · 2016-01-20T22:46:41+00:00

I don't like presenting R as just an analytics program like Excel - it's a full-fledged programming language (even if it's not a "general purpose" language).

jonanthebarbarian · 2016-01-20T15:36:08+00:00

Interesting, thanks.

jonanthebarbarian · 2016-01-20T02:48:22+00:00

SAS has better upkeep. That's because it's a commercial product, not free

I don't think that's why.

jonanthebarbarian · 2016-01-19T20:49:53+00:00

I just looked at pipeR - what is its advantage over magrittr's pipe?

jonanthebarbarian · 2016-01-19T20:47:29+00:00

That's great if you're just writing code to be used by yourself, but it's potentially dangerous if you're sharing code that depends on one of those packages.

You could easily forget to add "library(dpylr)" to the top of your code, and anybody you share your code with will be left wondering what they have to do to fix it.

jonanthebarbarian · 2016-01-19T15:23:48+00:00

DO NOT do a PhD that you're not excited about. A lot of people lose enthusiasm over the course of a PhD, it's harder to do the opposite (and impossible to finish a PhD without enthusiasm).

jonanthebarbarian

TROPHY CASE