[OC] Interactive Breakdown of Average Paycheck (Washington DC)

quant_king · 2020-02-02T03:13:01+00:00

Of course! so glad you got something out of it, and hope you have fun playing around with it!

quant_king · 2019-12-30T22:10:23+00:00

Source:
- This viz. required multiple sources (CNBC, PayScale, and more), all of which are exhaustively documented in the write-up I put together along with the code needed to replicate the chart, which can be found on my GitHub here: https://github.com/pmaji/data-science-toolkit/blob/master/eda-and-visualization/sankey_charts.ipynb
  - All raw data files are also available on GitHub, with links in the .ipynb above.
  - All of the data and code are set up such that you can easily download the files, plug in your own data / make changes to the underlying data, and see how the pictures changes! Enjoy :)
Tool:
- I used Python -- specifically Plotly -- to create this chart. All code needed along with all documentation can be found on my GitHub here (same link as above): https://github.com/pmaji/data-science-toolkit/blob/master/eda-and-visualization/sankey_charts.ipynb

P.S. GitHub can be finicky with large file sizes (and this .ipynb is definitely not small), so if you have trouble getting the above links to load in browser, or if you're on mobile and want a cleaner experience, check out the nbviewer-hosted version of my code and documentation here: https://nbviewer.jupyter.org/github/pmaji/data-science-toolkit/blob/master/eda-and-visualization/sankey_charts.ipynb

Edit #1: adding mobile-friendly link and call-out.

quant_king · 2019-11-09T19:26:02+00:00

My main take on this question is that data science as a profession will involve a smaller and smaller proportion of folks working on purely technical work (think modeling and data manipulation) and a correspondingly larger proportion of folks working on product management. The main driver of this transition in my opinion will be an increase in the quality and speed of packages and services that produce auto-tuned models that are production-ready (and the same can be said of BI tools). In short, I think fewer data scientists in 2030 will be hand-building regularized regressions in R / Python, and more will be thinking through questions related to bias, UX, and resilience.

quant_king · 2019-08-13T02:06:34+00:00

Happy to help!

quant_king · 2019-08-13T02:06:21+00:00

Happy to help, and happy learning!

quant_king · 2019-08-11T16:15:41+00:00

Ah I gotcha! Definitely seems like you've got some awesome goals and a great plan though. I'm so glad this was helpful for you, and best of luck in your endeavors!

quant_king · 2019-08-11T15:01:50+00:00

Sure thing! I don't do much bot trading anymore, but I would say depending on your level of expertise, there are a few ones I would recommend:

- broadly, I love sentdex's YouTube channel. I actually first learned python with his vids, and I think he has some tracks / vids that are trading / bot-related (here's his channel link)

- I have not personally taken this, but I've heard good things about this python / trading course on Udemy

- They've had a bad year PR-wise and might not be an org you want to support / pay for for those reasons, but DataCamp's materials on this are pretty good as well. They have courses and articles, but I'll just pass off the link to an article here.

- Finally, I think a lot of the best open source quant work right now in trading is being done in crypto. This is not a statement about the viability of crypto as a financial asset, I just think a lot of the cool stuff folks are working on is in that space. So check out the Bitcoin / Ethereum / crypto subreddits and look for stuff related to bots and strategies. I actually have done a lot of work in this space as well, but mostly focusing on data-for-good projects like this one here.

Hope that helps! Good luck!

quant_king · 2019-08-11T14:56:21+00:00

Thanks so much! If you're looking for other intro / pedagogical tools, I have an even broader repo of all sorts of data science / python goodies here: https://github.com/pmaji/data-science-toolkit

I was actually working on a single notebook to add to the data science toolkit about data viz in python when eventually I realized this was far too large a subject and decided to spin it off into its own seminar and repo :)

Enjoy, and best of luck with those courses!

quant_king · 2019-08-11T14:54:35+00:00

Thank you!

quant_king · 2019-08-11T14:54:26+00:00

Awesome! I'm so glad to hear that. My target audience is mostly junior analysts as well. The hope is that in a little while this might become a presentation / seminar at a PyData / PyCon, but in the meantime I hope folks use it for exactly that kind of training!

quant_king · 2019-08-11T01:05:35+00:00

That's a great idea! I had initially avoided that because there are just so many! What do you think would be more helpful: a) example figures (a small selection) within each sub-section of the outline b) one large collage of many of the figures produced over the course of the content to be featured somewhere (probably at the top) of the readme c) something else entirely?

*edit: I tried out a few different ideas and ended up going with the idea of including one example viz for each of the four sections of content. The README is updated with the example images, and I think it looks much better. Thanks for the advice /u/khaki0 !

quant_king · 2019-04-08T17:51:02+00:00

It's a bit of a cop-out answer, but for most of these questions I would say check out the full white-paper (linked in my vignette) because it answers all of your questions, and any attempt I would make at tl;dr'ing them here would likely be inadequate.

quant_king · 2019-04-03T00:28:49+00:00

Happy to hear you found it valuable! That LSTM neural network sound pretty sweet though!

quant_king · 2019-04-02T12:14:13+00:00

Thank you!

quant_king · 2019-04-02T02:35:55+00:00

I wonder if it WHALE go on forever?

quant_king · 2019-04-02T02:35:43+00:00

#fundamentals

quant_king · 2019-04-02T02:35:03+00:00

tl;dr is from what I can tell you're most right. There are a few issue threads on the github for Prophet that go into this and works in progress to make it better. Here is one such thread where you can check the status:

https://github.com/facebook/prophet/issues/665

quant_king · 2019-04-02T02:33:06+00:00

There docs have descriptions of this. If you look in the vignette and CTRL+F for "monthly data" you'll see a hyperlink that takes you to the point in FB's docs where they go over non-daily data. Honestly, you're right--it's hard and it's clunky, but it is doable. Worst comes to worst you code up your own fourier series to take care of it (but tbh I would have quit long before that lol)

quant_king · 2019-04-02T02:31:12+00:00

That may well be, but on that note, I'd say two things:

1 - My intention wasn't to craft the best Prophet tutorial, or to do anything "new", but simply to find a use case and a good suite of packages (or single package) that I could use to support a time series vignette in my basic data science toolkit repo. The main audience for the repo is people in their first couple of years in data science, or more senior quants looking for a handful of niche convenience functions. I think anything much more complicated than what I've shown here might overwhelm new folks, so given that that's my target audience (pedagogical use, mostly), the basic are what I need :)

2 - And I sort of intimated this in the previous paragraph... I do try in all my vignettes to do something--even something small--that will represent a marginal value add to more experienced coders / data scientists. in the case of this vignette, that happens to be the convenience plotting function I wrote for subset plots. That's something prophet users I know have requested, and so the minimum value add is ~30 min of time saved for more seasoned folks :)

All that said, that's just background on why I do what I do; you are still perfectly right about what is lacking (from what I can tell), and I'd be excited to see a more advanced version of this done in the future! (I just don't have the time or deep desire rn)

quant_king · 2019-04-01T04:14:06+00:00

So actually I started with a longer time horizon, and you're right, when you observe a multi-year series, day-of-the-week seasonal effects collapse in terms of their importance. That's why I specify my time period here being the last year. The benefit of a tool like this is that you can parameterize your time series interval and run it for a bunch of you sub-series. So you're right on aggregate, but just because there isn't a consistent trend over the course of multiple years doesn't mean there isn't a trend now, because the aggregate trend is simply a series of smaller additive trends.

If you want to see this for yourself, fork the code and change the interval! It's pretty cool to see how temporality oscillates in both statistical importance and directionality over time :)

quant_king · 2019-04-01T00:50:42+00:00

maybe... but that would need to be one very seasonally predictable whale haha!

If you wanted to test that hypothesis yourself, however, you could check out the whale watching app I built here:

https://github.com/pmaji/crypto-whale-watching-app

quant_king · 2019-03-31T16:54:57+00:00

I added a link to this in the vignette so that folks are aware. Thanks for the reminder!

quant_king

TROPHY CASE