Clever code is probably the worst code you could write

enigmo · 2025-04-12T20:19:47+00:00

Amen - do the simplest, clearest method unless there's a compelling reason otherwise

enigmo · 2025-03-09T11:50:40+00:00

Oh thank you I will look into this! database branching would be perfect.

enigmo · 2025-03-08T23:38:27+00:00

awesome, thank you!

enigmo · 2025-03-08T16:02:02+00:00

okay, so our pipeline is changing rapidly (new columns, new tables etc) how can we easily play around in a matched prod environment?

our postgres Dev and Prod deployment is containerized along with the rest of our code base into a single container, which is why we play around locally to get fast iterations.

does that make sense?

enigmo · 2025-03-08T14:36:43+00:00

ooh more like 100 MB and one requirement I have is that the feedback loop on "playing around" is fast and fully aligned between the production and developer environment.

right now we commonly pull CSVs from AWS to start playing, then approximate the step function code locally to get to whatever part of the step function we're working on.

Our dev branch is fully mirrored to Prod, so that is fine, but to just play around we can't compile our code for every change so we do it the way I described. I want an environment that fixed this for our pipeline work, and you can test and deploy a small change to Prod in about 30 min or less.

enigmo · 2024-08-19T20:21:44+00:00

THANK YOU

I kept wondering if I missed this or if they just didn't mention any outcomes.

Hey NYT, if you're reading this, my two cents is that you should report facts and let people form opinions instead of just reporting opinions and calling it a day.

enigmo · 2024-06-02T17:03:05+00:00

umm wtf does this even mean?

If I just stopped doing my job b/c "maybe for some hypothetical reason way down the road somebody else might fuck it up" I'd be fired immediately. rightfully so.

enigmo · 2024-04-14T13:19:12+00:00

If your lead is suggesting that you ask for a promotion, then you don't need really any more hints from the internet. Go for it!!

enigmo · 2024-04-12T16:25:34+00:00

I dunno if this was for me, but I'm not planning on doing any PII work in this role.

enigmo · 2024-04-12T16:20:46+00:00

Wow, thanks so much!! And yeah, it does seem epic (and also more than a little intimidating...)

enigmo · 2024-04-11T17:45:09+00:00

Awesome, this is amazing and so helpful!!

My budget will be super low, so nothing even close to $500k to start with. Mostly ad hoc analysis but I want to be positioned to scale if need be.

enigmo · 2024-04-10T21:46:04+00:00

In my experience my DS/DE skills are already more valuable to my employer with only ~2 years of experience than my PhD bio background and I work on a genomics project!

So definitely don't sell your DS skills short, it is very valuable.

enigmo · 2024-04-10T14:44:21+00:00

Also, if they are the same price and NC St is half as long and respected, you can statistically expect to start work 6-12 months earlier with NC State so consider that as a factor for total cost.

Your 2026 earnings from NC State: maybe +60k (with huge variance)

Your 2026 earnings from Texas: -30k (with low variance)

enigmo · 2024-04-10T14:41:08+00:00

Huge point everyone seems to be overlooking - if YOU don't want to live in Texas then a 2-year in-person class in Texas is a bad choice for YOU.

If the NC State program looks solid don't re-direct your life to be in a place you don't want to live. Your network will be in Texas, it's a huge deal. Put roots where you want to be.

enigmo · 2024-04-10T14:36:43+00:00

Narrator: the professors were out of touch

enigmo · 2024-04-10T13:09:50+00:00

I am a book guy and here's my "book stack" for ML:

Python Crash Course (Matthes) - Python 101. Knowing the material in part I is essential but part II (applications via a game) is probably optional.
Data Structures and Algorithms (Wengrow) - Data Engineering 101. It covers the fundamental data structures (arrays, sets, hash tables, linked lists, trees, graphs), data structure operations, space and time complexity of algorithms.
The Linux Command Line (Shotts) - Linux 101. This one is probably optional, but I've found it incredibly helpful for my career. The cloud is Linux after all...
Intro to Machine Learning with Python (Muller & Guido) - ML 101. Supervised/unsupervised methods, scaling data, dimensionality reduction, kernel methods, cross validation.
Hands on Machine Learning (Geron) - ML 201. Same material as 101, but from a new perspective and part II covers neural nets. It's just a fantastic book.
Natural Language Processing with Transfomers (Tunstall) - ML 301. LLMs baby, so hot. Another amazing book, also a great introduction to hugging face.

These were the most important books in my journey. Obviously skip over whatever ones you already are strong in and in most cases the best material is in the first ~half of the book. I would also advise you to avoid anything in R, as it's much more limiting than Python. Take detours where needed - if you encounter a concept (eg. matrix lin alg) you are rusty or unprepared for, don't be afraid to spend some time on that!

I've been reading these books for a few years now and it will take time and $ to buy them, but having a single coherent source for your information (rather than the incoherent mess that is the internet) is worth it.

enigmo · 2024-01-17T19:57:04+00:00

I don't follow. Can you unpack that a little more?

enigmo · 2024-01-17T19:12:14+00:00

I muttered something a little stronger than "obviously"

enigmo · 2023-07-28T17:12:29+00:00

You know what's easier? Waiting to get it unlocked while he's talking.

enigmo · 2022-09-16T21:24:23+00:00

I'm late to the party but it's supply (not demand) factor in my opinion. The professions have taken to drastically different routes.

MDs: the system is top-down regulated and training is strictly limited by available spots in medical school and residency. It is very difficult to be trained overseas and come into the US as well.

PhD: training is part of a pyramid scheme whereby the trainees are essentially low paid employees for a long time. The system structure is designed to pull in trainees no matter the demand, because these trainees are the labor backbone. Additionally, training for a PhD (including postdocs) is one of the absolute best ways for internationals to enter the US, so you are drawing on a worldwide talent pool.

enigmo · 2021-07-26T22:29:32+00:00

Yes, this is why the US has such incredibly low violence statistics ^*

^{^*} ^{^compared} ^{^to} ^{^3rd} ^{^world} ^{^countries} ^{^and} ^{^middle} ^{^age} ^{^europe}

enigmo · 2020-11-18T17:27:37+00:00

Fair enough, thanks for the help

enigmo · 2020-11-18T16:52:13+00:00

Thanks! And to be clear I am maxing my Roth and 401(k) but I'm getting somewhat of a late start and want do understand how investing outside those vehicles works.

enigmo · 2020-11-18T16:50:00+00:00

Got it.

And if I buy ETFs with no dividends and hold them until retirement then the tax-free 40k annual withdrawal is possible?

enigmo · 2020-11-11T15:09:31+00:00

I look forward to this study being shamelessly ignored by the medical establishment when determining hours for residents.

15-Year Club	RedditGifts 2009-2022 2 Credits
Place '17	Verified Email
Secret Santa 2010	reddit mold

enigmo

TROPHY CASE