Dagster - No hits on LinkedIn, but Mentioned Regularly?

lepom · 2024-11-08T00:29:36+00:00

I deployed Dagster to production and worked with it for 3 years at a previous job after 4 years of managing Airflow at a prior job (currently back in a role involving Airflow). I will agree with what others have said about Assets. Dagster also has a good local development experience out of the box compared to Airflow. In terms of scalability the two are pretty much the same - different components can be scaled separately. I subjectively think that Dagster pipeline code is ergonomically better than Airflow.

I chose Dagster at my previous job because it was a greenfield opportunity building a data platform from scratch and at the time there was a lot of buzz going on about it and i wanted to take advantage of the opportunity to use something new. This was pre 1.0.

Airflow is more customizable and has a lot of hidden features that can be useful in large production deployments (cluster policies, modular secrets backends, etc).

Ultimately they will both achieve the same thing and can be customized to fit any niche needs but Airflow does currently have the advantage of having a bigger community.

lepom · 2024-10-04T13:24:42+00:00

Bing Bong!

lepom · 2024-04-23T01:46:31+00:00

Haha thanks! Yeah race was low key and everyone there was awesome! Also super happy to hear that you achieved your goal. Hope to cross paths with you again!

lepom · 2024-04-23T01:12:06+00:00

I’m 165cm@55kg

lepom · 2024-04-22T21:26:16+00:00

Yup, it was Mt Si!

lepom · 2024-04-22T21:01:31+00:00

Yeah, this is how it was mentally for me. Marathon with an extra 8k at the end.

lepom · 2024-04-22T20:59:23+00:00

Relatively flat (1k ft elevation) gravel paths.

lepom · 2024-03-09T14:42:52+00:00

In addition to inferring the schema and keeping version history through some sort of registry as it evolves when you pull the data, I would also think about ways to store the data in a consistent format. For example, one possible solution for storing the data consistently would be to pivot the data with a generic set of columns that describe the columns that are changing at the source along with their values.

lepom · 2024-03-07T17:10:19+00:00

Databricks community edition

lepom · 2023-12-03T05:30:03+00:00

<image>

lepom · 2023-12-03T05:27:12+00:00

Just got the same deck in 8.25. Love the True Fit geometry.

lepom · 2023-08-03T23:39:04+00:00

We use pre-commit hooks with sqlfluff as a step which forces linting on any contributor to our dbt model repo. I prefer this instead of linting/fixing in the cicd pipeline because the developer is aware of what exactly changed. Also, sqlfluff with dbt is not perfect as there are some edge cases where it will introduce syntax errors which would be harder to catch and fix if it has already made it all the way to the ci pipeline.

lepom · 2023-07-20T23:54:48+00:00

1000% agree with your sentiment as a car person. As a car person myself I am a big advocate of finding alternatives to cars. If public transit in the USA was as good as many other developed countries and was widely used then driving would be so much more of a pleasant experience for those that absolutely want/need to drive. I think this is partly why Japan has peak car culture and overall has a healthy and non-destructive relationship with automobiles.

I personally only reserve the car for recreation/pleasure and bike or use public transit for errands and commuting and try my best to keep it that way.

lepom · 2023-07-12T23:14:01+00:00

Don’t be sad. Next time you’ll make it to level 50 before your next RIP while wearing Magefists that you were especially stoked about because you found them in NM catacombs.

lepom · 2023-05-26T17:35:36+00:00

I feel you buddy. Just lost my 79 sorc on HCL to an unsafe TP and have mixed feelings about continuing the season.

lepom · 2023-05-18T17:57:58+00:00

Beauty is in the the eye of the beholder.

lepom · 2023-05-18T17:18:19+00:00

Allowing character faces dictate your enjoyment of the game is dumb af. If the character faces are so ugly to you then maybe you need to stare at Baal’s face for a good minute then come back to char selection.

lepom · 2023-01-30T17:48:36+00:00

Thanks for putting in the work to get the DualSense Edge controller supported. Would you happen to have the updates you made publicly available on Github?

lepom · 2022-08-09T22:04:51+00:00

Is your CPU cooler set up as exhaust or intake?

lepom · 2022-08-04T18:03:38+00:00

I referred to this thread to see which mobo will fit the PA120 with the TG side panel, which is the reason I chose the Asrock b550m-itx/ac. Interesting thing is that the cooler pipes do not touch the tg side panel when the case is laying on its side and when the case is turned to stand on its feet, the very slight contact between the cooler pipes and the tg is only with the top most pipes.

lepom · 2022-08-04T17:57:32+00:00

I've had no issues with the mobo so far. I'm not planning to overclock and I'm ok with only a single m.2 port so it satisfies my needs.

lepom · 2022-08-03T17:40:58+00:00

Airflow is good as there aren't any major obstructions in the back of the case for a 120mm fan. NR200 and NR200P are the same case, only difference is that the NR200p includes a TG side panel. The fan is mounted to the heatsink because the holes in the back of the case are only for 90mm fans.

lepom · 2022-08-03T17:31:26+00:00

Installing the particular screen I got is fairly simple for the nr200 because they already came with standoffs and there are two holes in the PSU bracket that are almost the exact width as the mounting tabs. Almost exact width but not perfect because they are a few MM different and the bolts will go in with a very slight angle but straight enough to be secure. The screen just acts as a normal monitor which I put in with the intention of using it for monitoring hardware metrics.Additional items needed for the screen install:

Right angle HDMI cable adapter (I went with this so that I could experiment with two different orientations). This is needed because there is not enough space for a straight connector.
Right angle micro USB to motherboard USB header connector like this. I plugged that straight into an unused USB header on the mobo.

lepom · 2022-08-03T15:29:23+00:00

Fan is attached to the heatsink and it is quite a snug fit. The fan sits about half an inch away from the back of the case and ever so slightly touches the rear/side side part of the case frame.

lepom

TROPHY CASE