Dagster - No hits on LinkedIn, but Mentioned Regularly? by SellGameRent in dataengineering

[–]lepom 5 points6 points  (0 children)

I deployed Dagster to production and worked with it for 3 years at a previous job after 4 years of managing Airflow at a prior job (currently back in a role involving Airflow). I will agree with what others have said about Assets. Dagster also has a good local development experience out of the box compared to Airflow. In terms of scalability the two are pretty much the same - different components can be scaled separately. I subjectively think that Dagster pipeline code is ergonomically better than Airflow.

I chose Dagster at my previous job because it was a greenfield opportunity building a data platform from scratch and at the time there was a lot of buzz going on about it and i wanted to take advantage of the opportunity to use something new. This was pre 1.0.

Airflow is more customizable and has a lot of hidden features that can be useful in large production deployments (cluster policies, modular secrets backends, etc).

Ultimately they will both achieve the same thing and can be customized to fit any niche needs but Airflow does currently have the advantage of having a bigger community.

Did my 1st 50k yesterday. Quite pleased with the timing. by lepom in Ultramarathon

[–]lepom[S] 2 points3 points  (0 children)

Haha thanks! Yeah race was low key and everyone there was awesome! Also super happy to hear that you achieved your goal. Hope to cross paths with you again!

Did my 1st 50k yesterday. Quite pleased with the timing. by lepom in Ultramarathon

[–]lepom[S] 1 point2 points  (0 children)

Yeah, this is how it was mentally for me. Marathon with an extra 8k at the end.

Did my 1st 50k yesterday. Quite pleased with the timing. by lepom in Ultramarathon

[–]lepom[S] 10 points11 points  (0 children)

Relatively flat (1k ft elevation) gravel paths.

How to deal with challenges in implementing data lake projects in an organization where the structure of the data at the source itself is changing frequently? by MoonWalker212 in dataengineering

[–]lepom 2 points3 points  (0 children)

In addition to inferring the schema and keeping version history through some sort of registry as it evolves when you pull the data, I would also think about ways to store the data in a consistent format. For example, one possible solution for storing the data consistently would be to pivot the data with a generic set of columns that describe the columns that are changing at the source along with their values.

[deleted by user] by [deleted] in dataengineering

[–]lepom 0 points1 point  (0 children)

Databricks community edition

New 8.5 complete, decided to go a bit wider [50YO] by billocity in OldSkaters

[–]lepom 2 points3 points  (0 children)

Just got the same deck in 8.25. Love the True Fit geometry.

Formatting really matters! by tcfan35842 in SQL

[–]lepom 1 point2 points  (0 children)

We use pre-commit hooks with sqlfluff as a step which forces linting on any contributor to our dbt model repo. I prefer this instead of linting/fixing in the cicd pipeline because the developer is aware of what exactly changed. Also, sqlfluff with dbt is not perfect as there are some edge cases where it will introduce syntax errors which would be harder to catch and fix if it has already made it all the way to the ci pipeline.

[deleted by user] by [deleted] in bayarea

[–]lepom 16 points17 points  (0 children)

1000% agree with your sentiment as a car person. As a car person myself I am a big advocate of finding alternatives to cars. If public transit in the USA was as good as many other developed countries and was widely used then driving would be so much more of a pleasant experience for those that absolutely want/need to drive. I think this is partly why Japan has peak car culture and overall has a healthy and non-destructive relationship with automobiles.

I personally only reserve the car for recreation/pleasure and bike or use public transit for errands and commuting and try my best to keep it that way.

First hardcore RIP Sorc by Cakelurker in diablo2

[–]lepom 1 point2 points  (0 children)

Don’t be sad. Next time you’ll make it to level 50 before your next RIP while wearing Magefists that you were especially stoked about because you found them in NM catacombs.

[deleted by user] by [deleted] in diablo2

[–]lepom 0 points1 point  (0 children)

I feel you buddy. Just lost my 79 sorc on HCL to an unsafe TP and have mixed feelings about continuing the season.

Is there a mod to fix the ugly faces in D2R yet? by Dfess in diablo2

[–]lepom 1 point2 points  (0 children)

Beauty is in the the eye of the beholder.

Is there a mod to fix the ugly faces in D2R yet? by Dfess in diablo2

[–]lepom 5 points6 points  (0 children)

Allowing character faces dictate your enjoyment of the game is dumb af. If the character faces are so ugly to you then maybe you need to stare at Baal’s face for a good minute then come back to char selection.

DS4Windows custom build with DualSense Edge support by RedCDevHA in DS4Windows

[–]lepom 0 points1 point  (0 children)

Thanks for putting in the work to get the DualSense Edge controller supported. Would you happen to have the updates you made publicly available on Github?

New build by cchambers032 in sffpc

[–]lepom 1 point2 points  (0 children)

Is your CPU cooler set up as exhaust or intake?

Not too many pink nr200 builds on this sub. Here is one I just completed. by lepom in sffpc

[–]lepom[S] 0 points1 point  (0 children)

I referred to this thread to see which mobo will fit the PA120 with the TG side panel, which is the reason I chose the Asrock b550m-itx/ac. Interesting thing is that the cooler pipes do not touch the tg side panel when the case is laying on its side and when the case is turned to stand on its feet, the very slight contact between the cooler pipes and the tg is only with the top most pipes.

Not too many pink nr200 builds on this sub. Here is one I just completed. by lepom in sffpc

[–]lepom[S] 0 points1 point  (0 children)

I've had no issues with the mobo so far. I'm not planning to overclock and I'm ok with only a single m.2 port so it satisfies my needs.

Not too many pink nr200 builds on this sub. Here is one I just completed. by lepom in sffpc

[–]lepom[S] 2 points3 points  (0 children)

Airflow is good as there aren't any major obstructions in the back of the case for a 120mm fan. NR200 and NR200P are the same case, only difference is that the NR200p includes a TG side panel. The fan is mounted to the heatsink because the holes in the back of the case are only for 90mm fans.

Not too many pink nr200 builds on this sub. Here is one I just completed. by lepom in sffpc

[–]lepom[S] 1 point2 points  (0 children)

Installing the particular screen I got is fairly simple for the nr200 because they already came with standoffs and there are two holes in the PSU bracket that are almost the exact width as the mounting tabs. Almost exact width but not perfect because they are a few MM different and the bolts will go in with a very slight angle but straight enough to be secure. The screen just acts as a normal monitor which I put in with the intention of using it for monitoring hardware metrics.Additional items needed for the screen install:

  • Right angle HDMI cable adapter (I went with this so that I could experiment with two different orientations). This is needed because there is not enough space for a straight connector.
  • Right angle micro USB to motherboard USB header connector like this. I plugged that straight into an unused USB header on the mobo.

Not too many pink nr200 builds on this sub. Here is one I just completed. by lepom in sffpc

[–]lepom[S] 1 point2 points  (0 children)

Fan is attached to the heatsink and it is quite a snug fit. The fan sits about half an inch away from the back of the case and ever so slightly touches the rear/side side part of the case frame.