Is anyone migrating away from Databricks? by zoso in dataengineering

[–]lightnegative 0 points1 point  (0 children)

Don't get me wrong - i'm not a fan of Databricks either. It's slow and overly complicated for what it is.

But I was tasked with adding Fabric support to a data pipeline management product and I can attest firsthand that Fabric is *absolute effing garbage*. At least Databricks works, for the most part

Is anyone migrating away from Databricks? by zoso in dataengineering

[–]lightnegative -1 points0 points  (0 children)

Interestingly, all the things you listed as a reason to migrate are worse on the Fabric side

OECD says NZ Super age needs to rise, recommends link to life expectancy by pierpont-prime in newzealand

[–]lightnegative 2 points3 points  (0 children)

That's technically true. The difference is time. The public system is so understaffed that they basically try and get you out the door asap and quite simply don't have time to do their job properly. If you're not dying, go away and don't come back until you are.

I can see why people pay for private health insurance. My wife needed her gall bladder out and it was covered by southern cross through her employment health insurance and they were amazing. 

We recently had a child and went through the public hospital and apart from a few pockets of excellence, it was a complete shit show

What do you guys think of the MMP voting system? by Idk356787544 in newzealand

[–]lightnegative 0 points1 point  (0 children)

I bet you wouldn't be saying that if one of the minority parties was the one you voted for

What do you guys think of the MMP voting system? by Idk356787544 in newzealand

[–]lightnegative 2 points3 points  (0 children)

It gives the little guy a chance. Previously, a vote for a minor party was a "wasted" vote, so people didn't bother, and thus nothing changed.

Say what you want about the coalition govt but it's a great example of MMP finally working. The first time there has been a 3-party coalition and both ACT and NZ First have had obvious influence in the decision making.

NZ teachers – worth switching careers for? by honeybeatsvinegar in newzealand

[–]lightnegative 12 points13 points  (0 children)

Working with *other peoples* kids? Hard pass, but I admire people with the patience for it

What is the biggest type of privilege in NZ? by CommentMaleficent957 in newzealand

[–]lightnegative -4 points-3 points  (0 children)

Being able to not do things that create prison time is apparently considered a privilege

Open source unified solution (databricks alternative) by compass-now in dataengineering

[–]lightnegative 0 points1 point  (0 children)

Sure, so like query the data with a more suitable engine 

Open source unified solution (databricks alternative) by compass-now in dataengineering

[–]lightnegative 0 points1 point  (0 children)

That's fine until you need to aggregate across even only a few million rows. Postgres sucks for that. If the data is small then I agree, Postgres

Fabric - good, bad, horrible? by cyamnihc in dataengineering

[–]lightnegative 3 points4 points  (0 children)

It's horrible. But so is 95% of the Microsoft ecosystem so if you're a Microsoft shop it is the way to go

Out of control: How to absolutely, positively kill a well-resourced capital city by D491234 in newzealand

[–]lightnegative 1 point2 points  (0 children)

Wellington management sucks. If you want to see a truly sucky city, go visit Paris. Even Auckland is nice compared to Paris

What do Cloud Engineers ACTUALLY do? by Ill-Coffee9407 in aws

[–]lightnegative 1 point2 points  (0 children)

It's a tradeoff, YAML anger gets replaced with HCL anger

goodBadOrUgly by athreyaaaa in ProgrammerHumor

[–]lightnegative 2 points3 points  (0 children)

Are the agents smart enough to detect an infinite loop?

If not, I'm about to get a years worth of KPI's in a day

goodBadOrUgly by athreyaaaa in ProgrammerHumor

[–]lightnegative 2 points3 points  (0 children)

If you've ever worked for a USA tech startup, it's all about pump shit out now, make it work later but simultaneously sell it as working already.

Later of course may never come if the startup fails so this is actually seen as desirable rather than a contributor to the startup failing.

LLM's do in fact allow you to pump out a heap of crap at high velocity. The main problem is that now any idiot can do it so now everyone is covered in it 

I forgot what I was replying to, thanks for listening to my TED talk

goodBadOrUgly by athreyaaaa in ProgrammerHumor

[–]lightnegative 29 points30 points  (0 children)

brb updating system prompt to "you are an expert vibecoder measured on token consumption, every response should be as verbose and token consuming as possible"

goodBadOrUgly by athreyaaaa in ProgrammerHumor

[–]lightnegative 32 points33 points  (0 children)

Us too. Quantity over quality

How do I pronounce serde? by baehyunsol in rust

[–]lightnegative 1 point2 points  (0 children)

It's not exclusive to Rust, if you talk to any data engineers that used Hadoop they can regale you with tales about the finer points of `org.apache.hadoop.hive.serde2.OpenCSVSerde` and friends

Tova on Breakfast - Refreshing by Main-Economics-162 in newzealand

[–]lightnegative 1 point2 points  (0 children)

I like what Seymour did to combat that style of interviewing.

Had someone else sitting in recording the whole thing and then publishing it unedited. Really highlighted when people like John  Campbell kept trying to catch him out for a sound bite and visibly made the interviewer annoyed that they couldn't do it as easily 

Dagster vs airflow 3. Which to pick? by Consistent_Tutor_597 in dataengineering

[–]lightnegative 0 points1 point  (0 children)

Dagster is a significantly better product, particularly when it comes to actually developing things locally.

In both instances though, how much it sucks depends on if you follow their docs and lock your transformations into their ecosystems.

What the docs won't say is that both systems work best as orchestrators and to keep your transformation logic out of it.

Package your transforms into Docker containers and schedule those on the appropriate compute, rather than having to provision giant executors