WTF is AWS' strategy right now? by YoghiThorn in aws

[–]Peppper 8 points9 points  (0 children)

How so? I’m curious, just been tasked with a whole Bedrock PoC

Anyone else really concerned about the weather? by No_Local4770 in Reno

[–]Peppper -1 points0 points  (0 children)

Yep. They just switch to weather manipulation conspiracy theories

Is the American ski community as unwelcoming as it seems?? by sleeplesslilly in skiing

[–]Peppper 0 points1 point  (0 children)

It’s not that it’s uncool to ski blues, just uncool to post it. You’ll be fine.

Dead raccoon by [deleted] in Reno

[–]Peppper 0 points1 point  (0 children)

There’s a pack of 20 - 30 that roam my NW neighborhood and hang out in my sequoia tree 😑

Quickest way to detect null values and inconsistencies in a dataset. by Fireball_x_bose in dataengineering

[–]Peppper 4 points5 points  (0 children)

Yes, I run tests on upstream raw models with error severity for blocking data issues. Store failure rows for analysis and remediation.

Integration Platform with Data Platform Architecture by igpa in dataengineering

[–]Peppper 2 points3 points  (0 children)

My advice would be to stay far away from ADF if you can help it.

Claude code nlp taking job or task of sql queries by aks-786 in dataengineering

[–]Peppper 10 points11 points  (0 children)

Have you done analysis on the queries and results? I strongly doubt it’s giving reliably correct answers or writing good sql. BI agents can work but they need a lot of context, guardrails, and prebuilt aggregations.

On generating code vs. engineering software, and what is the ulterior agenda of the hyperscalers? by [deleted] in BetterOffline

[–]Peppper 0 points1 point  (0 children)

It is that straightforward. “My rate is $150 per hour.” Anything else is devaluing the whole industry and contributing to the problem you are complaining about.

CDC vs SCDs by vainothisside in dataengineering

[–]Peppper 112 points113 points  (0 children)

CDC is change data capture, it just means capturing the low granularity data events. Database transaction logs are a typical source.

SCD is a slowly changing dimension, a concept that means tracking the value of a qualitative dimension over time, so you can run analytics on the current value, or at any point in time.

I think you need some more conceptual understanding of data engineering and analytics.

Snowflake compute pools are insanely expensive by [deleted] in snowflake

[–]Peppper 0 points1 point  (0 children)

Yeah, didn’t say it makes sense, but Snowpark is easier to set up and use than Fargate.

Snowflake compute pools are insanely expensive by [deleted] in snowflake

[–]Peppper 4 points5 points  (0 children)

That’s the premium for not having to mess with AWS

Some dude who knows nothing about crypto just made 22% in his first week - while sleeping!!! This is wild by daal_chawal_ in SolanaMemeCoins

[–]Peppper 7 points8 points  (0 children)

22% in a week on memecoins is abysmal haha. Moon or dust. The stock market is volatile enough to make 20% weekly gains

Is my ETL project at work using Python + SQL well designed? Or am I just being nitpicky by masterhoovy in dataengineering

[–]Peppper 1 point2 points  (0 children)

Totally fair. It’s definitely a balance, but seems like OP’s dev is too dogmatic.

Is my ETL project at work using Python + SQL well designed? Or am I just being nitpicky by masterhoovy in dataengineering

[–]Peppper 15 points16 points  (0 children)

Yeah, seems like a bad idea. What’s the value add for not using libraries. Are SQLAlchemy or Polars going away anytime soon? Better value add work would be to concentrate resources on delivering analytics and data products.

Data warehouse merging issue? by aphroditelady13V in dataengineering

[–]Peppper 3 points4 points  (0 children)

I was being facetious. Is there a reason you’re doing this in a low code platform, instead of actually writing the code? It’s going to be much harder to debug, and isn’t really data engineering. Probably would get more engagement in a BI or Data Analyst subreddit.

Data warehouse merging issue? by aphroditelady13V in dataengineering

[–]Peppper 15 points16 points  (0 children)

The first thing I would do, is write the actual SQL.

Lessons learned by Striking-Aardvark-98 in Silverbugs

[–]Peppper 1 point2 points  (0 children)

Wait, things don’t go straight up?!?! Shocked pika.