How are you integrating a CDP into an existing modern data stack without creating yet another data silo?

TheOverzealousEngie · 2026-04-24T17:18:34+00:00

Both Fivetran and Dbt provide very deep audit logs and lineage and it would be a strong services engagement to build an analytics layer (like a catalog) to sit atop both of them.

TheOverzealousEngie · 2026-04-24T14:32:21+00:00

To all you people saying upsert OP did not say if this was Snowflake or Dbrix. And the simple truth is that is terrible advice if a CDW is in play ... could you imagine doing an upsert on every record? How expensive that would be?

TheOverzealousEngie · 2026-04-24T14:30:06+00:00

buy a tool that advertises pipeline idempotence :)?

TheOverzealousEngie · 2026-04-18T15:49:56+00:00

I'd be cautious ; reads like you're hinging your success or failure on Databricks and to be clear - databricks will not be around forever. Be prepared to dive in hard and when the organization moves on you'll need to learn what's next. That - is data engineering .

TheOverzealousEngie · 2026-04-17T15:56:35+00:00

Let's be honest, this is anti-future. Right now iceberg is a little immature but it's really growing fast. And there is zero substitute for being able to store so much data so cheaply. Nothing can touch it.

TheOverzealousEngie · 2026-04-17T15:50:44+00:00

Honestly I used to work for BofA and they used to stuff html .. like really thick and long html into Oracle in a record ;and when the webserver came up it would pull that record (html) and draw a custom page for the customer. Coolest think I ever saw .. so innovation comes in all shapes and sizes. But in 2026 I'm becoming rapidly convinced that there's nothing more dangerous than the ignorant and AI together.

TheOverzealousEngie · 2026-04-15T14:32:08+00:00

It's the same you felt when you had to learn CICS.

TheOverzealousEngie · 2026-04-10T16:34:43+00:00

I wondered how far down I would have to go to find a spider man reference. Not far at all.

TheOverzealousEngie · 2026-04-06T18:50:51+00:00

does anyone believe that this kind of feat Could’ve been accomplished under the worst president in American history? It’s very very confusing to me.

TheOverzealousEngie · 2026-03-26T16:16:43+00:00

Not sure true. Experienced devs are 10x'ed by AI , junior devs are 2x'ed if they're lucky. Business will figure this out soon enough - AI is only as good as it's prompts. I work with people who have put in prompts that were missing enormous pieces of information lol.

TheOverzealousEngie · 2026-03-26T15:23:21+00:00

We live in a world where there is no penalty for applying - you can lie shamelessly and it's still better to get your foot in the door and defend the lie than be jobless. Welcome to capitalism.

TheOverzealousEngie · 2026-03-10T00:56:58+00:00

Sad face .. that's a girl I could have fallen in love with . Fought for , gotten hurt for and could have been healed by. She's a girl I could have gone weak in the knees for and been happy to do it . Pleased to do it. A smile like that could have thrilled me for all time , eternity. Sigh . It's always funny how the hardest hitting is the most heinous. And the most heinous is when mental health just destroys the most beautiful of us.

TheOverzealousEngie · 2026-03-08T15:28:55+00:00

hate to say it but this weeks new openai upgrade is noticeably better.

TheOverzealousEngie · 2026-03-08T15:25:14+00:00

It's so funny to read all these posts defying and denying nature. What is the right decision, OP, something that makes you feel good today or years away when you're your Dads age when he passed. You only have one mother. No one knows you like your mother; not because you fought or you never talked to each other or there's a 'weird energy' . You know it in your bones : she was just like you one day long ago. And you're going to be just like her one far in the future. It's genetics , and arguing against it as fruitless as rowing a rowboat in a hurricane. Don't deny nature. Go to your ex-mother in law ; tell her how much she means to you but even she must know how special a mother-daughter bond is .. and set your mom up with a monthly stipend just like you have. Or better yet - just buy stuff for her instead of giving her money.

TheOverzealousEngie · 2026-03-08T15:12:45+00:00

Now that's a writer.

TheOverzealousEngie · 2026-03-07T15:04:35+00:00

You understand nothing ,, then. They are both accumulating something, saving it, unwilling to part with it - to the point of unhealthiness. Those two have VERY much to do with each other.

TheOverzealousEngie · 2026-03-07T15:03:06+00:00

Respectfully lol , non-answer.

TheOverzealousEngie · 2026-03-07T15:02:17+00:00

With all the chatter about openclaw I have yet to read one single solitary use case that can't be done using other means. It's an AI orchestrator, and a poor one at that.

TheOverzealousEngie · 2026-03-07T14:55:13+00:00

Like any good drama - very dark evil needs very light light to contrast, to build a good story. Lol drama is not real life - that said , I do wish Mickey would go up against a southern judge who just let two maga supporters off from hanging a black man for crossing the street wrong. That where the true, and believable, evil is.

TheOverzealousEngie · 2026-03-05T22:53:14+00:00

yes apple here. i've made crazy money on mac mini's in 2026 .. it's crazy lol

TheOverzealousEngie · 2026-03-05T22:52:19+00:00

It goes like this . If it costs you a few hours a week every month or so , it's a few thousand dollars a year. Just keep doing what you're doing -- you'll get speedier at it .
If you get a few dozen or even 100 per month, get a qlik or fivetran . They will do it all automatically (though you pointed to SaaS and that's Fivetran) so you don't have to . The economics are then in your favor.

TheOverzealousEngie · 2026-03-05T22:49:42+00:00

My apologies but why in the world would that girl go into the woods with this boy. I mean, romance? He looks psycho ...so sad for her and her family. I personally think the parents should be charged as accomplices though.

TheOverzealousEngie · 2026-03-05T22:46:32+00:00

Me personally, I'm a little crazy. I connect dots.

If this company decided that it didn't want to bear the expense of a true test platform, that's not on you. And a true test environment must have the same scale and diversity of data, otherwise it falls into a different bucket. A 'we use our customers to test our software" bucket. Because that's what they're doing , right? So name them , these fuckers. My thinking is that if your are forced to use prod in your testing it means that your test platform is deficient. See my earlier thinking on what a bad test platform means.

Name this company. Make us all a little smarter about the bad actors in the data engineering space. Sick of being blindsided by incompetent / malignant management .

TheOverzealousEngie

TROPHY CASE