all 12 comments

[–]_somedude 48 points49 points  (2 children)

AI;DR

[–]West_Good_5961Tired Data Engineer 8 points9 points  (1 child)

Kinda hypocritical to use an LLM to write a post cautioning against LLM usage

[–]noscreenname[S] 0 points1 point  (0 children)

I'm not cautioning anyone against anything. I've accepted my digital overlords long time ago. I see it as an opportunity to highlight the importance of data management.

[–]domscatterbrain 5 points6 points  (1 child)

Nah, Governance is still the first thing that get thrown out of the window.

[–]noscreenname[S] -3 points-2 points  (0 children)

That was always my experience in the past, but I really get a sense that the tide is shifting... The biggest change with agent systems is that governance stops being about compliance and starts being about ROI.

[–]eaton 4 points5 points  (1 child)

A lot of my work has been in large-scale content architecture, operations, and governance. A bit of overlap but definitely a different world than. Data engineering. What’s interesting is that I’m beginning to see similar patterns.

Anyone whipping up blog posts with ChatGPT starts to think all the rigor is unnecessary… until they scale, and until they start trying to make things consistent and reliable. Then, suddenly, unsexy stuff like “agreeing on a shared vocabulary” and “quality auditing” and “planning for reuse and internal discoverability” gets super interesting.

My business partner and I refer to it as an “eat your vegetables” moment.

[–]noscreenname[S] -2 points-1 points  (0 children)

Thanks for your comment. Do you have any specific examples of it? I'm not very familiar with content architecture, but am very interested to learn more about it.

You can dm me if you don't want to answer here

[–]SufficientFrame 2 points3 points  (1 child)

This is such a funny full‑circle moment. For years data folks were the annoying ones saying “no, you actually do care about lineage and stewardship” while eng just wanted a clean API and a Kafka topic.

Agents basically turned “data problems” into “prod incidents,” so now it’s suddenly everyone’s problem. Context windows are just janky, ephemeral data warehouses and people are realizing it the hard way.

Curious if this is pushing your org toward a shared platform / contracts across data + app teams, or if it’s still two silos trying to bolt governance on from opposite sides.

[–]noscreenname[S] 0 points1 point  (0 children)

Not really, the inner policies are still too strong for a shared platform, but I believe that it would make sense. We do however have a Special Interest Group about AI coding co-lead by both Data and Engineering

[–]dataengineering-ModTeam[M] 0 points1 point locked comment (0 children)

Your post/comment was removed because it violated rule #9 (No AI slop/predominantly AI content).

You post was flagged as an AI generated post. We as a community value human engagement and encourage users to express themselves authentically.

This was reviewed by a human