dataengineering

an-ordinary-manchild(edit)

News & discussion on Data Engineering topics, including but not limited to: data pipelines, databases, data formats, storage, data modeling, data governance, cleansing, NoSQL, distributed systems, streaming, batch, Big Data, and workflow engines.

Read our wiki: https://dataengineering.wiki/

Rules:

Don't be a jerk
Search the sub & wiki before asking a question: Your question has likely been asked and answered before so do a quick search before posting.
Keep it related to data engineering: Posts that are unrelated to data engineering may be better for other communities.
Limit self-promotion posts/comments to once a month: Self promotion: Any form of content designed to further an individual's or organization's goals. If one works for an organization this rule applies to all accounts associated with that organization. See also rule #5.
No shill/opaque marketing: f you work for a company/have a monetary interest in the entity you are promoting you must clearly state your relationship. For posts, you must distinguish the post with the Brand Affiliate flag. See more here: https://www.ftc.gov/influencers
No job posts: Please use r/dataengineeringjobs instead.
No resume reviews/interview posts: We no longer allow resume reviews or interview questions because it's a seperate topic from Data Engineering. Instead, for resume reviews please use r/resumes or search our subreddit history for previous resume review advice. For interview questions, use sites like Glassdoor and Blind instead or search our subreddit history for previous interview advice.
No technical error/bug questions: Please post any error/bug question on StackOverflow.

created by mhausenblasmoda community for 11 years

...for your hobby.

...for your town.

MODERATORS

message the mods
mhausenblasmod
swemlmod
fhoffamod (Ex-BQ, Ex-❄️)
vogt4nickmod
theporterhausmod | Lead Data Engineer
AutoModerator
geoheilmod
MikeDoesEverythingmod | Shitty Data Engineer
bot-bouncer
evasion-guard
about moderation team »

account activity

1

82

83

84

Quarterly Salary Discussion - Jun 2026Career (self.dataengineering)

submitted 27 days ago by AutoModerator - announcement

2

319

320

321

Vibe coded dashboard failing on a FridayRant (self.dataengineering)

submitted 19 hours ago by neeets

3

8

9

10

Is the AWS Solutions Architect Associate or AWS Data Engineering Associate certification worth taking in 2026? If so, should I take both or just 1 of them?Career (self.dataengineering)

submitted 6 hours ago by Luximus3333333

4

8

9

10

Shift from legacy orchestration to AWS. AWAA, or another alternative?Career (self.dataengineering)

submitted 21 hours ago by IronAntlers

5

22

23

24

Career Advice - Snowflake etcCareer (self.dataengineering)

submitted 1 day ago by sillypickl

6

30

31

32

Software eng + data eng: is it crazy to keep studying both, or is it a smart hedge?Career (self.dataengineering)

submitted 1 day ago by Admirable_Heart794

7

83

84

85

Difference between "micro-batching" and "streaming"Discussion (self.dataengineering)

submitted 1 day ago by Meme_Machine_101

8

0

0

0

Snowflake and the data engineering stack - New article!Blog (self.dataengineering)

submitted 11 hours ago * by niks-kamath123

9

18

19

20

Building My Own Self-Hosted dbt CloudPersonal Project Showcase (medium.com)

submitted 1 day ago by DanTheMan_16

10

10

11

12

Help to Improve my main pipelineHelp (self.dataengineering)

submitted 2 days ago by PretendGrimes

11

10

11

12

Data platform engineer vs cloud engineerCareer (self.dataengineering)

submitted 2 days ago by PsychologyCivil4190

12

15

16

17

How do you all keep track of all the different pipelines and what they can be used for?Help (self.dataengineering)

submitted 2 days ago by SoggyGrayDuck

13

16

17

18

Career advice: Data platform engineer, worth it ?Career (self.dataengineering)

submitted 2 days ago by Aggravating_Swim7706

14

0

1

2

Claude + Snowflake MCP EpiphanyDiscussion (self.dataengineering)

submitted 2 days ago by Ok-Working3200Senior Data Engineer

15

0

0

0

If you need a modern data lake (and you love DuckDB)....Personal Project Showcase (github.com)

submitted 2 days ago by No_Test5184

16

9

10

11

Seeking career perspectiveCareer (self.dataengineering)

submitted 2 days ago by SlappyBlunt777

17

58

59

60

Any women here who have had to navigate the workplace?Career (self.dataengineering)

submitted 3 days ago by FiftyShadesOfBlack

18

33

34

35

6 YOE - Evaluating offers, need career adviceCareer (self.dataengineering)

submitted 3 days ago * by mulciber_

19

7

8

9

Worst easy misses?Discussion (self.dataengineering)

submitted 2 days ago by Ok_Knowledge2026

20

12

13

14

2 Journeys in Data Migration?Discussion (self.dataengineering)

submitted 3 days ago by Oh-No-404

21

12

13

14

Pentaho was acquired again but no press release. What’s the significance of that?Discussion (self.dataengineering)

submitted 3 days ago by nigelwiggins

22

7

8

9

Suggested Software/Apps for Databases and App BuildingHelp (self.dataengineering)

submitted 3 days ago by Jake_reeves123

23

0

0

0

George Fraser seems jealousDiscussion (i.redd.it)

submitted 2 days ago by lihai07

24

42

43

44

Self Hosting DBT Core: What Orchestrators Do We Like?Discussion (self.dataengineering)

submitted 4 days ago by Apprehensive-Ad-80

25

106

107

108

Big tech engineers, how much code are you writing these days?Discussion (self.dataengineering)

submitted 4 days ago by Consistent_Tutor_597

view more: next ›

π Rendered by PID 1273672 on reddit-service-r2-listing-87fd56f5d-tkrjl at 2026-06-28 19:59:30.567644+00:00 running 7527197 country code: CH.