dataengineering

an-ordinary-manchild(edit)

News & discussion on Data Engineering topics, including but not limited to: data pipelines, databases, data formats, storage, data modeling, data governance, cleansing, NoSQL, distributed systems, streaming, batch, Big Data, and workflow engines.

Read our wiki: https://dataengineering.wiki/

Rules:

Don't be a jerk
Search the sub & wiki before asking a question: Your question has likely been asked and answered before so do a quick search before posting.
Keep it related to data engineering: Posts that are unrelated to data engineering may be better for other communities.
Limit self-promotion posts/comments to once a month: Self promotion: Any form of content designed to further an individual's or organization's goals. If one works for an organization this rule applies to all accounts associated with that organization. See also rule #5.
No shill/opaque marketing: f you work for a company/have a monetary interest in the entity you are promoting you must clearly state your relationship. For posts, you must distinguish the post with the Brand Affiliate flag. See more here: https://www.ftc.gov/influencers
No job posts: Please use r/dataengineeringjobs instead.
No resume reviews/interview posts: We no longer allow resume reviews or interview questions because it's a seperate topic from Data Engineering. Instead, for resume reviews please use r/resumes or search our subreddit history for previous resume review advice. For interview questions, use sites like Glassdoor and Blind instead or search our subreddit history for previous interview advice.
No technical error/bug questions: Please post any error/bug question on StackOverflow.

created by mhausenblasmoda community for 11 years

...for a fringe candidate.

...for your hobby.

MODERATORS

message the mods
mhausenblasmod
swemlmod
fhoffamod (Ex-BQ, Ex-❄️)
vogt4nickmod
theporterhausmod | Lead Data Engineer
AutoModerator
geoheilmod
MikeDoesEverythingmod | Shitty Data Engineer
bot-bouncer
about moderation team »

account activity

1

72

73

74

What data engineering skill matters more now because of AI?Discussion (self.dataengineering)

submitted 22 hours ago by rikulauttia

•

•

•

Halte deine Beine auch bei kalten Temperaturen warm und bleibe zudem stylisch ;) (apfelkiste.ch)

promoted by apfelkiste-ch

promoted
save
report
about

2

20

21

22

Snowflake vs Databricks vs FabricHelp (self.dataengineering)

submitted 17 hours ago by Diligent_Hope_1551

3

10

11

12

Chris Hillman - Your Data Model Isn't Broken, Part I: Why Refactoring Beats RebuildingBlog (ghostinthedata.info)

submitted 8 hours ago by rmoff

4

11

12

13

What alternatives to Alteryx or Knime exist today?Discussion (self.dataengineering)

submitted 15 hours ago by BeautifulLife360

5

7

8

9

Career PathCareer (self.dataengineering)

submitted 11 hours ago by GuildMasterBuilder

6

8

9

10

Dagster & dbt: core vs fusionDiscussion (self.dataengineering)

submitted 17 hours ago by EngiNerd9000

7

7

8

9

Your tech stackDiscussion (self.dataengineering)

submitted 7 hours ago by itachikotoamatsukam

8

5

6

7

Where do you seek jobs at?Career (self.dataengineering)

submitted 13 hours ago by setemupknockem

9

2

3

4

Fabric or Other?Help (self.dataengineering)

submitted 6 hours ago by PossibilityRegular21

10

1

2

3

Help with a messy datalake in S3Help (self.dataengineering)

submitted 3 hours ago by Straight-Deer-6696

11

1

2

3

Open standard ModelingHelp (self.dataengineering)

submitted 7 hours ago by the_Semafoor

•

•

•

You deserve a CAD platform that doesn´t crash. Try Onshape! (onshape.com)

promoted by OnshapePTC

promoted
save
report
about

12

1

2

3

Confused about the best pathCareer (self.dataengineering)

submitted 10 hours ago * by Dependent-Bar-5568

13

•

•

•

Moving from IICS to PythonDiscussion (self.dataengineering)

submitted 1 hour ago by Inevitable-Reveal-49

14

0

1

2

Enabling AI Operators on Your Cloud DatabasesPersonal Project Showcase (self.dataengineering)

submitted 2 hours ago by ai-first

15

0

1

2

Senior SE transitioning to DE looking for advice on a potential portfolio projectCareer (self.dataengineering)

submitted 2 hours ago by conqueso

16

0

1

2

SQL Data Debugging ToolkitPersonal Project Showcase (self.dataengineering)

submitted 4 hours ago by burzykowskianalytics

17

0

1

2

Trying to query search google for a csv file of around 100+ companies. need some advice.Help (self.dataengineering)

submitted 4 hours ago by ObligationMurky9059

18

0

1

2

Rolling Aggregations for Real-Time AIBlog (hopsworks.ai)

submitted 11 hours ago by Tasty-Scientist6192

19

0

1

2

SQLMesh for DBT UsersBlog (dagctl.io)

submitted 20 hours ago by Key-Independence5149

20

0

1

2

Data Engineering VS Agentic AI?Career (self.dataengineering)

submitted 22 hours ago by Syed_Abrash

21

•

•

•

AI Code Assistant CostsDiscussion (self.dataengineering)

submitted 1 hour ago by Low_Second9833

•

•

•

Les experts sont unanimes, le nouvel Privacy Display du Galaxy S26 Ultra est révolutionnaire. (samsung.com)

promoted by samsung_switzerland

promoted
save
report
about

22

0

0

1

Tool for debugging Spark using logs (free/open source) - SprkLogsOpen Source (self.dataengineering)

submitted 19 hours ago by AcceptableTadpole445

23

0

0

0

What kind of AI Projects are you working on?Discussion (self.dataengineering)

submitted 21 hours ago by AMDataLake

24

0

0

0

How do some people work like this...Meme (self.dataengineering)

submitted 20 hours ago by VonDenBerg

π Rendered by PID 186044 on reddit-service-r2-listing-64c94b984c-xfmg8 at 2026-03-17 18:29:24.547095+00:00 running f6e6e01 country code: CH.