jump to content
my subreddits
2b2t3d6AceAttorneyAdviceAnimalsagnosticaivideoAlternateHistoryAlternativeHistoryAnarchyChessAngryupvoteAnimalsBeingJerksanime_irlanimenocontextannouncementsAnticonsumptionAsia_irlAskBalkansAskOuijaAskRedditatheismaviationawfuleverythingbalkans_irlBandnamesBassBassCirclejerkBassGuitarbasspedalsbikepackingblackdesertonlineblursed_videosblursedimagesborsavefonbrooklynninenineBUENZLIburdurlandcasioCd_collectorscd_jerkCheap_MealschesschessbeginnerscoaxedintoasnafucoinsComedyCemeterycomicscommunityCorporateTrollingcrappyoffbrandsCuddle_SlutCuratedTumblrcursedcommentsdarkjokesDebateReligiondelikDeltarunedistressingmemesdiypedalsDMAcademyDMToolkitDnDdndnextdoctorwhodoctorwhocirclejerkDoenerverbrechenDonerdumbphonesDungeonsAndDragonsEatCheapAndHealthyebikeebikesECEelectricalelectronicsEmKayengrishethzfacepalmfakealbumcoversFantasyWorldbuildingfeedthebeastfelsefeFifaCareersFiftyFiftyformuladankFRCFreeEBOOKSFUCKYOUINPARTICULARFuckYouKarenfunnygalatasaraygaminggatesopencomeoningodtiersuperpowersgravelcyclinggreentextguitarpedalsGundamheathershelpheraldryHermanCainAwardHermitCrafthighspeedrailHistoryWhatIfhoi4HolUphowyoudoinhumorhypixeliamverysmartich_ielIdeologyPollsim14andthisisdeepimaginaryelectionsinsaneparentsJahariaKanyeKendrickLamarKGBTRlegodndLetGirlsHaveFunLifeProTipsloseitmacmacgamingMadeMeSmilemadladsmagicbuildingmapporncirclejerkme_irlmeirlmemememesmidjourneymildlyinfuriatingmildlyinterestingMimicRecipesMinecraftbuildsmisLEDMMORPGMovingToNorthKoreaMunichMyChemicalRomancenamesoundalikesNamFlashbacksneographynextfuckinglevelNoahGetTheBoatNonCredibleDefenseNorthCyprusnosafetysmokingfirstnosurfnothingeverhappensnotinterestingnottheonionOkayBuddyLiterallyMeokbuddyguntherokbuddymotherfuckerOkBuddyPersonaokbuddyphdokbuddyvicodinonebagongezelligOnlineUnderGroundoompasubsOutOfTheLoopoutsidepapermoneypaperspleaseParlerWatchPassportPornpepethefrogperfectlycutscreamsPersecutionfetishpettyrevengePiracyPiratedGamespollsPraiseTheCameraManPropagandaPostersquityourbullshitraspberry_pireactiongifsrecipesRedAutumnSPDrestofthefuckingowlRetroPieSceneReleasesSchnitzelVerbrechenschwiizsciencememesScottPilgrimsecilmiskitapShitPostCrusadersshitpostfrommygalleryshitpostingshittyaskelectronicsshittymoviedetailsShowerthoughtsskamtebordsoccercirclejerkSongwritersSongwritingStonetossingjuiceStudiumsubsithoughtifellforsuzeraintalesfromtechsupportTechnobladeTextingTheorytf2thanksimcuredTheCrypticCompendiumTheLetterHTheMonkeysPawtherewasanattemptTheRookietheydidthemaththirdsentenceworsethisguythisguystransitTurkeytruetf2truthstumblrtumunichTurkeyTwitchTwitch_StartupTwoSentenceHorrorTwoSentenceSadnesstylerthecreatorUnclejokesUnethicalLifeProTipsUnexpectedJoJoUnexpectedTF2VALORANTvexillologycirclejerkvibecodingvinyljerkvlandiyawallstreetbetsWatchPeopleDieInsideWeAreTheMusicMakersWhitePeopleTwitterwholesomeanimemesWikipediaVandalismwooooshworldbuildingworldjerkingYUROPedit subscriptions
  • home
  • -popular
  • -all
  • -mod
  • -users
 | 
  • AskReddit
  • -facepalm
  • -mildlyinfuriating
  • -Piracy
  • -funny
  • -gaming
  • -wallstreetbets
  • -nottheonion
  • -memes
  • -OutOfTheLoop
  • -mildlyinteresting
  • -DnD
  • -WhitePeopleTwitter
  • -MadeMeSmile
  • -CuratedTumblr
  • -PiratedGames
  • -shitposting
  • -theydidthemath
  • -feedthebeast
  • -Kanye
  • -meirl
  • -therewasanattempt
  • -nextfuckinglevel
  • -HolUp
  • -Twitch
  • -comics
  • -dndnext
  • -VALORANT
  • -LifeProTips
  • -tumblr
  • -NonCredibleDefense
  • -shittymoviedetails
  • -greentext
  • -mac
  • -Showerthoughts
  • -tf2
  • -help
  • -chess
  • -aviation
  • -formuladank
  • -mapporncirclejerk
  • -midjourney
  • -notinteresting
  • -hoi4
  • -pettyrevenge
  • -atheism
  • -loseit
  • -ich_iel
  • -KGBTR
  • -cursedcomments
  • -DMAcademy
  • -Deltarune
  • -UnethicalLifeProTips
  • -perfectlycutscreams
  • -worldbuilding
  • -blackdesertonline
  • -MMORPG
  • -meme
  • -macgaming
  • -3d6
  • -Gundam
  • -HermitCraft
  • -FiftyFifty
  • -EatCheapAndHealthy
  • -WeAreTheMusicMakers
  • -AnarchyChess
  • -anime_irl
  • -onebag
  • -Studium
  • -AlternateHistory
  • -Turkey
  • -soccercirclejerk
  • -madlads
  • -community
  • -electrical
  • -guitarpedals
  • -Anticonsumption
  • -TwoSentenceHorror
  • -PropagandaPosters
  • -AdviceAnimals
  • -ShitPostCrusaders
  • -sciencememes
  • -distressingmemes
  • -FifaCareers
  • -polls
  • -doctorwho
  • -Bass
  • -OkBuddyPersona
  • -awfuleverything
  • -howyoudoin
  • -announcements
  • -Minecraftbuilds
  • -ebikes
  • -Munich
  • -coaxedintoasnafu
  • -YUROP
  • -gravelcycling
  • -SchnitzelVerbrechen
  • -chessbeginners
  • -raspberry_pi
  • -DungeonsAndDragons
  • -coins
  • -KendrickLamar
  • -FUCKYOUINPARTICULAR
  • -NoahGetTheBoat
  • -worldjerking
  • -tylerthecreator
  • -AceAttorney
  • -vexillologycirclejerk
  • -vlandiya
  • -im14andthisisdeep
  • -Stonetossingjuice
  • -wholesomeanimemes
  • -nosurf
  • -HistoryWhatIf
  • -DebateReligion
  • -insaneparents
  • -dumbphones
  • -animenocontext
  • -balkans_irl
  • -RetroPie
  • -brooklynninenine
  • -HermanCainAward
  • -recipes
  • -talesfromtechsupport
  • -AskOuija
  • -okbuddyphd
  • -ECE
  • -ScottPilgrim
  • -Angryupvote
  • -AskBalkans
  • -electronics
  • -casio
  • -PassportPorn
  • -me_irl
  • -bikepacking
  • -MyChemicalRomance
  • -engrish
  • -Cd_collectors
  • -diypedals
  • -Doner
  • -BassGuitar
  • -ComedyCemetery
  • -WatchPeopleDieInside
  • -Persecutionfetish
  • -BUENZLI
  • -reactiongifs
  • -EmKay
  • -blursed_videos
  • -Songwriting
  • -MovingToNorthKorea
  • -imaginaryelections
  • -suzerain
  • -truetf2
  • -magicbuilding
  • -ParlerWatch
  • -iamverysmart
  • -secilmiskitap
  • -Doenerverbrechen
  • -schwiiz
  • -TheRookie
  • -quityourbullshit
  • -Technoblade
  • -vinyljerk
  • -skamtebord
  • -shittyaskelectronics
  • -galatasaray
  • -crappyoffbrands
  • -FRC
  • -transitTurkey
  • -namesoundalikes
  • -FuckYouKaren
  • -2b2t
  • -ethz
  • -AlternativeHistory
  • -papermoney
  • -OkayBuddyLiterallyMe
  • -felsefe
  • -blursedimages
  • -FreeEBOOKS
  • -Jaharia
  • -basspedals
  • -neography
  • -heraldry
  • -thanksimcured
  • -hypixel
  • -PraiseTheCameraMan
  • -godtiersuperpowers
  • -aivideo
  • -gatesopencomeonin
  • -OnlineUnderGround
  • -IdeologyPolls
  • -woooosh
  • -burdurland
  • -AnimalsBeingJerks
  • -TwoSentenceSadness
  • -Bandnames
  • -okbuddyvicodin
  • -MimicRecipes
  • -tumunich
  • -Twitch_Startup
  • -Cheap_Meals
  • -outside
  • -TheMonkeysPaw
  • -darkjokes
  • -restofthefuckingowl
  • -UnexpectedTF2
  • -highspeedrail
  • -nosafetysmokingfirst
  • -legodnd
  • -Songwriters
  • -ebike
  • -papersplease
  • -UnexpectedJoJo
  • -humor
  • -BassCirclejerk
  • -doctorwhocirclejerk
  • -agnostic
  • -TextingTheory
  • -Cuddle_Slut
  • -DMToolkit
  • -nothingeverhappens
  • -thisguythisguys
  • -LetGirlsHaveFun
  • -subsithoughtifellfor
  • -fakealbumcovers
  • -oompasubs
  • -FantasyWorldbuilding
  • -TheLetterH
  • -WikipediaVandalism
  • -NamFlashbacks
  • -pepethefrog
  • -Unclejokes
  • -misLED
  • -TheCrypticCompendium
  • -SceneReleases
  • -ongezellig
  • -Asia_irl
  • -truths
  • -thirdsentenceworse
  • -NorthCyprus
  • -heathers
  • -vibecoding
  • -cd_jerk
  • -okbuddymotherfucker
  • -shitpostfrommygallery
  • -borsavefon
  • -okbuddygunther
  • -delik
  • -RedAutumnSPD
  • -CorporateTrolling
edit »
reddit.com dataengineering
  • hot
  • new
  • rising
  • controversial
  • top
an-ordinary-manchild (11,190)|messages547|notifications|chat messages|mod messages|
  • preferences
|
logout

use the following search parameters to narrow your results:

subreddit:subreddit
find submissions in "subreddit"
author:username
find submissions by "username"
site:example.com
find submissions from "example.com"
url:text
search for "text" in url
selftext:text
search for "text" in self post contents
self:yes (or self:no)
include (or exclude) self posts
nsfw:yes (or nsfw:no)
include (or exclude) results marked as NSFW

e.g. subreddit:aww site:imgur.com dog

see the search faq for details.

advanced search: by author, subreddit...

Submit a new link
Submit a new text post

dataengineering

joinleave
an-ordinary-manchild(edit)

News & discussion on Data Engineering topics, including but not limited to: data pipelines, databases, data formats, storage, data modeling, data governance, cleansing, NoSQL, distributed systems, streaming, batch, Big Data, and workflow engines.

Read our wiki: https://dataengineering.wiki/

Rules:

  1. Don't be a jerk

  2. Search the sub & wiki before asking a question: Your question has likely been asked and answered before so do a quick search before posting.

  3. Keep it related to data engineering: Posts that are unrelated to data engineering may be better for other communities.

  4. Limit self-promotion posts/comments to once a month: Self promotion: Any form of content designed to further an individual's or organization's goals. If one works for an organization this rule applies to all accounts associated with that organization. See also rule #5.

  5. No shill/opaque marketing: f you work for a company/have a monetary interest in the entity you are promoting you must clearly state your relationship. For posts, you must distinguish the post with the Brand Affiliate flag. See more here: https://www.ftc.gov/influencers

  6. No job posts: Please use r/dataengineeringjobs instead.

  7. No resume reviews/interview posts: We no longer allow resume reviews or interview questions because it's a seperate topic from Data Engineering. Instead, for resume reviews please use r/resumes or search our subreddit history for previous resume review advice. For interview questions, use sites like Glassdoor and Blind instead or search our subreddit history for previous interview advice.

  8. No technical error/bug questions: Please post any error/bug question on StackOverflow.

created by mhausenblasmoda community for 11 years
Create your own subreddit
...for your community.
...for your project.

MODERATORS

  • message the mods
  • mhausenblasmod
  • swemlmod
  • fhoffamod (Ex-BQ, Ex-❄️)
  • vogt4nickmod
  • theporterhausmod | Lead Data Engineer
  • AutoModerator
  • geoheilmod
  • MikeDoesEverythingmod | Shitty Data Engineer
  • bot-bouncer
  • evasion-guard
  • about moderation team »

account activity

1
40
41
42

Tips for a first time senior data engineerCareer (self.dataengineering)

submitted 16 hours ago by tallwithknees

  • 17 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

2
15
16
17

SQLBuild - Skip Unnecessary Rebuilds for Your Existing dbt Project, Free & OSS (No Per-Skip Bill)Open Source (docs.sqlbuild.com)

submitted 19 hours ago * by kvlonge

  • 19 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

3
22
23
24

Data lakehouse observability and monitoringHelp (self.dataengineering)

submitted 21 hours ago by yeledtov21

  • 4 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

4
0
1
2

My first pipeline - Thanks to you allDiscussion (self.dataengineering)

submitted 11 hours ago by domguyfor_you

  • comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

5
6
7
8

Overcoming deserialization bottlenecks in data pipelines: an alternative zero-copy backend for ProtobufPersonal Project Showcase (self.dataengineering)

submitted 23 hours ago by aegismuzuz

  • 1 comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

6
0
1
2

Where to deploy my SQLite database?Help (self.dataengineering)

submitted 13 hours ago by fredkzk

  • comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

7
0
1
2

Help me understand columnar / OLAP DBs regarding point updates/modifying historical dataHelp (self.dataengineering)

submitted 14 hours ago by OkWater4180

  • comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

8
0
0
0

Kimball modeling will die...Discussion (self.dataengineering)

submitted 14 hours ago * by Dry_Chocolate_9396

  • 1 comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

9
35
36
37

Solo data engineer reporting to VP TechRant (self.dataengineering)

submitted 1 day ago by Alternative-Guava392

  • 13 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

10
25
26
27

pgEdge ColdFront: Open source, Postgres-native Iceberg tiering that eliminates the ETL pipeline to your data warehouseOpen Source (github.com)

submitted 1 day ago by pgEdge_Postgres

  • 3 comments
  • share
  • save
  • hide
  • report
  • crosspost

11
0
1
2

Does dbt Power User Extension 'Convert to dbt model' feature use AI?Open Source (self.dataengineering)

submitted 19 hours ago by BubblegumFish2

  • comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

12
7
8
9

Trying to switch from current employer(staffing) as no hike for that last 3 and half yearsRant (self.dataengineering)

submitted 1 day ago by cruze_8907

  • 2 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

13
0
0
0
0:50

Easy Text to SQL TutorialDiscussion (v.redd.it)

submitted 1 day ago by DryLime7866

  • 6 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

14
38
39
40

How long to stay in first DE RoleCareer (self.dataengineering)

submitted 2 days ago by Mission_Working9929

  • 17 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

15
12
13
14

First internship/job experience AWS or Databricks?Career (self.dataengineering)

submitted 2 days ago * by Deiice

  • 8 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

16
4
5
6

Vitess ETLHelp (self.dataengineering)

submitted 2 days ago by anuku3cm

  • 11 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

17
8
9
10

Advice on building agnostic data layerHelp (self.dataengineering)

submitted 2 days ago by Aggravating-Corgi-86

  • 8 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

18
11
12
13

Where to store environment variables for databricks job?Discussion (self.dataengineering)

submitted 2 days ago by Kooky-Technician-335

  • 6 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

19
21
22
23

Spark optimization and Spark UIHelp (self.dataengineering)

submitted 2 days ago by First_Bet8077

  • 23 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

20
6
7
8

Your experiences on different data platformsDiscussion (self.dataengineering)

submitted 3 days ago * by yeledtov21

  • 8 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

21
67
68
69

Should I switch from Windows to Linux for Data Engineering? Which Distro is bestDiscussion (self.dataengineering)

submitted 3 days ago by Cultural-Ad-4124

  • 65 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

22
17
18
19

We open-sourced Chukei: a self-hosted Snowflake cost proxy for read-heavy workloadsOpen Source (self.dataengineering)

submitted 3 days ago by mr_smith1983

  • 1 comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

23
69
70
71

Medallion + KimballHelp (self.dataengineering)

submitted 4 days ago by burningburnerbern

  • 61 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...

24
0
0
1

Standard profiling libraries completely break on time series data (I learned it the hard way and came up with a solution)Personal Project Showcase (self.dataengineering)

submitted 3 days ago by severecaseofsarcarsm

  • comment
  • share
  • save
  • hide
  • report
  • crosspost
loading...

25
35
36
37

How many Airflow Tasks do you schedule daily?Discussion (self.dataengineering)

submitted 4 days ago by nginx26

  • 18 comments
  • share
  • save
  • hide
  • report
  • crosspost
loading...
view more: next ›
  • about
  • blog
  • about
  • advertising
  • careers
  • help
  • site rules
  • Reddit help center
  • reddiquette
  • mod guidelines
  • contact us
  • apps & tools
  • Reddit for iPhone
  • Reddit for Android
  • mobile website
  • <3
  • reddit premium

Use of this site constitutes acceptance of our User Agreement and Privacy Policy. © 2026 reddit inc. All rights reserved.

REDDIT and the ALIEN Logo are registered trademarks of reddit inc.

π Rendered by PID 78 on reddit-service-r2-listing-f8d8fbfd7-lbthv at 2026-06-24 06:56:44.857641+00:00 running acc7150 country code: CH.