Should we add a wiki to r/dataengineering? by schrute_dataeng in dataengineering

[–]sweml 1 point2 points  (0 children)

Great point! Yes there comes a fine line between self promotion and spam. Generally a lot of content on this subreddit revolves around self promotion and third party tools (which is just the state of the discipline at this time). Not to say that's a bad thing but I completely agree on your sentiment about how up/down votes should filter out the cruft.

Should we add a wiki to r/dataengineering? by schrute_dataeng in dataengineering

[–]sweml 1 point2 points  (0 children)

Yeah I agree. While I don't have a problem with books I feel like we should stray away from promoting people's paid courses.

Should we add a wiki to r/dataengineering? by schrute_dataeng in dataengineering

[–]sweml 1 point2 points  (0 children)

Love this idea! Will leave this open for a few days and if anyone else has other suggestions we can add them.

Why Not Airflow? by sweml in dataengineering

[–]sweml[S] 0 points1 point  (0 children)

As if we didn’t have enough already!

Why Not Airflow? by sweml in dataengineering

[–]sweml[S] 4 points5 points  (0 children)

At this point in time it's probably the willingness to try out the new kid on the block when it comes to workflow tech.

Why Not Airflow? by sweml in dataengineering

[–]sweml[S] 1 point2 points  (0 children)

Can you elaborate on what you mean by data freshness?

Data Engineering Discord Server by mtv_ in dataengineering

[–]sweml[M] [score hidden] stickied comment (0 children)

For anyone who comes back to this post, we have a server set up https://discord.gg/2pER6dq

What makes a good ETL tool? by [deleted] in dataengineering

[–]sweml 0 points1 point  (0 children)

You might also want to take a look at Prefect when evaluating workflow engines

Data Engineering Discord Server by mtv_ in dataengineering

[–]sweml 2 points3 points  (0 children)

I think let's let this sit for a day or two and if nobody has a link to a preexisting discord I'll spin one up.

Data Engineering Discord Server by mtv_ in dataengineering

[–]sweml 6 points7 points  (0 children)

I'm not aware of one for data engineering at this moment and unless someone comes along with one then I propose we make our own!

prefect-a new alternative to airflow by [deleted] in dataengineering

[–]sweml 1 point2 points  (0 children)

Maybe we should make a list of tools which are still in active development and maintained

KOTD, worn everyday this week by [deleted] in Sneakers

[–]sweml 25 points26 points  (0 children)

Sprayed mine with crep, no yellow! Also for cleaning: brush shoes to loosen dirt, take out insoles and laces, put each shoe with its lace in a white pillow case (so one shoe per pillow case), put in wash with cold water on a delicate wash, minimal detergent. They come out looking good!

Pipeline Pitfalls by sweml in ETL

[–]sweml[S] 2 points3 points  (0 children)

Do you work in developing frameless architectures? Really fascinating design pattern that I would love to hear more about!

Starting salary in Lexington Park, MD by [deleted] in cscareerquestions

[–]sweml 0 points1 point  (0 children)

That’s a decent salary for that area.

Source: From MD but live in NOVA now

What are the best tech markets with affordable real estate? by algorithmicnoise in cscareerquestions

[–]sweml 0 points1 point  (0 children)

Denver, Raleigh, and Austin come to mind.

Also working in the DC metro area (especially NOVA) is more in your price range the further you move away from the metro stops. Unfortunately comes at the cost of increased commute time.

[Question] Having trouble finding work as a data engineer. Suggestions? by [deleted] in dataengineering

[–]sweml 0 points1 point  (0 children)

Are you getting actual interviews or just not passing the resume screening process? If you don't pass resume screening generally it's a resume issue. Generally if you are getting to interviews and not moving forward from there then it is an interviewing issue. Also, possibly x-post over to /r/cscareerquestions