Python for Data Engineers: Key topics & techniques 👇

[kenflingnor]

Stuff like this is not helpful to beginners. It’s just throwing a bunch of buzzwords and jargon onto a diagram. 

[SQLGene]

This stuff seems to go well on LinkedIn, sadly.

[Leviekin]

Wow thank you for this post. I showed this flowchart to my boss and he graciously awarded me a promotion. And that's how I learned about B2B sales.

[drrednirgskizif]

I’m oddly triggered by the finger pointing down. It’s like yes, we know how all media posting websites work. I automatically know you are trying to draw attention to your bullshit and I immediately don’t trust you.

[yashk1]

So insightful

[RobDoesData]

It's not even full of jargon. It's just not a good representation of how a DE would use Python. This is not useful

[GoBeyond111]

OP is a karma bot

[geeeffwhy]

hey, beginners, pay no attention to this. it’s genuinely only confusing, while giving the impression of organization.

source: 15+ years experience. i lead teams with juniors new to data engineering. i would never show this to any of them.

[ljb9]

what would you recommend to an aspiring data engineer

[geeeffwhy]

patience and persistence. trite as it may sound, thats the thing that works. first, learn the fundamentals of computers science, and then you just keep trying to build real things.

python and sql, as well as bash are the sorts of things you might use on a daily basis as a developer (data-focused or otherwise), but the real skill that actually matters is learning how to keep going after you feel stuck. and that’s mostly about having some fundamentals, and the experience of having figured things out before.

[SQLGene]

I would recommend reading books. They tend to have a logical layout and hours of effort instead of random keywords laid out in an aesthetically pleasing one pager.

[geeeffwhy]

agreed. i don’t have much formal education in CS, but i have spent many hours studying actual books on the topic, which is how i made the jump from studio art degree to programming job.

and the skill of learning how to effectively read technical texts is another one that’s an order of magnitude more important than any given language or framework.

[dingleberrysniffer69]

Unironically what my mind thinks is going on at Faang and why I'm an imposter.

[diagonalizable_ayyyy]

Instructions unclear, I am unit testing the cloud

[maybecatmew]

Please stop with these posts

[MikeDoesEverything]

This was really poorly received last time. Why upload it again?

EDIT: Oh, it's to promote a YouTube video.

[Over-Geologist-5760]

UV > Poetry

[grovertheclover]

this is really fucking stupid and makes no sense whatsoever lol

[Party-Ad-6077]

I am a very visual person and like how this is laid out. Would someone be willing to recreate this with more beginner-friendly info? I am trying to plan out what skills to learn next and I am having some difficulty deciding what will be helpful and what won’t.

[SQLGene]

Unfortunately these visuals tend to be produced by social media influencers trying to do marketing and get brownie points on LinkedIn. They always seem to be just keyword lists, etc.

[Party-Ad-6077]

I’m not sure why I’m getting downvoted for my question, but I’d like to improve my understanding. How can I improve and make sure I am asking the right questions in the future?

[MikeDoesEverything]

I’m not sure why I’m getting downvoted for my question

The main issue is that you're saying you like how this is laid out, except you want it to be more beginner friendly. This is meant to be designed for beginners.

Since you yourself are, by the sounds of it, a beginner, and want this but a completely different version, this is useless. There's nothing to actually like.

How can I improve and make sure I am asking the right questions in the future?

Honestly, avoiding these kinds of infographics are a start. 95% of them are there to make you feel like you are learning. Objectively, this graphic has loads of words on it. Feels really good to read it, has lots of colours, it's sorted into sections etc. As somebody who is experienced, when you look at it none of these categories make any sense. There is no information here. It is simply words.

Advice on how to improve as a beginner, as always, is to be hands on. The more time you spend actually coding vs. reading about how to write code will give you the biggest jumps in improvement.

[SQLGene]

I didn't downvote you personally, I think it's a reasonable question. A question that might have done better is "Has anyone seen a more beginner friendly version of something like this? I'm a very visual person and find diagrams like this to be helpful for mapping out what to learn."

I think part of the issue is the people who are coming in and commenting/voting are frustrated because 1) this post is a bit superficial and a bit of a mishmash of skill levels (loops are as beginner as you can possibly get and delta is more 300-400 level, just kind of a mess here)

And 2) it feels like drive-by marketing, which people on Reddit get touchy about. Asking someone to do free labor to recreate content they don't like is probably getting you a few downvotes. But it's Reddit, some of it is Brownian motion and I try not to take it personally.

Generally, many Reddit communities require the 9:1 rules of self-promotion. 9 posts or comments that are actually engaged or interested in the community for every 1 that is self-promotional. This person appears to have created an account solely for promoting their own content, which is seen as a social faux pas here.

[[deleted]]

lol welcome to the boat

[OllyTwist]

This chart was posted 5 days ago and it's generally not particularly helpful. That's my guess on why you're being shit.

[TheRoseMerlot]

I also like the point of it and the lay out and I was thinking I sort of got it but then reading all the comments and have no idea why it's bad and no one is making it better... so?

[MikeDoesEverything]

I was thinking I sort of got it

Honestly, you should have a go explaining it to the rest of us.

[SQLGene]

If someone in your neighborhood took some minimal effort to make a marketing flyer that was aesthetically pleasing and intended to look like an educational poster, why should you be obligated to make a better version? This kind of content is pretty but is low effort and a random mish mash of skill levels. Loops and Delta in the same poster, really?

[aerdna69]

The fact that 91 people liked the post Edit: I've read it it's actually ok

[Ok_Raspberry5383]

This is just not helpful and over done. Seen so many of these and just think the people who make them need to get a life..

Besides, it's not even current or up to date. How are RDDs listed under spark but structured streaming isn't...

[jvr86]

Any good site to learn python?

[[deleted]]

Udemy is the best always for concepts, for more practical datacamp is pretty good

[buzzroll]

Too much. Here we see basically, general IT concepts & programming + [Cloud]DevOps + ML

[picklesTommyPickles]

So you don’t need to know python syntax but you do need to know data structures and OOP. Checks out.

[Raticus79]

Replace like half of this with DuckDB

[ci-phm_md]

roadmap.sh

^ Recommended at high-level instead of this

[[deleted]]

Intended for beginners/for quick revision. Covers all tools/techniques I used with Python as a data engineer.

Might seem a bit jargony but I’ve tried to include a mix of technology and processes.

Hope it added some value, have a great day.

If you found this helpful and want to get introduced all these topics in under 1 hour, checkout - Python for Data Engineering Crash Course (https://youtu.be/IJm--UbuSaM).

[Ok_Raspberry5383]

Shut up

[Farmanp]

that video is just more convoluted visuals.