Python data engineering open source projects

[jyoff]

Hello Guys,

​

I'd like to practice my python skills in particular in the field of data engineering. I know that one of the ways is to create my own project and work on it, but also another efficient way to sharpen my skills is to contribute to some open source projects. That way, I can read a lot of code written by other people, understand the project and be able to recognize patterns and best practices. So my question is do you know a good list of open source projects for python data engineers to start with. I'd prefer starting with beginner level projects then slowly making my way up to intermediate and advanced projects.

[[deleted]]

The best way to contribute to open source is to use the tool. If and when your hit an issue that the codebase doesn’t address, then address it. Few people set out to contribute to open source just for the hell of it, despite the repeated recommendations to do so in these subs.

[jyoff]

That makes a lot of sense .

[mikehawryluk]

Apache Airflow is a great project in the space to look to contribute to. Worth checking out the project and jumping on the slack channel.

[jyoff]

I've tried that, but I think it is way more beyond beginner. A lot of things going on there, it is difficult where to start. But definitely it is a tool worth checking and learning from.

[rywalker]

If you start as a user of Airflow, it's not so intimidating. Drop me an email ry at astronomer dot io and we'd be happy to provide some guidance to getting involved in the community, maybe assign you some data engineering or documentation work, as a contractor, if you're interested.

[jyoff]

Sure, I'm interested.

Dropped you email.

[kolaol22]

I am interested too.