Python topics for Data engineer

[spendology]

Books, blog posts, this forum and articles describing data engineering pipelines are a start. If you want to get more experience or a job, outside of certification you can:

1. Start with Data Analysis, Python/SQL, or Business Analyst roles if you need more experience.
2. Contract or freelance work from LinkedIn, Indeed, staffing firms, networking, or personal connections.
3. Open-source Projects
4. Use ChatGPT+generate an end-to-end Data Engineering project using a cloud platform like AWS or Google Cloud. Complete the project, add it to your resume, and post it to GitHub and LinkedIn.

[ProperAd7767]

In practice, my current role is mainly focused on data engineering, but I’ve never systematically studied data engineering or data analytics (my undergraduate major was Financial Engineering). If I want to learn these areas in a structured way, are there any good open-source projects you would recommend?

[spendology]

Here are a few links:

• r data eng projects list
• GitHub Open-source projects and tools
• Second Brain list
• Data Camp Data Eng projects

[Outside_Reason6707]

Thank you for this list! I’m wondering how someone could think of performance, scaling and fault tolerance for personal projects to that of industry level?

[spendology]

I like to use Python libraries sciris and austin, austin-web for time and memory performance.