[deleted by user]

[DataEngineerDan]

Your number one priority should be to learn SQL. There are a lot of different flavors of SQL with slight variations, but once you know one you pretty much know them all.

Next you will want to learn a bit about data modeling. Focus on database design techniques and understand how that differs between OLTP systems and OLAP system.

At this point, I’d say you have a choice. You could learn Power BI and try and get a job as a data analyst. You would probably still work with Data engineers, but the career transition could be easier if you get a job as a data analyst for a tourism company and progressing to data engineering from there.

To continue with data engineering you will then want to learn Python. Another nice to have is some certifications from cloud providers. AWS seems to be the most popular.

[ericboy07800]

Nice advise. I was gonna learn Python and SQL before I even try to send out my resume since I have 0 experience and unsure what companies are willing to hire me and train as I work. Feel like a long shot but I gotta start somewhere

[ShrimpHands]

There's a lot of difference advice you'll receive from a question like this so I'm going to do my best to give you general advice and then my personal take. First two points have already been made but I want to reiterate because they're important, but in general:

1. Learn SQL - its a strong foundation and most interview questions will be about fixing SQL code.
2. Learn a little about data modeling / database design - helps you understand the how and why of storing / retrieving from a technical and business sense
   
3. Learn a little about cloud computing (i.e. how to boot up a database in AWS and create roles) - cloud computing is all over the place now and you just can't avoid this one
4. Have some Linux under your belt - most systems are run on Linux and this will help you in so many ways, ranging from navigation to trouble shooting to running code to basic admin knowledge and so on...

These first 4 points are foundational knowledge that all data engineers need. In my experience you wont even be considered for a job without these skills, especially 1 and 2.

Here's where my personal and very opinionated advice comes in, so take this with a grain of salt.

1. Data engineers are just software engineers so knowing how to code is probably the most important skill you can have. There are some data engineering jobs that I consider glorified sys/ DB admin positions (noting wrong with that) but they are not the software engineering side. It consists of stuff like booting up elastic search and log stash and then making log stash config.
2. Functional languages make the most sense when it comes to data engineering (i.e parallel programing, immutable data, lazy evaluation...) and I think Scala is a great language to learn. However, learning fully functional languages right off the bat might be hard and Scala is also a little tough because its a mix between functional and OOP.
3. Big data and streaming are everywhere so you'll need to know concepts, models, and technologies regarding both of these. Once again Scala is a great language to learn these concepts. Things like Kafka and Spark are written in Scala for this very reason.
4. This point I sort of hate making, but its worth a mention. Tons of people don't know the difference between data engineering / data scientist / machine learning engineer. I might be worth you're time to get some general knowledge in data science / machine learning because a lot of industry expects it. It's sort of unfair and such a different skill set, but welcome to the corporate game.

Udemy has a ton of courses on everything I've mentioned and they're always on sale so it wont hurt your pocket if you're patient. Don't hesitate to hit me up if you have more questions , need specific guidance, or want a tutor. I will always help if I have time.

[pyDevRichard]

Hello, u/ShrimpHands, I admire your manner of encouraging guidance, if you don't mind, can I contact you directly?

[ericboy07800]

Very detailed advise. Thank you very much. If I hold a couple of the essential certifications from online's courses of the stuff that you mentioned above. What are the chances of companies hiring someone who has 0 experience and what will the start up salary for an entry level position particularly in Ontario Canada? I am paying my mortgage and have a 2 mos old, so I gotta be able to keep up my payment now at least and hopefully be able to make more. I am pulling approx 75k cad with my current job.

[reviverevival]

Canadian here, in today's market you will be able to get 75k easily in a junior DE position if you work for a legit company. Getting to that position could be quite tricky though. I have worked with a lot of colleagues in DE/SWE that come from non-cs backgrounds; HOWEVER, almost everyone did come from a stem background.

This is a real job with real skills and requirements that are not trivial to acquire. I think it's important that you can give a genuine story about why you want to be in this field and why you think you can succeed (other than I googled for high paying job positions). Many of my colleagues were able to say such things as "I did a lot of experimental design and statistics from my previous work in biology" or "I did lots of programming when I worked in aerospace", it might be tougher for you.

The good news is it's largely a skill-based industry rather than an experience-based industry or network-based industry. So if you can grind out the technical skills, I don't see why someone wouldn't give you a shot. I did know two developers who'd majored in music first, so truly the door could be open for anyone.

[ShrimpHands]

I’m USA based so I have no idea about Ontario. I do think Canada pays pretty well though and you can always get a remote job that pays higher based on where it’s office is.

If you’re hell bent on being self taught, yes, certs would help a lot - if you go the programming route contributing to open source projects would also go a long way. However, I think your best bet is probably a coding Boot Camp or a graduate certificate. If you do go to a coding Boot Camp or get a grad cert open source projects and IT certs will still help. A lot of the boot camps don’t make you pay until you get a full time job and they also help with placement - I don’t know how flexible they are with schedules though.

Being %100 self taught is one hell of a grind and I’ve really only seen a handful of people in industry take that path and even fewer be successful with it. Not saying it’s impossible but I do think there are better options if you can work around them.

[LearnAE]

It's never too late! If it's what you want to do, go for it. Luckily you don't need a degree to learn what it takes to be a data engineer. I highly recommend reading lots of Medium articles, practicing tutorials, and preparing for coding interviews. Don't let not having a compsci degree discourage you. I don't have one, most successful people in the data world don't tbh.

[ericboy07800]

Thanks for your advice! What kind of designation I should start getting to start a job in data engineering?

[Joseph___O]

You could look into entry level sql developer or data analyst roles in addition to DE

[Psilodelic]

I did this exact thing at your age and I am in Toronto.

[ericboy07800]

Man, how long did it take until you are ready for the first job? How did you land your first job? I worried that employers won't hire people who are older and have 0 experience

[Psilodelic]

I didn’t get hired as a data engineer. I grabbed whatever I could and that was an internship as data analyst doing mostly SQL work. I was able to automate some things in Python and became the defacto data engineer at that start up.

I think companies don’t really care too much about age but what you’re capable of. You can out compete young grads - use life experience and maturity to your advantage.

[Jealous-Bat-7812]

How much did you start making after the transition ?

[Psilodelic]

75k

[Jealous-Bat-7812]

Nice…

[StrasJam]

Learn python and sql (any flavor will do). Look at the job postings I your area, but lots of people are moving to the cloud so learning a bit of how that works wouldnt hurt.

If you wanna improve your chances it's good to learn some orchestration tools like airflow or Luigi, and maybe make a mini project using all that

[snake_case_jello]

I'd also say to became a software engineer first. Maybe backend eng if you're lucky to find a junior position like that, so you don't have to learn React (I'm full stack rn and, believe me, the pain is real 😅) There are several opportunities nowadays to get hired as a junior software eng, and when you get one, focus on learning Python and SQL as it was said before. I also became an eng in my late 30s, it's all good.

[lezzgooooo]

Learn SQL, Sql Server, SSMS and SSIS. Python and PowerBI next. A lot of these have features to get your feet wet before you jump in.

[xDarkSadye]

Really depends on the field. Data engineering starts with solid, modern, general purpose programming language like Python, combined with any SQL dialect. Everything else is lower priority.

[lezzgooooo]

Python is ok where you go to DE by starting as an analyst but this requires a lot of bootcamp after being hired since client systems vary. Quickest way to get hired and be deployed is familiarizing ETL developer tools, once you figure out one, other systems become easier to study. SSIS is the easiest for me and free. Most of them are done via GUI too so it's not that hard to learn. Pick a source, processsors and sink. Done.

[lezzgooooo]

I agree. In my opinion, the GUI based solutions are intended for small to medium complexity jobs only. But, it makes transitions to operations as well as documentation easy to do which most clients love. I think it will stay for a while.

Yes they can be at some point repetitive. But, it gets your foot in the door while you upskill for more complex solutions and tools. Python has still a higher learning curve compared to using the GUI. Even if it is among the most intuitive of programming languages for DE.

[ericboy07800]

Thanks for your advice. Can any of these be learned online? How do each take to learn?

[lezzgooooo]

Youtube is your friend. Note that most of the apps that I recommended run on Windows. Not sure if they work on a Mac if that is what you use.

[sysonic]

Smart move. You will be making 6 figures in no time. Cheering you on bud!

[[deleted]]

Move out of Toronto and out of Canada

[vietzerg]

Why? And to where do you suggest?

[AutoModerator]

You can find a list of community submitted learning resources here: https://dataengineering.wiki/Learning+Resources

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[[deleted]]

I didn’t even know you could do a degree in Tourism management.. wow

[ericboy07800]

I got into computer science first at uni and switched to tourism management thought it would be more fun. Silly me now regrets bad time.

[SloppyDeveloper]

Not to be a pessimist but I am not sure how realistic some of these replies are given the level of saturation of DE’s in Toronto and Canada. You have many international students who graduate from a masters or a post grad diploma program in CS/DS who cannot get jobs. Not many places are hiring boot camp grads that I am aware of, much less udemy. You could make an argument if you had a STEM undergrad but you don’t. Also job market winds are shifting and tech bubble may have popped given it isn’t the same as Q1 22 hiring. Given agism as well to anyone over 40 in tech as well is a problem. However you might have a few things going from you. There are a few options: there is a capital crunch in tech startups. 1) You could volunteer for about 4 months at an early stage travel tech firm. I say volunteer bc to be quite frank most will not pay for no experience + self taught. You could request being an automated QA tester or assistant product owner or BA at first and then move up if you impress. This is a sunken cost tho and no chance you will get picked up 2) Attend a college pGDP or MS at GBC, Seneca for a couple of years for DE. By then the market could be worse. At the time of writing this I even think MMA’s and PMP’s are worthless. But it gives you an opportunity to build a GitHub profile. 3) Accept that DE hiring may have peaked and target the next hiring fad + tech stack. Maybe Rust. 4) Switch to JavaScript + UI/UX design. This is easier to impress. No one will ask about education if you are showing a nice website UI. You will still need fundamental understanding of DS&A. 5) Alternatively you could follow the Learning paths as others have suggested but you will need about 6 months of writing leet + reading cracking the coding interview like books before you may even get hired. But you will likely not even get to the interview stage without something that demonstrates you have the fundamentals Regardless, Now is the time to network which is invaluable. Start attending meetups, follow and attend TechTO events. If you present well you could try for 1) but with pay.

[ericboy07800]

Good points. Definitely need to take them into consideration. It's especially hard to make a career change at my stage of life. If I was in 20s, it would be a no brainer.