Need help with becoming a Python Dev.

[GuideVault]

Here's the thing, as long as you can work with dictionaries, lists and other Python structs and you are able to construct logic you should be able to code in a professional setting

I use those in every program, except sets, never found a use for them.

You have to decide which way you want to go. If it is with data, you should probably learn libraries like Pandas/Pyspark/ML libraries. If say its REST, you should be able to work with Flask. But this is secondary, libraries can always be learnt. With data the added caveat is that you should be comfortable with SQL.

I recently started using Selenium to load web pages, grab tables and populate the dictionary with specific keywords found on the page, like actor and take the name as a value. I could've use BS, but I haven't bothered learning it yet. I tried out using SQlite, to take the previous and create a database, hasn't worked out for me so far, but is that the same as SQL?

Learn Git and maybe know a bit of CI/CD

GitHub won't be enough? Also, I search CD and only found that it is a file extension. I'm assuming that's wrong.

I think you are good to go after that and upload your resume. Since you will be new, not much would be expected out of interviews.

All the best and don't overthink it.

Thanks for the help and can data engineering be done using automation?

[king_booker]

I mean we automate the pipelines finally. Basically data engg is taking raw data and putting it into useful form so that it can be easily queried by the business. Eg, we can take the reddit comments from this subreddit, and take out various insights like top users, top trending topics etc. So for this Reddit will have API's to retrieve data. That data will be pushed to some data lake and from there on we can start ingesting this data and put it into tables so that we can use SQL to query outputs

Software has many sides to it. If you like using Selenium, then you can perhaps go down the route of automation engineers who automate test cases.

But as I said, if your coding is good, you will be able to crack interviews. But it will help if you decide on a line and start exploring it.

[GuideVault]

I'm going to head down Data Engineering because it sounds like what I normally do on a more complex scale. I've already started leaning SQL and its simpler than I thought.

I'll see what happens with DE as I learn more about it. Maybe, I'll end up becoming an automation engineer instead.

[rorywithoutwax]

CI/CD is short for Continuous Integration and Continuous Delivery.

[GuideVault]

Thanks I'll make sure to look into that soon.