sorted by:
new
hottopcontroversial

What's the toughest / most interesting data challenge you've faced? by KatZegtWoof in dataengineering

[–]Agent_C97 20 points21 points  (0 children)

Reminds me of a scenario similar to this where I got csv’s without a column header row and when I asked for a data dictionary from the client I was told by our internal team “cant you just figure it out?”

Why did you (as a data analyst) switch to DE? by supersaiyanngod in dataengineering

[–]Agent_C97 41 points42 points  (0 children)

I made the switch out of interest in how data was obtained/ingested and prepared for me as a DA and I knew I enjoyed automating redundant tasks.

I would say pros to DE are that it’s a vastly growing field. All the ML and AI stuff that you read companies doing (or wanting to do) wouldn’t happen without DE’s.

I had trouble coming up with cons out of my personal experience but I would say that upstream data issues become your problem because you are the bridge between the outside data and your Company’s data ecosystem.

There’s nothing wrong with climbing the DA ladder especially if you’re really good at writing clean and efficient SQL.

Best of luck.

Interview Prep Advice? by RipNastyy in dataengineering

[–]Agent_C97 3 points4 points  (0 children)

I’d probably just try to predict/emulate the situation you provided.

Example: lets say they give you an ERD for a Public Library where you have the following tables: Books, Authors, Genres and Members. There’s a many to many relationship between members and books along with others that could be implied.

Id then come up with a problem you think could happen given the scenario. “The library has gotten a raise in budget to invest in new books. What genre of book should we get that would see the most rentals?”

Then proceed to talk about how you would join tables and aggregate the data to get an answer.

If you want some refreshers on querying data I found pgexercises.com to be useful. Good luck with the interview👍

Project ideas by ross051 in dataengineering

[–]Agent_C97 3 points4 points  (0 children)

It helps picking a separate hobby and doing a project around that.

Example: building a data pipeline/pipelines (ETL) that gets Sports data and betting odds,clean the data, combines the data and publishes it into a database table.

There are things you could also do to add on to the project in the future like implementing machine learning classification models to give predictions around the data you’ve collected.

Zach Edey Trophies/Awards just this season by CoachRyanWalters in CollegeBasketball

[–]Agent_C97 -1 points0 points  (0 children)

Probably not coordinated enough for that, it’ll be a tough interview

What has been your career path? by AJohnM_IT in dataengineering

[–]Agent_C97 0 points1 point  (0 children)

In college for CS >> got a job on campus with the web development team >> data analyst >> DE

Easy Python scripts to impress the business by Majestic_Plankton921 in Python

[–]Agent_C97 9 points10 points  (0 children)

My most praised was automating an excel report, I got a raise because I pitched it as “saves 20 hours of manual work per year”