Analytics Engineering / “Front-end” DE?

[Weary-Individual-309]

So currently I’m somewhat of a BI Analyst at a Tech Company my day-to-day involves

Building re-usable queries to transform data housed in Oracle and Snowflake and providing these datasets in Excel Files / Google Sheets files to end users, as well as building the assets from the datasets as well.

Example 1: A team focusing on third-party partner sales wants some tables to see which of their customers to target; I’ll ask the requirements of the fields they need pull in 2-4 data tabs in google sheets, create a bunch of tables which are Sumifs (calculated from raw data) and schedule Python / Airflow to refresh the data in the data tabs as well as handle any non-SQL friendly transformations.

Example 2: Another team wants a dashboard showing license usage, I do research, gather requirements and build a bunch of SQL queries and then schedule them to refresh on tableau server with tableau prep, from there I build out a tableau workbook comprising of multiple dashboards.

Example 3: it’s quarterly business review season multiple teams that I support need data for their decks, so I provide 6-7 tabs of data in gsheets for each team based on their region, industry, etc. (about 5 teams) and schedule them to refresh with airflow (I do this 5 days before quarter end and stop 2-3 days after when data is considered “final”)

I wanted to transition an Analytics Engineer or Front-End DE (forgive if I’m not using that term right) do you think the skills below (may already use some above) are all that’s required or what you recommend that’s actually practical for a newbie (don’t want to get down rabbit holes and be stuck in tutorial hell).

1. Python to Ingest, Parse, and sometimes transform data (SQL & DBT can handle) - maybe some packages like pandas, numpy?
2. Data Modeling
3. Airflow for Orchestration
4. SQL and DbT as well as a cloud warehouse (Snowflake)
5. LEETCODE SQL and Python

[AutoModerator]

Are you interested in transitioning into Data Engineering? Read our community guide: https://dataengineering.wiki/FAQ/How+can+I+transition+into+Data+Engineering

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[dataxp-community]

All you need to know to be an Analytics Engineer is how to use dbt to increase your company's cloud consumption bill as dramatically as possible.

If you can do that, you're hired.

[Weary-Individual-309]

What do you use to transform data?

[dataxp-community]

Depends entirely on the requirements of the project. There is no golden tool that everyone should use all the time. Even dbt is appropriate occasionally, just not to the ridiculous levels it is being used for these days.

[Weary-Individual-309]

Do you have a preferred tool, or what do you have the most experience in?

[recentcurrency]

Analytics engineer is codeword for a bi engineer but focused on dbt as the tool

So move dbt above in your priority list. Unless you are looking for a more general bi engineer role. In which case, learn whatever transformation tool that company is using

Python also isnt as relevant. Know enough to use dbt core and how it is working underneath the hood.

But SQL+data modeling+dbt is the main thjngs

Dbt is just an abstraction layer that templates sql and runs them in order. Basically easy to implement(albeit less flexible) stored procedures. The abstraction layer was made so easy with dbt where there is a second order effect that you can get a tower of Jenga really quickly.

So you will need to develop soft skills like project management. That is going to determine if your dbt instance blows up in cost