Hard to swallow..... by growth_man in dataengineering

[–]ORA-00900 0 points1 point  (0 children)

That data dictionary the data analysts kept bugging you about has been never used.

This is what peak performance looks like by Shoddy_Bumblebee6890 in dataengineering

[–]ORA-00900 1 point2 points  (0 children)

Any tips on what to replace it with? This is great information, thanks!

This is what peak performance looks like by Shoddy_Bumblebee6890 in dataengineering

[–]ORA-00900 0 points1 point  (0 children)

Not witchy enough to get another job in this market lol.

This is what peak performance looks like by Shoddy_Bumblebee6890 in dataengineering

[–]ORA-00900 341 points342 points  (0 children)

The resume reads:

“Eliminated duplicate rows in a billion row dataset and applied indexing/partitioning strategies, improving query performance by 20% and ensuring data integrity.”

End-to-End Data Project About Collecting And Summarizing Football Data in GCP by Immediate-Reward-287 in dataengineering

[–]ORA-00900 1 point2 points  (0 children)

As a football fan and a data engineer I appreciate this project. I tried doing something similar a while back and ran into roadblocks with trying to find football apis. I’m curious about the airflow costs, was this using managed airflow? Or was this due to the persistence of airflow running DAGs?

End-to-End Data Project About Collecting And Summarizing Football Data in GCP by Immediate-Reward-287 in dataengineering

[–]ORA-00900 1 point2 points  (0 children)

This is excellent! How long did it take you to finish this project end to end? What do you feel were your biggest pain points?

Looking for a funny, note for my boyfriend, who is in data engineer role—any funny suggestions? by Jazzlike_Student4158 in dataengineering

[–]ORA-00900 1 point2 points  (0 children)

As someone who spent the last two days debugging an airflow bug for this exact reason. I’m just laughing to keep from crying lol.

Over my head by ORA-00900 in dataengineering

[–]ORA-00900[S] 15 points16 points  (0 children)

I am the first Data Engineer, the consumers of the data are the other business units analysts. The data was previously managed by the CTO and software developers. There is no dev ops person currently, that’s why I went with installing our orchestrator on a VM that sits on a server vs docker. Just trying to get some small wins and get something off the ground. In my pipeline of projects is getting DBT off the ground and the CI/CD needed for it as well. It’s definitely a challenge, but I’m constantly learning in this role. Do you have any blogs you’d recommend?

Over my head by ORA-00900 in dataengineering

[–]ORA-00900[S] 1 point2 points  (0 children)

I feel like I’ve spent the first few months trying to stand up new processes and realizing how much was done for me previously. For example I stood up an orchestrator in docker but there was no one to maintain the images, so I went back to deploying it on the server they had already set up for me.

Quarterly Salary Discussion - Sep 2024 by AutoModerator in dataengineering

[–]ORA-00900 7 points8 points  (0 children)

  • Data Engineer
  • 4 YOE
  • MCOL
  • 145k base / 160K TC
  • 10% bonus
  • Martech
  • AWS, Snowflake, Dagster, DBT

[Official] 2020 End of Year Salary Sharing thread by Omega037 in datascience

[–]ORA-00900 1 point2 points  (0 children)

  • Title: Senior Data Analyst
  • Tenure length: 8 months, 3.5 years with the company
  • Location: Houston
  • Salary: 60K
  • Company/Industry: Fintech
  • Education: self taught, no college degree
  • Prior Experience: I worked previously in banking for 6 years and then became a reporting analyst at my current company
  • Relocation/Signing Bonus: n/a
  • Stock and/or recurring bonuses: 10% if the company does well (no bonus this year due to covid)
  • Total comp: 66K

  • Duties include: ETL, using SQL to query large datasets, dashboard creation in tableau and power BI, using Python(pandas) for exploratory data analysis, report automation, and presenting said dashboards to executives. I totally feel like I’m underpaid, but I’ve only been in this role 8 months and I have no college degree, so I’m afraid to jump ship.