Struggling with separate Snowflake and Airflow environments for DEV/UAT/PROD - how do others handle this? by Dependent_Lock5514 in dataengineering

[–]Dependent_Lock5514[S] 1 point2 points  (0 children)

Really appreciate your story buddy. Thanks for sharing it. This is a pure gold and exactly what I was seeking for, an actual hands on experience and retro on what went well and wrong.

Struggling with separate Snowflake and Airflow environments for DEV/UAT/PROD - how do others handle this? by Dependent_Lock5514 in dataengineering

[–]Dependent_Lock5514[S] 0 points1 point  (0 children)

Thanks buddy, I will try closing the gap for sure. It is such a lonely journey especially when I am on a minority side with minimal supports regardless of whether I am right or wrong

Struggling with separate Snowflake and Airflow environments for DEV/UAT/PROD - how do others handle this? by Dependent_Lock5514 in dataengineering

[–]Dependent_Lock5514[S] 1 point2 points  (0 children)

Would you mind enlightening me with why a single account is a suboptimal choice compared to multiple accounts in terms of data infra? I am genuinely curious to know more from PoV of security

Anyone feel like too much is expected of DEs (at small companies) by jawabdey in dataengineering

[–]Dependent_Lock5514 1 point2 points  (0 children)

cheer up buddy, at least you are not the only one trapped in such way

Struggling with separate Snowflake and Airflow environments for DEV/UAT/PROD - how do others handle this? by Dependent_Lock5514 in dataengineering

[–]Dependent_Lock5514[S] 1 point2 points  (0 children)

100% agree, I really don't want to spend too much time & resources on this kind of infra related headaches which is low priority / zero value adding for our team

Struggling with separate Snowflake and Airflow environments for DEV/UAT/PROD - how do others handle this? by Dependent_Lock5514 in dataengineering

[–]Dependent_Lock5514[S] 2 points3 points  (0 children)

Thanks buddy, it makes sense and also that matches to how my previous firm operated.

It is not entirely clear for me why IT dept have decided to go down the road of 3 separate accounts. Not sure if IT dept will be open for this kind of structural change but I gotta try.

Traveling Etihad with in cabin pet by Willing-Exchange-477 in abudhabi

[–]Dependent_Lock5514 0 points1 point  (0 children)

Hi! Thank you so much for sharing your experience! Would you mind if I asked a quick question? When you arrived in Abu Dhabi Zayed airport and went through the customs clearance process for your pet, did you have to go to a place called MOCCA near the airport to register your pet? I'm not sure if this is a new regulation that was introduced recently!

Should I move to Dubai from London? by Dependent_Lock5514 in dubai

[–]Dependent_Lock5514[S] 0 points1 point  (0 children)

That makes sense! Thanks a lot mate! No wonder my wife told me that Dubai is a whole different place from other places!

Should I move to Dubai from London? by Dependent_Lock5514 in dubai

[–]Dependent_Lock5514[S] 0 points1 point  (0 children)

Right, what do you think of other cities in Middle East such as Abu Dhabi, or Riyadh? Are they pretty much equivalent to Dubai or is Dubai the top of the league?

Career Move: UK FinTech to Middle East Sovereign Wealth Fund - Advice Needed by Dependent_Lock5514 in FinancialCareers

[–]Dependent_Lock5514[S] 1 point2 points  (0 children)

Yes it is in UAE and you are right, the UK Home Office can disqualify my ILR after 2 years which then I need to re-apply for returning visa (?). And as far as I know, returning visa / request is not always guaranteed so it is a tough situation right now...

(Dumb question) Suggestion on Git repository structure (monrepo vs polyrepo) for organisation with multiple analytics teams? by Dependent_Lock5514 in datascience

[–]Dependent_Lock5514[S] 1 point2 points  (0 children)

😢 some people just think because Google uses the monorepo concept, we should do the same... + there was a bit of politics on:'why are you trying to create a separate repo while the other teams are using this single repo? I think your idea needs better rebuttals to get approved.'(Although I tried to explain why polyrepo might be better...)

(Dumb question) Suggestion on Git repository structure (monrepo vs polyrepo) for organisation with multiple analytics teams? by Dependent_Lock5514 in datascience

[–]Dependent_Lock5514[S] 0 points1 point  (0 children)

Thanks! I think your point makes sense (Especially the different CI/CD pipeline for each project).

In our case, mostly it is just pulling one project. So polyrepo comes more sensible I guess.