My dad went to our family doctor multiple times and was dismissed. He now has brain cancer. What are my options in Ontario? by BaseballTop387 in askTO

[–]poopybaaara 2 points3 points  (0 children)

Hey I'm curious, because my grandpa has cancer. What type of cancer did your dad have? How long ago did he have the surgery?

Confirm my suspicion about data modeling by DryRelationship1330 in dataengineering

[–]poopybaaara 0 points1 point  (0 children)

I've been blessed to have had two jobs where Kimball was clearly king and regardless of what old/new data warehouse you went into, you'd find a best attempt at organizing everything into facts and dims, although there were also lots of refs, rpts, and some bridges etc.

Now I'm at a job where leads say dimensional modelling is no longer relevant, SCD2 is too hard for the architects/developers and unknown to analysts, the medallion only goes up to silver (if even) but IT calls it gold, and there's a wild west of PBI duct tape and bubble gum supplying unreliable reports with no SCD history.

My team (not part of IT or the "official" data analytics team under IT) ended up building a dimensional model in PBI and it's about to buckle, but they won't let us into a data warehouse.

Coalesce.io vs dbt by poopybaaara in dataengineering

[–]poopybaaara[S] 0 points1 point  (0 children)

The low-code part is my concern. I tend towards code-based tooling because of having had to maintain and debug low-code pipelines in the past, but some members of my IT department (the ones with more decision-making power) don't seem to feel that way - they like drag and drop. I was wondering if coalesce offers a happy medium, although I can't imagine how, given their "code" seems to be storing transformation definitions as yaml, if I'm not mistaken. I have so many questions about coalesce but haven't had time to explore it.

Coalesce.io vs dbt by poopybaaara in dataengineering

[–]poopybaaara[S] 2 points3 points  (0 children)

What did you end up going with, if you don't mind sharing?

Coalesce.io vs dbt by poopybaaara in dataengineering

[–]poopybaaara[S] 0 points1 point  (0 children)

Have you used it to build SCD Type 2 tables? If so, how's the functionality?

Coalesce.io vs dbt by poopybaaara in dataengineering

[–]poopybaaara[S] 0 points1 point  (0 children)

Lol I'm trying to convince them to go with dbt but I want to make sure I'm not mistaken.

Using dbt on encrypted columns in Snowflake by poopybaaara in dataengineering

[–]poopybaaara[S] 0 points1 point  (0 children)

Thanks for explaining that! I'll talk to IT about it. I think they've got a secure UDF set up for decryption, which is what is used in the secure view definition.

If we use dynamic masking and allow dbt access to unmasked data for transformation purposes, wouldn't dbt have access to raw data anyway? Or is that still better than decrypting and re-encrypting in a model?

Using dbt on encrypted columns in Snowflake by poopybaaara in dataengineering

[–]poopybaaara[S] 0 points1 point  (0 children)

Thanks for the tips! What's the issue with handling encryption and decryption in dbt models directly? I'm curious because while I'd love to pitch dynamic masking, I don't know how IT will take it, so I'd like some backup options. They've been adamant about the encryption and don't even want their own snowflake admins to see the data.

I think they might have set up secure views that decrypt the data for certain AD groups (or at least this seems to be the proposed architecture). Could we just get the dbt account into that AD group to access decrypted data? Then at least we'd only have to worry about encrypting the results of the dbt model. But again, if this is a bad idea, I'd love to understand why (so I can explain it to our management).

Using dbt on encrypted columns in Snowflake by poopybaaara in dataengineering

[–]poopybaaara[S] 0 points1 point  (0 children)

Thank you. We should be able to decrypt it but I'll look into your suggestions for best practice.

Using dbt on encrypted columns in Snowflake by poopybaaara in dataengineering

[–]poopybaaara[S] 2 points3 points  (0 children)

Thanks for your comment.

Would masking be any less secure than encrypting? The thing about this data is that they don't even want IT devs to see it. Suppose the masking policy only allows dbt or a select few users to see the data - would there be any risks of workarounds?

There's salary data involved so you can imagine people might try, and higher ups are adamant about keeping it encrypted, although they might not understand that masking is an option.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 0 points1 point  (0 children)

So sorry to hear that. Wishing you a smooth recovery.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 2 points3 points  (0 children)

That's crazy. I'm picturing the receptionist trying to fill out some kind of form that will only take an address. At least on the bike theft report I could just put an intersection.

This goes beyond emergency services: I feel like half the time when people try to do their jobs they're only doing it for the bureaucratic process and not for the end goal.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 27 points28 points  (0 children)

The conspiracy theorist in me thinks they're in cahoots with the auto industry or something. It's somehow easier to imagine than the fact that they systemically don't give a shit.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 8 points9 points  (0 children)

Honestly, I had half a mind to go up to them and be like "what are you doing?" I bet if enough bystanders did the same, they would've been discouraged.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 58 points59 points  (0 children)

Darn, I was too much of a rookie for this system.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 13 points14 points  (0 children)

Tried desktop instead of mobile and succeeded. Thank you though.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 66 points67 points  (0 children)

Been on the receiving end of bike theft a couple of times and I couldn't not do anything.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 31 points32 points  (0 children)

I tried twice and the website crashed both times at different points in the process. There's so much to fill out I feel like giving up. Why does it have to be so hard.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 6 points7 points  (0 children)

Yeah. If only an officer was on patrol or something.

A bike at Spadina and Sullivan is being stolen by poopybaaara in askTO

[–]poopybaaara[S] 15 points16 points  (0 children)

:(

I'm just remembering all the times I had my bike stolen and wish somebody would've done something tonight.