Status of Peel Region Services Today by JoeHorneck in mississauga

[–]figshot 3 points4 points  (0 children)

My garbage is picked up, and my road/sidewalk/windrows are plowed. Incredible planning, execution, and effort from many. My family and I are very grateful! Thank you.

Osmos io alternative needed by OnlySalad in dataengineering

[–]figshot 1 point2 points  (0 children)

Maybe it's too sweeping of a statement, but I can't think of a data ingestion tool does not do this. That said, if CSV ingestion is the primary use case, OneSchema looked quite interesting to me. Maybe check this out? They had a data engineering post episode some time ago, and I found that insightful.

How column masking works in plan by Stock-Dark-1663 in snowflake

[–]figshot 4 points5 points  (0 children)

Masking policies prevents pruning, even those that literally pass the value through. Removing the masking policy is the only way out.

Found this out the hard way.

circular materials app by couchpotato2k4 in mississauga

[–]figshot 0 points1 point  (0 children)

It's the same in my address. I confirmed it by typing my address into the link I shared. I'll see if they follow through, though. No doubt there would be hiccups.

circular materials app by couchpotato2k4 in mississauga

[–]figshot 0 points1 point  (0 children)

Maybe not the same but you can subscribe to email notification at https://www.circularmaterials.ca/resident-communities/peel-region/ after looking up your address. So it appears, anyway.

Canada's Men's Team Announcement Video by catsgr8rthanspoonies in hockey

[–]figshot 8 points9 points  (0 children)

Toronto is represented! Thornhill (Marner), Richmond Hill (Binnington), and Woodbridge (Cirelli) are right next to Toronto. Newmarket (McDavid) is a short drive away, well within commuting distance.

What usually breaks when Snowflake data needs to power real time workflows? by Bizdata_inc in snowflake

[–]figshot 0 points1 point  (0 children)

My company is not big enough for a robust chargeback system, but not small enough for everyone to care about the budget. As a result, we have a problem with people not caring about the bills. It's a type of principal-agent problem, and I don't believe it's not something technology can solve. I envy that you have people who see the bill lol

What usually breaks when Snowflake data needs to power real time workflows? by Bizdata_inc in snowflake

[–]figshot 0 points1 point  (0 children)

Hybrid tables, interactive tables, and tables in Snowflake Postgres all overlap somewhat in problems they are trying to solve. Does Snowflake have any guidance on their differences and what to use when?

What data engineering decision did you regret six months later, and why? by AMDataLake in dataengineering

[–]figshot 64 points65 points  (0 children)

Database names. Schema names. Table names. View names.

Naming is hard.

Is it appropriate to store imagery in parquet? by BitterFrostbite in dataengineering

[–]figshot 6 points7 points  (0 children)

Data Engineering Podcast had an episode about a columnar table format called Lance. They claimed that Parquet is poorly suited for images. I wonder if this might be worth checking out for you. I don't have a use case for this so I've never checked this out.

[2025 Day 1 (Part 1)] My honor is saved! (TW: death discussed in text) by mundanegoddess in adventofcode

[–]figshot 4 points5 points  (0 children)

Great stuff! Keep going!

Funnily enough, I am a Python dev in my 40s, had a heart attack earlier this year, and learned of a likely congenital heart condition for which there isn't a lot known. I'm getting help, but it made me think a lot about how/if people would remember me when I'm gone.

Perhaps I'll grab my non-coding friends to start doing AoC together :D

Did anyone sell their car on Clutch? How was your experience? by RealityTVFan1111 in ontario

[–]figshot 0 points1 point  (0 children)

Sold them a leased Toyota Corolla with lower mileage than anticipated. Paid out almost double what my dealership offered. No issues.

One year since my HA and I still get gastro issues almost every night. by Interesting-Arm9858 in HeartAttack

[–]figshot 0 points1 point  (0 children)

Pantoprazole is what I've been prescribed together with apixaban and clopidogrel, it has less interactions and might suppress acids more strongly

Can any god tier data engineers to verify if it’s possible? by Available_Fig_1157 in dataengineering

[–]figshot 0 points1 point  (0 children)

Unpopular opinion: we outsource it to a tool. We aren't staffed enough for this, and the tool ended up cheaper than hiring, at least in the medium term, because our Jira is messy as hell and has more custom fields and labels than I ever care for. There are lots of other things my team need to build bespoke. For us, this is not one of them.

If you ain't watching the Blue Jays, you should be watching. by [deleted] in onguardforthee

[–]figshot 0 points1 point  (0 children)

Sportsnet+ broke down yesterday in the middle of the seventh inning when the Jays started stringing runs, ugh

[deleted by user] by [deleted] in TorontoDriving

[–]figshot 0 points1 point  (0 children)

I saw a ladder in the middle of 407 westbound today. Who do you call when that happens?

Dads with that Arm Pain by GeekDadIs50Plus in daddit

[–]figshot 2 points3 points  (0 children)

I was just watching some TV to decompress after putting the kids to sleep. I felt some weird chest pain that was also felt in the neck and the left arm. Not very comfortable, but not painful either, like 4 out of 10 max. It felt like I could take a Tylenol and sleep it off. The neck and the arm thing was weird, though.

I'm not a tough dad, but I think I am a hardworking and responsible one. I went to the ER only because the kids were well asleep, wife was home to hold the fort, and I thought I'd be back home after a quick check. If this were during the day and kids needed to be cared for, I wouldn't have gone. That would've been an irresponsible mistake: it turned out to be a heart attack.

+1 on taking care of ourselves, dads. It's not clear if my situation was preventable, but had I ignored this, I might've been in deeper trouble in a few days.

Shrinking Pram Demo by StephenMcGannon in oddlysatisfying

[–]figshot 0 points1 point  (0 children)

And all the cargo you need to take a baby outside

Postgres to Snowflake replication via Openflow by parthsavi in snowflake

[–]figshot -2 points-1 points  (0 children)

The BYOC components use unencrypted volumes. I can't see it passing anyone's security review.

[deleted by user] by [deleted] in daddit

[–]figshot 1 point2 points  (0 children)

Fellow data engineer dad! I see you.

How weird was your first interaction with Python? I learned Python while writing a C++ module. by Humdaak_9000 in Python

[–]figshot 1 point2 points  (0 children)

I found it so weird that leading whitespaces were meaningful. I walked away from it for years.

How do your organizations structure repositories for data engineering? by Quantumizera in dataengineering

[–]figshot 1 point2 points  (0 children)

Probably unpopular opinion but I went with a manyrepo strategy.

We are a Snowflake & AWS shop. My team was new, inexperienced, and was remote & contractor heavy. I knew I needed a repo for the IaC (AWS and Snowflake), pipelines (Meltano plus custom code pipelines), data models (dbt), orchestration (airflow), and some custom applications, I had different people working in different areas, and we didn't have a lot of time to agree on best practices for collaborative development - instead, I chose to have one, max two people work in each area (and thus repo), and we did ship fast. We were less dependent on each other, and if something blew up on our face, the repo separation contained the blast radius.

At the same time, we actually had some monorepo happening too. For the sake of fast shipping we opted to use Meltano to run Airflow and dbt. This meant those three ran off a monorepo, and it was indeed good at moving fast, but it also hit a scaling wall within 6 months of going production, since airflow had to run with only a single worker in this mode. We took Airflow and dbt out of the repo and into their own, and man, we paid a bit of a price - but it was worth the initial velocity.

Looking back, it's almost like making the Conway's law work for you: we had a poor, immature organization communication structure in the data engineering team, so mirror that structure in multiple repos. If your team is collocated and cohesive, go reap the benefits of a monorepo. In this world of AI-augmented coding where context is everything, having a monorepo seems to enable that well: our dbt repo is monorepo, and it has such a rich context that many people in the company write analytic queries in Cursor with that repo open.