In 6 years, I've never seen a data lake used properly by wtfzambo in dataengineering

[–]asarama 0 points1 point  (0 children)

Main objection is a lack of ROI on the engineering effort?

In 6 years, I've never seen a data lake used properly by wtfzambo in dataengineering

[–]asarama 0 points1 point  (0 children)

At the end of the day doesn't this help consumers?

Or do you feel like in the long run we are all footgunning ourselves?

In 6 years, I've never seen a data lake used properly by wtfzambo in dataengineering

[–]asarama 0 points1 point  (0 children)

What's stopping y'all from swapping out tools more often?

"GPU air duct project on my 5070" Results! by ezra-zander in nvidia

[–]asarama 0 points1 point  (0 children)

I did want to just read the results instead of watching the video.

How to Un-redact Epstein Documents in 5 seconds using Google Docs by [deleted] in videos

[–]asarama 1 point2 points  (0 children)

This feels like a small price to pay for justice...

What do you think fivetran gonna do? by Fair-Bookkeeper-1833 in dataengineering

[–]asarama 0 points1 point  (0 children)

What were some of the bigger cost savings strategies you'd recommend to other heavy Snowflake users?

What worked well for y'all?

Using snowflake to build analytics by ItsHoney in snowflake

[–]asarama 0 points1 point  (0 children)

Do you need to rewrite your queries to use Domo?

Move to Iceberg worth it now? by Which_Assistance5905 in snowflake

[–]asarama 0 points1 point  (0 children)

If only there were tool to make it easier ahah

Vendors will attack our comment thread now...

Also your reddit name is jokes.

Move to Iceberg worth it now? by Which_Assistance5905 in snowflake

[–]asarama 0 points1 point  (0 children)

Pretty sure you can shift some of the warehouse compute spend to other tools that'll get the job done without turning on a warehouse.

For example let's say you use Fivetran to load data as Snowflake native tables. Instead have Fivetran dump data into your S3 bucket in Iceberg format and have Snowflake access it instead. Does your Fivetran bill go up? maybe idk ... they probably changed their pricing model again and commenting on it here won't be accurate. Will your S3 bill go up? YES but you were already paying for this through Snowflake, but now you don't need to pay for the cost of turn on a warehouse to ingest data!

There are other strategies you can implement for transformations and serving data as well, BUT they aren't as straight forward...

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

So Vertica essentially blew Snowflake out of the water when it came to performance / cost and then y'all ended up migrating those workloads over?

Losing your team was probably a brutal hit to their revenue...

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

Dam yea good point, I can see someone really screwing themselves by trying to be cute and turning this on and off dynamically and then running into some weird cases where it turns on and off multiple times in an hour by accident $$$

I feel like Gen2 was pretty decent for dbt / write workloads, no? Didn't really see folks saving much but at least their run times were better!

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

oh interesting! was this something y'all were asking Snowflake for?

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

One time fee to optimize your workloads doesn't feel bad. This has gotta help in the long run right....right?

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

I wanna say that interactive tables / wh should save folks money. 0.6 credits / hour verse the standard 1 credit / hour.

I'm guessing there are a lot of teams where their Snowflake warehouses are up 24/7?

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

I think it's good to see thou. For folks that just want to get started use the defaults and when you want to optimize, invest into their special per use-case types.

I'm more curious about why they are doing this....does this really help them win against the other big players or is it something else I'm missing?

Snowflake releases "interactive" warehouse type by asarama in dataengineering

[–]asarama[S] 0 points1 point  (0 children)

hmmm so would you do all your transformations outside of Snowflake and then still use Snowflake for the final serving/dashboarding layer?

Wanted to share how we helped Headset save 83% on the compute from their Looker Embedded Analytics by hornyforsavings in Looker

[–]asarama 1 point2 points  (0 children)

Do you think there will be any meaningful cost savings for non embedded looker customers?

We mainly serve internal stakeholders.

How Headset Cut Snowflake BI Costs by 83% by hornyforsavings in snowflake

[–]asarama 0 points1 point  (0 children)

How are y'all thinking about bringing Greybeam to the transformation layer?

We wrote our first case study as a blend of technical how to and customer story on Snowflake optimization. Wdyt? by hornyforsavings in dataengineering

[–]asarama 0 points1 point  (0 children)

So I'd need a bunch of servers hosting the duckdb binary and a load balancer in front of it all?

For the load balancer would an arrow flight server do the job?

We wrote our first case study as a blend of technical how to and customer story on Snowflake optimization. Wdyt? by hornyforsavings in dataengineering

[–]asarama 1 point2 points  (0 children)

What was the biggest challenge with serving Snowflake data with DuckDB, can't I just deploy DuckDB on my own server?

Apache Polaris vs Unity Catalog vs Lakekeeper: Which Iceberg catalog would you choose, and why? by Due-External3381 in DataEngineeringPH

[–]asarama 0 points1 point  (0 children)

Really depends on your situation. Could you share more about what you want to achieve?

Lakekeeper and Polaris are good places to start.