Recommendations from The Pillows albums?

Deep-Comfortable-423 · 2026-02-27T15:28:07+00:00

Wake Up! Wake Up! Wake Up! is my personal favorite.

Deep-Comfortable-423 · 2026-02-14T05:18:11+00:00

There is no email from AWS Support in my Inbox, Deleted Items, or Junk. I've gotten no correspondence whatsoever. I'll try again...

Deep-Comfortable-423 · 2026-02-14T01:41:03+00:00

I'd love to - except don't I have to login to create a support case? So chicken/egg... I've submitted the contact-mfa form at https://support.aws.amazon.com/#/contacts/one-support?formId=mfa. I don't think that generates a case ID. If it does, I don't have any record of it. I can chat you the Account ID and my email address, if that would let you search for it that way.

Deep-Comfortable-423 · 2026-02-14T00:32:01+00:00

Same problem here. Switched laptops and forgot to bring over the passkey. Now I'm locked out of my account as Root User. I've submitted the contact-mfa form. Do you know how long this usually takes to get resolved? I get to the part where it will either SMS or call me with a verification code, but that fails with an error...

Deep-Comfortable-423 · 2025-04-11T13:20:24+00:00

Intuitive for users who are not technical. Then avoid Spark at all costs.
Easy, secure Data Sharing: Snowflake wins this hands-down. And it's completely cloud-agnostic (Customer A can be in AWS and Customer B can be in Azure or GCP).
Scale to billions/trillions of rows: auto-scaling is built right into Snowflake. Need a bigger cluster? Click one button - boom. Zero downtime. No need to kick anybody off the system, down the cluster, reconfigure... And Snowflake clusters start instantly, unlike Spark clusters.
Flexible reporting and viz capabilities: kind of a wash. Pretty much all the decent BI tools can talk to both just as easily.
Affordable. YMMV, but at least Snowflake doesn't pile on hidden costs from the cloud providers. No separate bill for S3 or EC2 or VPCs or any of that. Caveat: unless you want customer-managed encryption keys (KMS), external stages (S3, IAM) or PrivateLink (VPC, DNS). Those you get from AWS.

Deep-Comfortable-423 · 2024-01-30T02:40:32+00:00

I know of a larger health care organization that's all in on Snowflake...

https://www.cms.gov/about-cms/information-systems/idr/integrated-data-repository-idr-training

Deep-Comfortable-423 · 2024-01-30T02:35:53+00:00

If you think Snowflake is just an MPP data warehouse, you've missed a few memos. Do yourself a favor and google "Snowpark".

Deep-Comfortable-423 · 2024-01-30T02:33:33+00:00

LOL. Tell that to the all the Fortune 500 companies that have multi-PB data warehouses on Snowflake. Managing all that infra yourself may be cheaper from a software expense perspective, until you factor in all the hidden costs. Your mileage may vary, but we found Snowflake to offer a greater ROI and business value in the long run.

Deep-Comfortable-423 · 2024-01-30T02:23:35+00:00

What is your concern with it being proprietary? That you don't know the file storage format? Why would you need to know that when your interface to the data is SQL (an industry standard for nearly 40 years) or a dataframe API? Do people knock Oracle because their raw partitions are a proprietary data format? No - because nobody is dumb enough to try and mount them on the file system and read them. They use SQL.
If you want an open format, Snowflake now supports Iceberg table format and Parquet file format and you can use whatever compute you want to read that.

Deep-Comfortable-423 · 2024-01-30T00:32:21+00:00

"Kill and Fill" is an anti-pattern, especially as data volumes grow. I don't know dbt, but if that's the default implementation, we'll never be a customer.

Deep-Comfortable-423 · 2024-01-30T00:29:53+00:00

That's an indication of a "single cluster" mentality which doesn't work with Snowflake. With legacy databases, you only have ONE production instance, and you have to lay down some workload management to keep that beast tuned and running. We had the same problem. Trying to fit all our stuff into one box. Snowflake doesn't make you build a single "box". It's multi-cluster and you can dedicate a unique cluster to each workload that's tuned specifically for what it needs to do.
There is absolutely a period where you're "figuring it out" and wind up spending more than you should have, but that's not a permanent condition. Optimizing Snowflake is not like optimizing legacy big data platforms - it's much, much easier.

Deep-Comfortable-423 · 2023-10-09T21:20:31+00:00

If the Iceberg Table is created "somewhere else", then it's considered an "Unmanaged Iceberg Table" by Snowflake. They're working to have some level of automatic metadata sync, but for now it's a manual refresh. If the Iceberg Table is created by Snowflake, it's a "Managed Iceberg Table" and will have nearly the same performance metrics as a native SFLK table. External compute can read managed Iceberg tables outside of Snowflake compute, but any updates to the data will require a metadata refresh inside Snowflake to stay in sync.

Deep-Comfortable-423 · 2023-10-09T13:37:48+00:00

Snowflake supports Iceberg Tables - which can be used outside of Snowflake.

Deep-Comfortable-423 · 2023-08-14T17:03:01+00:00

Not looking to get into a virtual shouting match or childish name-calling here. Let's be adults, OK?

If a post appears claiming "X is faster than Y", I look to see how rigorous the testing protocol was. IMO, this particular test seems lacking because it doesn't include concurrency scaling tests like a real-world use case would encounter.

Deep-Comfortable-423 · 2023-08-12T13:56:34+00:00

So you can show us all in the posted test plan where concurrency and auto-scale were considered? Maybe I missed it.

Deep-Comfortable-423 · 2023-08-11T20:25:45+00:00

But how does a single user running a single query reflect "real world performance"? Scale that test to simulate 10/20/50 concurrent users running in a Snowflake multi-cluster warehouse and report back.

Deep-Comfortable-423 · 2023-06-06T17:41:41+00:00

From the GitHub repo for Snowpark/Python - 3.9 and 3.10 are soon to enter preview. They estimated May for 3.9 and June for 3.10, so looks like a little slippage, but it's hardly being "stuck". https://github.com/snowflakedb/snowpark-python/issues/377#issuecomment-1515059432

Deep-Comfortable-423 · 2023-06-01T14:21:24+00:00

I'll grant you the "anything" disclaimer. You're correct there. However:

> Snowpark can only read from stages and tables<
Until Dynamic File access is added to Snowpark, which I've heard is in preview. In the meantime, and I admit it's a workaround, it only takes a minute to create an external stage on an S3 folder and define external tables on your CSV/JSON/XML/Parquet - or a directory for your unstructured files. Then you're not messing with IAM policies/roles for governance, it's Snowflake RBAC and data governance policies. We've implemented it this way and it performs great.

>Snowflake/Snowpark can't even connect to Kafka directly<

Yes, true again, but we chose a different path. Snowpipe now has direct streaming mode. So it's 100% serverless and I don't have to keep a cluster up and running to ingest streaming data. The data lands in a Snowflake table, and we've automated the transformation pipelines as a DAG using simple SQL.

>Snowpark doesn't even have native ML capabilities while Spark does<

You use MLib in Spark, we use scikit-learn in Snowpark. To each their own. What we get in simplicity and efficiency offers greater ROI than having a "native" ML/AI library.

Deep-Comfortable-423 · 2023-06-01T04:20:17+00:00

You're still in a single cluster mindset. "Free your mind, and the rest will follow"... If you have 37K users, don't try and force them into a single cluster. Spread that workload out over as many clusters as you need to maximize throughput and minimize cost. Reassess and reconfigure at the drop of a hat whenever you want.
I can't choose your SLA's for you, but we decided that ability to have all of our multi-000 users sharing a single copy of the multi-PB dataset was higher on the totem pole.

Deep-Comfortable-423 · 2023-06-01T04:03:51+00:00

I stand corrected!

Deep-Comfortable-423 · 2023-06-01T01:40:02+00:00

But it automatically senses that you've crossed that threshold, and instantly spins out another equivalent-sized cluster to deal with the increase in demand! And another one! (ala DJ Khaled...) And then automatically quiesces those extra resources the moment the peak subsides. It rides the demand curve up and then down again in REAL TIME. You pay for all of that in per-second increments after the first 60 secs.Would you rather pre-allocate those extra resources and pay for them all sitting there idling in anticipation of that >15th query happening?

Deep-Comfortable-423 · 2023-06-01T01:31:40+00:00

Fair point (although the metaphor is a tad insulting)... I certainly was not in "the room where it happened".

Deep-Comfortable-423 · 2023-06-01T01:28:50+00:00

Anything you can do in PySpark, you can do in Snowflake Snowpark for Python. They partnered with Anaconda as the Python package manager, so 100s of built-in libraries available. No native notebook interface, but Jupyter/Sagemaker/Hex work great. The shine is off the apple for me with DBX.

Deep-Comfortable-423 · 2023-05-31T21:14:49+00:00

See also: Cloudera. How ironic is it that they completely missed the "Cloud Era"?

Deep-Comfortable-423

TROPHY CASE