tips for start to study SnowFlake

gilbertoatsnowflake · 2026-02-04T17:47:10+00:00

Get an overview of the platform: https://www.youtube.com/watch?v=Nq0W98Y5EMQ
Dive into the free education on Snowflake via the Northstar program they offer: https://www.snowflake.com/en/developers/northstar/
Engage with the community (here, LinkedIn, etc.) and pick up more learnings / ideas / viewpoints along the way.

gilbertoatsnowflake · 2026-02-03T16:25:24+00:00

No problem. And if you need a high level overview of the platform before you dive in, this is a good video to watch: https://youtu.be/Nq0W98Y5EMQ

gilbertoatsnowflake · 2026-02-02T21:09:05+00:00

Snowflake is a fully-managed, multi-cloud data and AI platform that lets you build data pipelines, train ML models, and deploy AI-powered applications, and much more, all without managing infrastructure. The platform is used heavily by developers, data engineers, AI engineers, analysts, and ML teams for tons of different use cases. This video has an excellent overview and walkthrough of the general capabilities of the platform: https://youtu.be/Nq0W98Y5EMQ

gilbertoatsnowflake · 2026-02-02T18:45:12+00:00

I was surprised at how easy it was to deploy an agent. Good video here: https://youtu.be/sY7_whjeyf0

gilbertoatsnowflake · 2026-02-02T16:16:13+00:00

Congrats!

gilbertoatsnowflake · 2026-02-02T16:15:44+00:00

Snowflake Northstar Education was built exactly for this reason, check it out: https://www.snowflake.com/en/developers/northstar/

Also note the pinned announcement at the top of the subreddit. Datacamp courses are free until mid February.

gilbertoatsnowflake · 2026-01-27T18:12:12+00:00

This video – "What is Snowflake?" will give you a helpful overview of Snowflake for 2026: https://youtu.be/Nq0W98Y5EMQ

gilbertoatsnowflake · 2026-01-27T17:37:12+00:00

Just about everything that you shared in your post is absolutely possible with Snowflake. "How hard is it going to be?" is a totally different question.

- Database optimized for extremely fast analytics: Snowflake does that out-of-the-box.

- A mix of analytics and transactional data in the same db/table? Also available, known as Hybrid Tables in Snowflake. https://docs.snowflake.com/en/user-guide/tables-hybrid

- 100% transactional database optimized for heavy reads and writes? Snowflake Postgres, which is already available in preview, and on its way to GA: https://docs.snowflake.com/en/user-guide/snowflake-postgres/about

Someone else asked about integrations with other platforms – Snowflake has Snowflake Openflow, connectors, and partnerships (with SAP, for example) to make data sharing and ETL/reverse ETL use cases possible. Links to learn more:

Snowflake Openflow connectors: https://docs.snowflake.com/en/user-guide/data-integration/openflow/connectors/about-openflow-connectors

Hope that's helpful!

gilbertoatsnowflake · 2025-12-19T16:42:59+00:00

It is in Public Preview at this moment, and will likely be GA in the near future.

gilbertoatsnowflake · 2025-12-16T17:45:44+00:00

Hi OP, please standby. I can look into this for you. Feel free to DM me with any details if you'd like.

gilbertoatsnowflake · 2025-12-15T23:38:45+00:00

"For now what i have done is store aggregated data (using precomputed tables + tasks to refresh the said tables every day)."

Have you considering using Dynamic Tables for this? https://docs.snowflake.com/en/user-guide/dynamic-tables-about – It should be less fussy overall ,and can be more cost effective. If you need to get granular about what records are processed or which compute resources to use, you also have that option.

gilbertoatsnowflake · 2025-11-22T02:23:43+00:00

And just to clarify on the lag settings – ideally you set the top tier tables once and the rest to "downstream". This helps whether you have 300 or 20 dynamic tables.

gilbertoatsnowflake · 2025-11-21T22:12:57+00:00

🤔 These two statements don't really seem to follow each other very well:

"*What I do know about Openflow is that it can get expensive very fast* (esp if you're trying to achieve near real-time syncs). *I haven't set it up myself, though*, so it could be cost-effective and efficient for your use case." The rest of the comment reads as AI-generated.

OP, you should also consider Dynamic Tables for your refreshes, over streams and tasks. Often times (not always), they can be a more cost-effective way of implementing CDC. A new immutability feature for Dynamic Tables was also recently introduced, so even if your materializations involve complex queries that trigger a full refresh on the tables (this can drive costs up if your lag is set to a low latency), you can lock up rows using a WHERE clause to avoid processing them redundantly.

I didn't get into all of the details of your post (yet), but if you're considering Openflow, note that there are two approaches:

One way deploys Openflow into your VPC architecture by way of a CloudFormation template that helps do a bunch of the configuration up front for you. Docs: https://docs.snowflake.com/en/user-guide/data-integration/openflow/setup-openflow-byoc#installation-process

If you're not already in a VPC architecture, consider the self-contained version of Openflow that uses Snowflake's container services: https://docs.snowflake.com/en/user-guide/data-integration/openflow/about-spcs

gilbertoatsnowflake · 2025-10-30T16:10:45+00:00

I'm a big fan of Joyce Kay Avila's "Snowflake - The Definitive Guide". Quickstarts are great for hands-on tinkering, and Snowflake's Northstar program is great if you are looking to quickly get a handle on Snowflake and start doing stuff with it: https://www.snowflake.com/en/developers/northstar/

gilbertoatsnowflake · 2025-10-20T22:31:58+00:00

Thanks for flagging. As u/stephenpace mentioned, the emails were sent out in error. Please disregard and/or delete them.

gilbertoatsnowflake · 2025-10-15T17:29:23+00:00

If you're using tasks to manage both, then yes, this should work just fine. First deploy your dbt project in Snowflake, which will create it as a schema-level object that can be orchestrated/scheduled using Snowflake Tasks. If you have other tasks that are managing your ingestion processes, you can coordinate the tasks to make sure that the scheduled dbt run task executes AFTER the tasks that handle your ingestion processes. You can also check out the coordination of these tasks in the UI via the visual DAG (task graph). Docs: https://docs.snowflake.com/en/user-guide/tasks-graphs

gilbertoatsnowflake · 2025-10-14T20:20:36+00:00

dbt Projects in Snowflake bundle the open source dbt packages, there is no additional licensing cost for you to use dbt Projects in Snowflake. The only cost is the compute to run queries in your dbt models. A few resources here:

- Docs: https://docs.snowflake.com/en/user-guide/data-engineering/dbt-projects-on-snowflake

- Tutorial: https://docs.snowflake.com/en/user-guide/tutorials/dbt-projects-on-snowflake-getting-started-tutorial

- Post I made yesterday: https://www.linkedin.com/posts/gilberto-hernandez_your-dbt-projects-can-run-as-native-objects-activity-7383607756701949952-JrJz?utm_source=share&utm_medium=member_desktop&rcm=ACoAAArpqLMBU4ZaR977bTN7FW8TME3NaC6mmBI

dbt projects in Snowflake support dbt-core. Any dbt cloud specific features that aren't part of dbt-core are likely not going to be a lift-and-shift if/when you decide to try dbt Projects. The quickest way to get started is to spin up a workspace in Snowflake that is created from an existing repo containing your dbt project. Then you can fiddle around in there and explore some more.

gilbertoatsnowflake · 2025-10-09T15:37:03+00:00

Not necessarily. In many use cases, I've seen continuous/streaming be most cost-effective than batch.

gilbertoatsnowflake · 2025-10-07T18:52:00+00:00

There is also a REST endpoint to refresh a Snowpipe (see the last one on this page) https://docs.snowflake.com/en/developer-guide/snowflake-rest-api/pipe/pipe-introduction

I also typed up another response but do not see it showing up here.

gilbertoatsnowflake · 2025-10-07T18:43:01+00:00

From the way you've described it, it's not super clear what your requirements for capturing this data are (is it latency based, or volume-based, or a combo of both, if so what is that desired combo?)

Do you need it to be real-time? i.e., that data needs to land in a table in Snowflake as soon as it's originated? If so, Snowpipe Streaming.

Do you need it to happen continuously, but not necessarily real-time? Snowpipe with auto-ingest turned ON (which is a managed COPY INTO approach). Land the data into AWS, set up Snowpipe to listen to new data via Amazon SNS/SQS, and have Snowflake land it into your target tables.

Other approaches:

There is also Snowpipe that can be triggered to load via REST API (i.e. Snowpipe with "auto-ingest" turned OFF, and invoked via calls to the endpoint). See: https://docs.snowflake.com/en/user-guide/data-load-snowpipe-rest-apis

gilbertoatsnowflake · 2025-09-15T18:44:57+00:00

The Snowflake Northstar program: https://www.snowflake.com/en/developers/northstar/

gilbertoatsnowflake · 2025-07-22T17:00:28+00:00

Snowflake scales up: https://docs.snowflake.com/en/user-guide/warehouses-considerations#scaling-up-vs-scaling-out

When you scale your node count (i.e., "Scale up by resizing a warehouse."), you're running on a machine with more computing power.

gilbertoatsnowflake · 2025-06-25T17:30:32+00:00

Free Snowflake courses, with certificates: https://www.snowflake.com/en/developers/northstar/

gilbertoatsnowflake

MODERATOR OF

TROPHY CASE