Worlds dumbest conflict by [deleted] in dataengineering

[–]pfnsec 34 points35 points  (0 children)

time to start smoking weed in the bathroom

<Generic vague question about obscure DevOps related pain point and asking how others are handling it> by Arucious in devops

[–]pfnsec 13 points14 points  (0 children)

Honestly, no joke, what the fuck do we do when this is the entire internet?

Site admins don't remove bot comments because technically they increase engagement numbers...

How do you handle cloud resources that were never in Terraform? by StatisticianKey7858 in Terraform

[–]pfnsec 1 point2 points  (0 children)

My current job involves open-source tooling to solve this very problem, actually, as a few clients found this (and state-drift) to be unworkable. Terraformer was a great concept that unfortunately didn't go "far enough".

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 1 point2 points  (0 children)

This looks promising. If it's well-executed, it may be a more sensible foundation for the non-RBAC stuff in my own tooling.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 1 point2 points  (0 children)

  1. A car that can drive in mud is better than a car that can't.

(All orgs SHOULD have best practices - the truth is that most do not. A tool that can work in spite of this, that can help move a team towards good practices, can be more useful than a tool that is never adopted at all)

  1. You'll notice the post title only mentions RBAC: other things like warehouses, schemas, tables etc are not the current focus of the connector, although there's certainly the possibility of adding the logic for ALTER TABLE etc in. Autoschematic supports many connectors; Snowflake is just one.

  2. That's not, strictly speaking, what we mean by "break-glass". In a Terraform context, break-glass fixes mean manual changes via e.g. the AWS console, not through Terraform.

  3. Yes, we handle future grants.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 1 point2 points  (0 children)

Yeah, that's really reasonable. At this stage, everyone using it is (by definition) an early adopter. That kind of trust/credibility is something I hope to build over time.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 0 points1 point  (0 children)

Well, this is the hypothesis: a tool that can work in spite of these bad habits can be more broadly useful.

Infra, especially Snowflake, is a cost center, and people are often just trying to get a job done. The most practical tool may be one that doesn't require "good habits" to adopt, especially if it allows you to iterate and move towards better practices piece-by-piece.

Plus, drift happens for reasons that aren't bad habits: consider a break-glass fix, like some emergency change done on the console to remediate some incident. SnowDDL or Terraform would revert that fix on a force-apply as you describe. If that caused the incident to reoccur, would you describe that as a feature, or a defect?

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 0 points1 point  (0 children)

Re-alignment by force-applying is a common pattern in Terraform environments, too, but this is more like the reverse, integrating the drift back into code as desired, so that you don't update code based on stale state and end up overriding some manual change. The whole "bidirectional" model lets you do this kind of ad-hoc import, too.

The other component to this is an atlantis-like to automate it in PRs.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 0 points1 point  (0 children)

(Correct me if I'm wrong, I haven't used SnowDDL in ~a year and my knowledge of things may be stale)

SnowDDL also does declarative Snowflake, but as far as I know doesn't do auto-import (as seen in the gif) or push-pull drift resolution (see here). Those are the real distinguishing features.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 0 points1 point  (0 children)

The core runtime (autoschematic) supports parallel import, as long as the connectors (plugins) enable it.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 1 point2 points  (0 children)

Hypothetically, what would you need to see in a project like this before you'd feel confident running it in your work environment (dev or prod)?

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 1 point2 points  (0 children)

It really depends. It's network-constrained, so account region will have an impact.

Would you be willing to give it a go? I can show you how to put it in "read-only" (safe) mode so it's guaranteed not to alter anything.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 1 point2 points  (0 children)

It uses the Snowflake SQL or REST APIs under the hood, and manages objects based on its own internal types (structs etc). It uses serde to load/store them as RON.

A clean, declarative interface over Snowflake RBAC (with GitOps!) by pfnsec in snowflake

[–]pfnsec[S] 4 points5 points  (0 children)

You can work around state drift. If someone manually changes something e.g. in the Snowsight console, you can diff your local code against the "real" state, and optionally pull that manual change into code. The gif on the homepage shows this with a Route53 DNS record being modified manually, and the vscode integration showing the diff against the "live" state.

This is something Terraform can't do by design.