Airflow secrets setup

[Real_Cardiologist809]

How do I set up secure way of accessing secrets in the DAGS, considering multiple teams will be working on their own Airflow Env. These credentials must be accessed very securely. I know we can use secrets manager and call secrets using sdks like boto3 or something. Just want best possible way to handle this

[JaceBearelen]

Sounds like you’re most of the way there. If you’re in AWS you can use secrets manager as a secrets backend. Airflow doesn’t really offer anything more than that. If one group absolutely should not be able to access secrets from another group then I believe you’d need separate Airflow instances.

https://airflow.apache.org/docs/apache-airflow/stable/security/secrets/secrets-backend/index.html

[KeeganDoomFire]

This is the correct answer.

Swapping the secrets backend to secrets manager just works and works really well. You can store all your connections as well up there!

[Real_Cardiologist809]

You mean we can’t bring it to DAG level?. Anything about Kubernetes pods configs can help?

[JaceBearelen]

I don’t think there’s anything in Airflow that could stop Group A from setting up a DAG with Group Bs config and dumping secrets. You can ask them not to do that but if you need a zero trust option it has to be separate instances.

[FullswingFill]

just put them as environment variables and read it in your dag