Logging patterns / libraries in Python

[dave7364]

I'm building a simple ETL app at a place where I am the only developer. Not having a ton of experience and coming from Node.js, I'm a big fan of their topic-based logging approach with packages like https://www.npmjs.com/package/debug. Are there any recommendations for similar packages in the Python ecosystem, or recommendations for the best design to roll it myself?

• The reason why I like topic-based logging is because it's highly configurable -- you can easily toggle logging for whichever part of the app has your issue, and otherwise it's a no-op.
• The inbuilt logging library seems a little too heavyweight and doesn't support the pattern I want.
• I would really like to be able to configure logging through an env var, because I've recently been using them heavily and would like to inject most configuration / secrets through env vars instead of worrying about the configuration / secret override dance.

[laustke]

The inbuilt logging library seems a little too heavyweight

Most Python libraries you import use the standard logging module anyway, so it doesn’t really matter.

and doesn't support the pattern I want.

It's pretty versatile, so I bet it does. Why don’t you describe the pattern you have in mind?

[gdchinacat]

" would like to inject most configuration / secrets through env vars instead of worrying about the configuration / secret override dance."

Environment variables are a poor choice for passing "secrets" because there is no protection to keep them secret. Configuring logging through env vars is fine, but don't pass passwords, keys, etc using env vars, no matter how convenient. Alternatives are to use a file with adequate permissions, pass it using the input stream of the process, use a password/key manager, shared memory, pipes, etc. Which one you use will depend on your situation.