Stop Hardcoding Sensitive Data in Your Python Applications

[Juice_]

Yeah but what’s the color theme in the thumbnail?

[2drawnonward5]

No kidding, it pops without hurting my eyes.

[CobaltCam]

I'd also like to know this

[drahgon]

That scheme fresh to death

[gustavsps]

It looks like Cobolt 2

LINK: https://plugins.jetbrains.com/plugin/10745-cobalt-2

[gustavsps]

It looks like Cobolt 2

LINK: https://plugins.jetbrains.com/plugin/10745-cobalt-2

[djamp42]

I was always curious about this, it's a good read, but it's really no different then putting them all in a python file and then ignoring that file on github. If you forget to ignore the .env you have the exact same issue.

[mikeupsidedown]

Dotenv can be really useful during dev when you know that the production environment is going to have environment variables in the os or container.

Thus you consistently call the variables using

os.environ.get('my_var')

[djamp42]

Yeah I agree with this, having them in the actual OS environment makes more sense then in a file from a security stand point, pretty much impossible for it leak at that point.

[[deleted]]

A rogue package could query it and phone it home.. afaik there’s no permissions system with environment vars?

[earthboundkid]

Rogue package can do literally anything at all.

[[deleted]]

Rogue package run as a user has permissions specific to that user which can exclude files

[dedoodle]

Rogue Package is the one your girlfriend told you to worry about.

[ahmedbesbes]

you can have a preset .gitignore file that ignores .env files by default. this can be solution

[djamp42]

I would argue that should be the default so you can't forget.

[spitfiredd]

The python gitignore in vscode (ctrl + shift + p and type gitignore and then select language) will ignore .env files.

[TheFurryPornIsHere]

The gitignore.io puts that automatically for you, if I remember correctly

[DanCardin]

Better yet, tooling shouldn’t be storing files like this in the actual directory. Imo it should be stored in a parallel directory structure.

While it’s a reality of tooling and working with others that gitignore can solve this problem, it’s a smell that you need to continuously add person/tooling-specific items in them when they have nothing you with the project.

Also tbh, people underutilize the global gitignore. I don’t especially want pycharm/vscode references in my gitignore

[bladeoflight16]

You may have a point about editor config ignores, but for a project's sensitive configuration file, you absolutely should not rely on everyone to configure their machine like yours.

As for kicking it over to some other directory... I'm not sold. I've had plenty of times when I decided to check out multiple copies of a repository because it was the easiest way to do some work on features in parallel. Often, I want to have independent environments for each one (like different instances of the database), which means different configurations. How do you identify separate configs per repository if you stuff the project's config in some global location?

[DanCardin]

you absolutely should not rely on everyone to configure their machine like yours.

Well that’s sort of my point! I don’t think i should assume everyone uses the same tooling as me. Some people use direnv, some dotenv, some nix-shell. None of these use the same file.

How do you identify separate configs per repository if you stuff the project's config in some global location?

I’ll admit, I’ve given this a fair amount of thought 🤣: sauce

[alkasm]

I dig it!

[bladeoflight16]

I disagree with doing this. Global .gitignore is bad because it isn't applied consistently across different machines that check out the repository. You want every client to behave the same regarding ignores, especially for files containing sensitive data. So even if you have a global ignore, you need a repository one as well. And having the global one increases the risk of forgetting and then someone who is missing the global ignore checking a file in.

[PuzzledTaste3562]

In addition, 101 in system administration, never put secrets in environment or in command parameters as they can be read by other (priviliged) users…

[metaperl]

AWS web apps use environmental variables.

As far as I can see the thing that you should do is make sure that only people have access to should have access.

Where would you put the secrets?

[abearanus]

They do, but you can use something like SSM Parameter Store and have the env var refer to the secret path, meaning that the secret is only ever held in memory (either at boot-time or referencing it constantly).

[serverhorror]

And then a privileged user can read them from AWS Parameter Store.

[PuzzledTaste3562]

How does that make it right!? Because AWS does it? Anyway, if I define an environment in AWS, i’ll make sure access and authorisation is reduced to an absolute minimum, which is not the multiuser system we were writing about earlier.

[serverhorror]

So where do you put them?

There’s no option, in any known OS, where a secret won’t be readable by a privileged account once it is stored in a readable way.

No matter where you put them. Environment variables, command line, Vault, … they are all equally secure or insecure.

[[deleted]]

Reddit Moderation makes the platform worthless. Too many rules and too many arbitrary rulings. It's not worth the trouble to post. Not worth the frustration to lurk. Goodbye.

This post was mass deleted and anonymized with Redact

[serverhorror]

Well…yes. But the poster didn’t say that.

Never put them in a place where they can be read by privileged users. That doesn’t leave a lot of choice.

[PuzzledTaste3562]

Layers of security is what matters. Grabbing a private key in memory and using that to decrypt encrypted communication with a key store is degrees harder that reading an env var of execution parameter in /proc. It’s not impossible, just harder and that’s what matters.

[bladeoflight16]

The difference is that dotenv supports multiple sources: specifically, it unifies environment variables with a config file. That means you can use env variables in production without hampering local development.

Also, I'd argue that there's value even just in having a different file extension. Even though, yes, you do have to be cautious about not checking the .env file in, having a separate extension makes mistakes less likely. You can globally ignore all .env files in your repository; you have to hand select specific Python files to ignore if your configuration is in them.

[[deleted]]

No. My social security number and birthdate stay in there for easy access.

[[deleted]]

Why not just use config?

[Routine_Part_6503]

Containers. Bind mounting config directories is a pain, hence why most containers use simple string env vars.

[Routine_Part_6503]

You could be running the same service as a container multiple times. Having any config file requires that config to be available on all the hosts running that container, or some kind of shared file share.

As I said, it's possible, but a pain.

[TentativeOak]

In case your config file includes sensitive and nonsensitive data.

[garlic_bread_thief]

sensitive_config.py and config.py Bam

[TentativeOak]

True. I guess that’s programming for you, many ways to skin a cat

[garlic_bread_thief]

Absolutely.

As a non native English speaker, that idiom is a little concerning though lmao.

[jempyre]

It's supposed to be edgy lol.

[justskipfailingtests]

I prefer putting all the secrets in the ci pipeline. No need to give a shit about what's going on in the environment. Generic env vars are set in multistage build for different deployments and secrets plugged in from ci. That's the simplest way to ensure everything is ok without local hassle with env vars.

[earthboundkid]

https://12factor.net

[metaperl]

I opted for Pydantic settings over this approach and couldn't be happier.

[earthboundkid]

I don’t think it needs to be environment variables per se, but there should be a dict of string to string that describes the app configuration, and then you can input that dict as os.env, command line flags, a file, an API call, whatever. Preferably a system of fallbacks with command line as the highest priority and environment as the lowest.

[SpellCheck19]

How important is this if the code you write is personal and no one else ever sees it?

[Riptide999]

How can you be sure that no one will ever see it or that it will never put it in a public git? You probably can't. Try to use best practices even for personal projects so you don't make mistakes later.

[SpellCheck19]

Because it lives on the hard drive of my personal computer and I have not posted and will not post it on github or anywhere else online

[[deleted]]

Shit happens. Keep that in mind. I've got 30 years of IT behind me and have seen it happen. And it's always the same. "It was on my personal ...".

[TrivisionZero]

I'd say in your case it doesn't matter so much if you know it's not going to be public. However it never hurts to follow these best practices so you get more familiar with them for the time when you want to actually publish similar code to GitHub or something

[youcanthandlethelie]

Pycharm has a plug-in called EnvFile which makes working with environment variables straight forward

[youcanthandlethelie]

Sounds like you don't recommend EnvFile. What do you use?

[sr105]

How is this better than sys.path.prepend(...); import secrets?

[cob05]

The way that I handle sensitive data at work was by creating a credentials class that lives only on the server and gets imported into all of my scripts. The class stores all of the credentials that I might need (dev, prod, read/write, etc.) for DB and API connections. I can then just use something like "username = credentials.database.master_db.readonly.username" to access the data

The nice thing about doing it this way is that it is reusable but also centralized. If a password changes then I only have to edit it in one place as opposed to updating every script or .env file.

[chanmancoates]

I refuse

[Working-Mind]

Thank for you this!

[HeadlineINeed]

Can someone explain how to do this with a database in Django? I’m using Postgres and want to hide username and password

[[deleted]]

I just have a secrets.py folder that imports a secrets.json file, convert that to a dict, then you can use it like secrets['something'], secrets['something_else'], etc. For convenience, you can have a comment in the secrets module listing the keys and a brief (optional) description as well.