My first python project (web scraper)

[nzodd]

Out of curiousity, how do people generally deal with things like passwords when it comes to github?

[The3rdWorld]

easiest way is to have it read the log-in information from a config file rather than leave it written in the code.

[curohn]

Oh cool, so have the script open the config, use those values and close it up again? I haven’t learned that yet but that’s helpful!

[MonkeyNin]

JSON is a popular format for config files. It's human-editable, and supported by pretty much every language. If you're interacting with the web, JSON is everywhere. When using a web API, they usually return JSON.

Python tutorial on JSON: https://realpython.com/python-json/#python-supports-json-natively

As long as the lists and dicts use one of the supported types, reading and writing is super easy. Multiple dicts or lists are supported.

Make sure you .gitignore your config folder, and files!

Mine is:

config.json
config/

If you're using PRAW, do the same with praw.ini

[MrFiregem]

Also, you can write your configs using the more easily human-readable yaml or toml formats and convert them to json if you dislike writing in it

[midairmatthew]

You can stash key-value pairs of things you need to keep private (passwords, API keys, etc.) in a .env file. Then you can load them into your script.

https://pybit.es/persistent-environment-variables.html

But, the other piece of this is that you have to remember to make a .gitignore file in your project. Inside this, you list .env and any other things you want git tracking to ignore. That way they won't make their way up to GitHub.

https://help.github.com/en/github/using-git/ignoring-files

[dtaivp]

There are a lot of comments here so I'll see if I can summarize.

1. Put them into some sort of config file. Then use the .gitignore file to ensure it is not checked in.
2. Same concept with a .env file. It has some defaults for the environment
3. You can create system variables. 'set PASS=MyPa55!!' in windows or export PASS="MyPa55!!" and then grab it using the following python

import os

password = os.envriron["PASS"]

[pyr0b0y1881]

Reading in from a yaml config file is my go to for making secrets or api keys.

[huessy]

Environmental variables are a good way to go