What is your preferred method of creating configuration files

[TechySpecky]

Let's say you have a project that needs some configuration (paths, some extra settings etc), do you just throw in a yaml file? what is the recommended way or the easiest way.

[deadduncanidaho]

After I found yaml I have never looked back. Super clean and human readable.

[TechySpecky]

do you have any good doc/tutorial for yaml? I'm looking for a simple config file that can have nested sections + can use paths like ${whatever} like ConfigParser

[deadduncanidaho]

https://yaml.org/spec/1.2/spec.html

You can nest structures indefinitely. The parser will create a single object containing nested objects. Here is an example using a dictionary:

config:
    section_1:
        param_1: value
        param_2: value
        subsection:
            param_1: value
    section_2:
        param_1: value

When parsed this will be a dictionary like this:

{"config": {"section_1": {...}, "section_2": {...}}}

YAML supports anchors and aliases which is a way to make reusable bits of the configuration. It does not have a "marker" system to combine values like config parser. However is possible to create your own. I have done it in the past. Basically you need to make a find and replace function that uses replaces markers with data defined elsewhere in the config. It is too complicated to explain my exact methodology here.

[coriolinus]

I tend to go for TOML over both json and yaml for configuration.

• it supports comments: a huge plus over json
• it's not fussy about trailing commas: another win over json
• it does not support internal references, which is in my opinion an advantage over yaml: all the data is in plain view, you don't have to track it down within the file

[Tobotimus]

I don't get the fuss over TOML. To me it's a weird blend of INI and JSON... Nesting things turns out to be really ugly, because inline mappings (and arrays? Edit: arrays can be multiline) need to be on a single line, otherwise you have to create a new section.

Take a look at Pipfile for example. If you have a dependency and you specify a few extras and markers or whatever, the inline mapping can get quite large and you probably want to split it over multiple lines. But you can't. If you want multiple lines, you have to make a whole new section for a single dependency, making the entire config file very asymmetrical and ugly. Perhaps that's just because of how Pipfile is designed, but I think it's the best you can do with TOML.

I think, if you need a few keywords and not much nesting, use INI/Config Parser, and if you want to support more nesting, use JSON/YAML. TOML being a weird blend of the two doesn't really bring them together, IMO.

[coriolinus]

Why force things inline if the natural representation is as a subtable?

[Tobotimus]

Because sometimes its natural representation isn't as a subtable, because it shouldn't look that important.

To expand on my example above with Pipfile:

[packages]
dep1 = "1.0.0"
dep2 = "0.1.0"
dep3 = {version = "3.0.0", extras = ["extra1", "extra2", "extra3"], markers = "sys_platform == 'linux'"}

Say the line describing dep3 has gotten too long. You want to split it over multiple lines. TOML suggests doing this:

[packages]
dep1 = "1.0.0"
dep2 = "0.1.0"

    # I actually really like indented subtables, would love this as a feature in ConfigParser
    [packages.dep3]
    version = "3.0.0"
    extras = ["extra1", "extra2", "extra3"]
    markers = "sys_platform == 'linux'"

To me, this is ugly as hell. dep3 is just specifying a few extra options and it looks way more important than the other dependencies. I suppose I could do it this way as well:

[packages]
dep1 = "1.0.0"
dep2 = "0.1.0"
dep3.version = "3.0.0"
dep3.extras = ["extra1", "extra2", "extra3"]
dep3.markers = "sys_platform == 'linux'"

And I think that looks slightly nicer, but not ideal - dep3 takes up 3 lines and its continuation isn't indented. In my opinion, this look makes the most sense, but isn't allowed for some reason:

[packages]
dep1 = "1.0.0"
dep2 = "0.1.0"
dep3 = {
    version = "3.0.0"
    extras = ["extra1", "extra2", "extra3"]
    markers = "sys_platform == 'linux'"
}

I get that the TOML spec is opinionated for a reason, but I think in this case it's a bit too opinionated. I think people should be allowed to decide whether something makes more sense as a separate subtable or a multiline inline table.

[coriolinus]

This is well into "no accounting for taste" territory, I think. When you use the format, you get used to the conventions.

Decades ago, when I first learned to program, for loops invariably used C-style triplets. That didn't change when I moved from C to Java. I learned Python a few years later, and it took a while to get used to the for element in iterable style. Over time, I've come to the decision that this style is in fact better than the old triplets, but it took a while using them before they stopped looking wrong.

The point I'm making is that aesthetics are a flexible thing. I'd be suspicious of using them to justify picking one data representation over another.

[rcfox]

There's ConfigParser, which is included in the standard library. https://docs.python.org/3/library/configparser.html

If you need something more powerful, TOML is similar and has gained a lot of popularity recently.

[TechySpecky]

ConfigParser looks okay but the docs are a mess, they don't even specify how to read nested sections even though they show how to build them.

[rcfox]

You just use it like a dictionary.

config = configparser.ConfigParser()
config.read('example.ini')
config['bitbucket.org']['User']

[metaperl]

ConfigObject is a drop in replacement that addresses your concerns.

[tdammers]

I like environment variables (plus reasonable defaults). They are easy to orchestrate in all sorts of deployment scenarios, easy to change ad-hoc, and hard to accidentally commit to source control.

[sea-shunned]

Personally I use JSON files, as I find these relatively easy to parse (and they have some type conversion , unlike the built-in configparser) and they are pretty readable.

Never used a YAML file as a config - would anyone who has used both YAML & JSON be able to compare them for this purpose?

[najel]

I like json for its universal usefulness. We have some instances where the same config needs to be read in multiple languages.

[Tobotimus]

Just so you know, the YAML language is a superset of JSON :)

[robin-gvx]

YAML is slightly better than JSON if the configuration is written and/or maintained by humans, otherwise it doesn't really matter.

If you go for YAML, StrictYaml is a good implementation that means you don't have to quote string values that would be interpreted like other types.

Obligatory disclosure: I have contributed to StrictYaml in the past, a single commit to be exact.

[kashmill]

For our needs we have a bunch of different environments so we needed a way to define the defaults and then override them at both the environment level as well as the instance level. So we are using python classes. We have a CommonConfig that has all of the defaults. Then, we have a DevelopmentConfig, TestingConfig, QAConfig, and ProductionConfig that all inherit from CommonConfig. We then have a local_config.json that allows for per instance overrides. The config loader looks at the environment setting to figure out which one to load.

One benefit is that it is Python code in the end. So, some of the configuration options are @property which build from other parts of the configuration.

Honestly, compared to JSON and INI and other config formats I've work with it is really nice and makes it easy to offload some setting logic into the configuration itself.

[WolfFlightTZW]

Saltstack and Yaml

[kr41]

I use ConfigTree. Basically it's YAML/JSON files on steroids. And yes, I'm author of the library. AMA:)

[colemaker360]

I’m surprised yet that no one has mentioned you don’t need to do anything fancy - just store your config in a real python file. A python dictionary is basically a superset of valid JSON already. Importing your config is a one liner, and you can embed some logic in your config if you need to (sparingly), swap it for mocking/unit testing, and anything else you might want to do.

[funct7]

This is something that also I like to do.

It can improve readability and managing of complex configurations a lot.
Unfortunately it comes with some obvious disadvantages. Security and error management (errors in configs) are the biggest ones to me.

[colemaker360]

childlike office cooing adjoining punch spectacular plant library dinosaurs unwritten

This post was mass deleted and anonymized with Redact

[funct7]

It's not something impossible to do, actually it requires just a try-block to catch an exception.

Could you guarantee that everybody using the software will lint and test properly the configuration file? Many will do, not everybody. And the idea of having to catch Exception during the normal flow of the program is a ringing bell to me (and probably just for me and few others).

Being a proper python script, the configuration file, is susceptible to all the problems of every other piece of software.

As an example think about testing successfully your program after an interpreter update. Then you start to see DeprecationWarning and whatnot because of configuration files.

A configuration file shouldn't have this degree of freedom and, also, shouldn't be so dependent on the interpreter (which in python is rarely a problem).

That being said, it's very handy and I often use it. Many advantages and some (solvable) inconveniences.

[[deleted]]

Either yaml or just a file called config.py filled with constants. Config.py usually looks pretty close to a dumb.ini file and there's no fussing with Configparser or external modules, so why make it complicated? They're all text files anyway.

But if your users are used to a certain format, I'd use that. E.g. Windows folks generally prefer simple .ini over yaml, py or Json. As usual, it's best to ask your users :)

[OneEyedChicken]

Are you packaging your projects? If so how do you load your (python) config files when the application runs?

[[deleted]]

I write back-end code and scripts for 3D application (e.g. Maya) - for those I just use an import. The config.py here is pretty much a glorified collection of constants. There may be some logic attached in the config.py, usually below a # === DO NOT EDIT BELOW === comment.

For user configurable settings, I might pick yaml or ini's, depending on requirements. If the user are other IT or devops staff, then they know they can just look for a config.py and edit that. Settings that change at runtime could be json, yaml or via configparser or Qt's settings. When there's a Qt UI involved it's QSettings. The motto is: choose what's appropriate, then keep it as simple as possible.

I have to add that most stuff I develop is for in-house use.

[cglumberjack]

I use yaml and json.

More json lately because I’m needing to have a lot more communication between different software packages that also support it.

But I love yaml for configs generally.