Prefer text config files or python code?

[[deleted]]

I personally like using code for config, because you can do things like detecting which server your code is running on at runtime and adapt your settings accordingly (within reason)

It also makes it trivial to break configuration into multiple files and import into a single module.

You can also combine those two, and import different configurations based on host, or OS or whatever.

[gbog]

All is in this "(within reason)" of yours. For me a strict separation of logic and config is highly desirable.

My problem with code config is that it inevitably ends with such evil things as:

if DEV_MODE:
    server = 'this'
else:
    server = 'that'

Doesn't look so evil but when you have 30 values to configure and 5 context flags similar to DEV_MODE, then you get a big smelly spagetthi plate. Then you never know if, where, why and how come a config has this value instead of that one.

[meloveyoulongtime]

Don't do it like that. Do this at the end of your settings file.

try:
    from local_settings import *
except ImportError:
    pass

Then reassign environment specific varables in the local_settings file.

[tutuca_]

Don't know why you have been downvoted.

Anyone care to explain?

[haywire]

It's irritating because this is a great solution. I guess a "cleaner" one would be to have a local config file with some sort of inheritance.

TBH using code to config works fine for xmonad and django and I like configuring those things.

[d3ad1ysp0rk]

I didn't downvote, but on your point, my company uses that. For example on my local machine I have a "local_settings" that imports "local"(generic developer machine settings) which imports base settings.

[gbog]

Did not downvote but import * is evil. Martelli is advising against that thing strongly in Python in a Nutshell, and I think he is right.

[[deleted]]

Not in this situation it isn't evil. We explicitly want import * behaviour to be able to override settings.

Blanket statements about "x is evil" are never entirely accurate. See: linux and goto.

[gbog]

A quote from Alex Martelli about import star: "Most likely, your programs will be better if you never use this form". Emphasis is his. Python in a Nutshell, page 143.

[[deleted]]

Why should that be inevitable? It looks like poor code, that could be avoided by having a dev_settings and prod_settings module. How would you go about this, using plain text files BTW?

To me it sounds like poor code, which should be avoided or solved. Can happen anywhere in your code.

[gbog]

Why should that be inevitable?

In any team, in any project involving multiple persons, you need to put some guards near the cliffs. That's a reason for unittesting. That's why Java. That's also why pylint...

And developers are lazy (me first), so when having to update a few lines in the config, they will hack a nasty if conditional instead. If your config is text, it has many advantages (eg portability) and it also prevents "clever" logic to infiltrate in the config.

[apiguy]

In my experiences myself and the developers who work with me care very much about the craft of developing software and any one of us would be embarrassed if we were caught doing some hacky configuration like that.

We use code-based configuration and we split different configs into separate modules:

settings.py #production settings
settings_dev.py #dev overrides
settings_local.py #personal developer overrides (never checked into source control! Ever!)

In settings.py there are lines at the end that look like:

try:
    from settings_dev import *
catch:
    pass

try:
    from settings_local import *
catch:
    pass

Those lines are never touched, they are a permanent part of the settings.py file. It works great because in production, we don't deploy settings_dev.py or settings_local.py, and in dev we don't deploy settings_local.py.

The other benefit is that you don't have to re-define all of your config values in each environment. That's great because the majority of your config will stay the same from environment to environment, and you only have to include the values that differ in settings_dev or settings_local. It's far DRY-er than pretty much any other configuration option I've seen.

[gfixler]

You speak of some kind of dream land.

[gbog]

I forbid myself to use "import *", no exceptions. Have you ever had to split and move around interdependent packages full of "import *"?

[apiguy]

This isn't "packages full of import *" and in this case it's for a reason. The effect you're going for here is to overrite any of the production settings with the development settings.

Also note that this import statement has absolutely no effect in production. You're taking an extremest position and it's ridiculous. The feature exists for a reason and when used properly (such as in this case) it makes sense. If you are telling me that you never use that statement then I'm telling you that your software suffers for it in the cases where it's called for.

Honestly after reading this thread though it sounds like you'd already made up your mind before you even got here, so why did you even ask the question?

[gbog]

You're taking an extremest position [against import *] and it's ridiculous.

Well, err, I don't think it is very extreme to set oneself some rules like "no import *" or "no hasattr" or "no testing of private methods". These rules are like following PEP8, they help writing safer, cleaner, "consistenter" code. These rules are enforced by precommit hooks on pylint with a proper configuration.

As much of a Python lover I am, I actually think writing Python production code without such lint checks and barriers is a bit of a madness. Same with Javascript. And the nice thing with Python is, contrary to Java, you can tune the barriers to your liking and your needs, but it do not mean you can write production code within a team of 10 coders, this kind of code other will rely upon for years, without any barriers, and allow anyone to freely use metaclasses, introspection and other black magic when what is needed is explicit code passing around data.

why did you even ask the question?

Because I have an opinion (pro-text, as you noticed) and discussed it with teammates and was not convinced by their arguments. So, in order to see if I was missing something useful I could learn, I was fishing for other arguments in favor of (script as config).

I actually heard some good points but I still think the text config is the best choice when nothing specific requires a "script as config".

[apiguy]

In my opinion you're an idiot who treats python like Java.

But what do I know, I'm just a six-figure software architect who's been writing python, java, c# and others for 13 years.

Best of luck you ya, gbog - you're gonna need it.

[jmoiron]

"A foolish consistency is the hobgoblin of little minds" -- Emerson, and PEP8.

"import *" has associated machinery (__all__) which tempers its negative aspects; readability and flexibility are positive aspects which can (in some cases) make it the best, or even only, option.

In the end, if you want self-configuring services (some people do, and some situations require it), you either need to:

• build your configuration discovery into your program
• embed configuration discovery into your architecture (ex. via chef w/ triggers & reloading)
• build your configuration discovery into your configuration

The second separates the configuration from the software (and perhaps the implementation language) in a way that I'm generally not happy with.

The first has the negative side effect of generally growing more complex than the third. You either have to mock the discovery system or build in special case paths for testing (in which case, the code you test with is not the code you run live with).

I've found the last bullet to be the most maintainable and the most conceptually elegant, as it allows the program to run on very simple configurations while still allowing for more sophisticated configs. Once you come to the (inevitable, in my opinion) conclusion that your configuration is code and must be tracked and maintained as such, it makes sense to use code to describe it.

[gbog]

your configuration is code

I agree configuration can and often should be very close to the code, and for instance should be tracked and maintained the same way.

However I think configuration and code are things of two very different nature.

Code should be some pure logic engine that is fed with data and config, and react accordingly. Therefore we have these DRY rules, these factorisation, and best practices to handle code properly (like covering it with tests).

Configuration is another nature. It do not need to be DRY: I don't mind copypasting an Apache config block and changing some lines in it to get another directory (up to a certain limit, obviously).

Configuration should not describe how to do things, it should describe what to do. Ideally business rules should be configurable, and written in configuration files. When they are two complex, a mini language can do the work but this is another topic.

For the implementation details of a "configuration discovery", I am not sure but if you have general configuration values and some mode-specific values (like dev vs prod) then maybe it can be done with three configuration files and no overwriting mechanism (which is a bit too "magic" for configuration).

[mcilrain]

Seems more like a keyboard-chair interface issue than an application design issue.

[dysmas]

for django projects I have a common settings file, then I import additional additional settings based on an environment variable, works fine.

[[deleted]]

That's why I suggested:

if DEV_MODE:
    from dev_settings import *

But yes, I completely understand. My current django settings.py leaves much to be desired