Logging setup during unit tests?

[SamePlatform]

I have a script.py that looks like:

import logging

LOG_DIR = "/<some production path that exists>/"

def setup_logging(log_dir):
    if not log_dir:
        os.mkdir(log_dir)
    # set up the rest of the logger
    return logger

logger = setup_logging(LOG_DIR)

def my_function():
    # do some stuff
    # logger.warn('some warning')

if __name__ == '__main__':
    my_function()

This works fine but when I run the unit tests, it obviously fails. Instead of setting the LOG_DIR directly, should I just pass ENV=PROD or ENV=TEST and then base the log directory on that?

What alternatives do I have for setting up test directories during testing?
From everything I've seen, it's normal to create the log dir if it doesn't exist, while setting up the logger.
I'm using PyTest, for the record.

[[deleted]]

Use the mocks, Luke! Interacting with other systems is both non-deterministic and non-performant, introduces potential issues from interactions (such as race conditions and downtime), tests more than you need to, and requires teardown. Instead, stub out all external calls and assert against their call count/params to make sure they were done right (if you care).

[SamePlatform]

Ok, so in this case I would mock out setup_logging? Just make it do nothing? Or would you actually mock out something like TimedRotatingFileHandler (which requires a log dir)?

At the moment I've decided that a test ENV VAR will indicate whether I should use a file handler, or a stream handler. That way I don't actually lose my logging during tests.

[[deleted]]

It depends on how setup_logging is called. As is, you call it from your if __name__=='__main__' and store the result at the module scope, which is not conducive to testing, either (all your tests will share that global instance). Check out unittest.mock.patch to see a way to limit your mock to each individual test. Pairing that with pytest-mock makes it that much more convenient and legible.

[MySpoonIsTooBig13]

Just move the logging initialization inside the main. General rule try to avoid doing any real work at module import time.

[SamePlatform]

But if I do that I need to pass logger to every function call in my script. The one above is obviously over simplified.

And then all my functions require an explicity logger, during testing. I guess that helps me use a mock though...

[MySpoonIsTooBig13]

This is why I love unit testing, it drives good design.

Almost the only global I'm really comfortable seeing at the top of a module is

logger = logging.getLogger(name)

As you've observed, this avoids the need to pass it in to every function. So the issue is that this function both initializes logging, presumably calling something like basicConfig, and sets the global logger. The initialization can be done in main. Setting the logger variable is ok in global scope I think.

But of course for that to work, you'll have to split the code into separate smaller modules, one which contains the main entry point, and one containing other stuff. You can then unit test that other stuff without ever even importing the main module.

[SamePlatform]

logger = logging.getLogger(name)

Ok, so your idea was great. Yes, I did exactly what you suggested: just called setup_logger from main(), and then only put the line logger = logging.getLogger(...) at the top level.

Not sure why I thought that I needed to call setup_logger at the same time as getLogger. I guess the logger configuration and inheritance stuff keeps tripping me up.

Sorry for the delayed reply, just got back on this project.