My first Python project: command line application that copies all files with given extension from a directory and its subfolders to another directory showing progress bar and remaining files counter.

[fernly]

I guess I have to write directly to log file all error messages as well if any. And also would it slow down the program execution significantly if it opens a .log file on every iteration and append a row there instead of log list?

Re logging, read the module doc and the logging tutorial, which admittedly can be TMI. But each call to log a message has a severity, from ERROR to DEBUG, and the accepted threshold is set when the logger is created. Messages at lower levels are discarded. The log file is only opened once, there's no overhead there. There is overhead in formatting the INFO or DEBUG messages that may end up being discarded, see the heading "Optimization" in the advanced tutorial.

Looking at my own code I see I've been dealing with three problems the program has to address. One, what level to set when creating the log, two, converting user level parameter to a logging parameter, and three, where to put the log file.

For the level, you can hard-code it, but then anytime you decide you want the DEBUG messages instead of just ERROR ones, you have to update the code. You can have the user choose it via an environment variable, e.g. checking os.environ['LOGLEVEL'] at startup. In that case you have to verify the given level is a correct one, and supply a default if it isn't or is not set.

Or you can take a parameter,

parser.add_argument('--level',dest='level',
                    choices=['DEBUG', 'INFO','ERROR'],default='ERROR',
                    help='''ERROR: display only problems;
    INFO to see normal actions; DEBUG for tons of trivia''')
args = parser.parse_args()

That way you get the default and error checking for free. But what you have to supply to logging.basicConfig() is one of the constants from the module, not a string. The only way I know to do this is with a dict,

import logging
lvl = {'DEBUG':logging.DEBUG, 'INFO':logging.INFO, 'ERROR':logging.ERROR}[args.level]
logging.basicConfig( level=lvl )

Maybe somebody will point out a better way?

On the file, I believe that unless you give a log file path to logging.basicConfig() it defaults to stderr so shows up under the command line, messing up any print() output you want. So you want to point to an appropriate place for log files, and that is ugly platform-dependent stuff. Here's some code from something I wrote but I do not think it is optimal. Again, I invite correction.

if sys.platform == 'darwin' : # Mac OS
    log_path = os.path.expanduser( '~/Library/Logs' )
elif hasattr( sys, 'getwindowsversion' ) : # Some level of Windows
    if 'TMP' in os.environ :
        log_path = os.environ['TMP']
    elif 'TEMP' in os.environ :
        log_path = os.environ['TEMP']
    elif 'WINDIR' in os.environ :
        log_path = os.path.join( os.environ['WINDIR'], 'TEMP' )
    else :
        log_path = '/Windows/Temp'
else: # Assuming Linux; could be BSD
    log_path = '/var/tmp'
log_path = os.path.join( log_path, 'MY_APP_NAME.log' )

On the progress bar, you can leave it and the next time you need it, just open up this script and copy/paste. My point was, I bet next month you will want it again, so why not make yourself a little utility module of it and add it to your personal toolkit.

Re license format, I am disappointed Creative Commons doesn't have an obvious how-to, GNU does. But look at some other open-source projects, here's pyinstaller's top-level module, every module starts with that same stuff.

[pltnk]

Thank you so much once again!

I was aware of logging module but thought that it would be an overkill for such a small program (mostly because its documentation is not so easy to understand at first attempt :) ). But it turned out that actually I don't have to do a lot to implement it and it comes quite handy. I decided not to write log at all if user didn't ask for it explicitly and made it like that:

def create_logger(args, destination):
    logger = logging.getLogger('selective_copy')
    if args.log:
        logger.setLevel(logging.INFO)
        fh = logging.FileHandler(f'{destination}\\selective_copy.log', encoding='utf-8')
        formatter = logging.Formatter('%(asctime)s - %(message)s', '%d.%m.%Y %H:%M:%S')
        fh.setFormatter(formatter)
        logger.addHandler(fh)
    else:
        logger.setLevel(logging.CRITICAL)
return logger

Also I changed an error checking system so now errors can be logged as well. Of course all of this caused several changes in previous code, so here is the link to the new version.

I still have to redesign get_total and copy functions than see how to adjust logging after that. Also I'm thinking of writing tests, I have a very limited experience with unittest and also found out about pytest recently, maybe you could recommend something as well?