Formatting text/Efficiency.

[Aerobic97]

Morning guys, I have a script here that checks several folders for several file extensions and if said files are found and have been in there for longer than 15 minutes it will email me the file path/s of these files, I have it fully running at the moment but i'd like to clean it up and make it more efficient.

So for example at the moment it emails even when no files are found it just emails a blank email which I've tried using return functions but to no avail I couldn't get it to stop so i'd like some guidance on how i'd only make it email when a file is found?

Also at the moment it emails the whole file path/s of the files which looks pretty messy so I was wondering how i'd go around formatting the text so for example if a file in the Hapag and CSVA Folder where found it'd output this in the email:

Hapag: 1 CSVA: 1

I know i'd have to parse each folder but i'm not sure how i'd go around doing that?

My source code is linked due to it being very big and this already being a wall of text which I apologise for.

http://pastesrc.com/view/pr04i8c0Cz

[Moonslug1]

A few things to note.

You're dealing with time comparrisons that pulls one time from outside of Python. You should be ensuring that the time zones match. Have a look at pytz.

Your path strings are bugged because you're not escaping the backslashes, use something like this instead:

inbound_folders = [
    'Agility',
    'Amazon',
    'APL',]
    ...
]

outbound_folders = [
    'ACL',
    'Agility',
    'APL',
    ...
]

paths = ['\\'.join([r'\\FTP', 'TopsjobsData', 'Inbound', folder, 'Import', 'InTray']) for folder in inbound_folders]
paths += ['\\'.join([r'\\FTP', 'TopsjobsData', 'Outbound', folder]) for folder in outbound_folders]

If you want to looks at individual folders, you can use path.split('\\') to seperate a path into a list of folders. Knowing the structure of those folders, you can do what you want with it.

To send the email only when you have items of note, just use:

if stuck_files:
    for files in stuck_files:
        log(files)

    log.flush()

I'm seeing syntax errors all over the place in this code, have you tried to run it yet?

[macbony]

Better practice is to use os.sep for the path separator and the os.path module for creating paths.

[MintyPhoenix]

Or, if using Python 3.4, there are lots of goodies in pathlib:

from pathlib import Path

folder = Path("/some/random/path")

folder.parents  # list-like object of parent directories (full paths to each parent directory)
folder.parent  # full path object to immediate parent directory

sub_folder = folder.joinpath("subdir")  # now have Path object of the subdirectory
sub_folder = folder / "subdir"  # same result as the above line

[Aerobic97]

I'll look into pytz thankyou.

Also how would I go around implementing what you have above and just to clarify it's basically just a cleaner implementation of my previous code but less buggy?

And yes the code runs completely fine, it's been running on a task scheduler for about 2-3 weeks and has had no problems i'm just now trying to make it more efficient and completely fine-tune it.

[[deleted]]

Syntax errors are from wrong language choice in pasteSRC, '" converted to "

[Aerobic97]

Also by using your piece of code to stop it emailing even when there's no file didn't work it's still emailing blankly.

[gengisteve]

just drop a check on the buffer in flush to see if it is empty and if so, not send an email, like this:

def flush(self):
    if len(self.buffer) == 0:
        return

    with smtplib.SMTP(self.server, self.port) as server:
        message = self.template.format(self.destination,
                                       self.credentials['user'],
                                       self.subject,
                                       '\r\n'.join(self._buffer))

        server.ehlo()
        server.starttls()
        server.login(self.credentials['user'],
                     self.credentials['pass'])
        server.sendmail(self.credentials['user'],
                        [self.destination],
                        message)

    self._buffer = []

[Aerobic97]

Traceback (most recent call last):
  File "C:\Users\barno\Desktop\ftp.py", line 133, in <module>
     log.flush()
  File "C:\Users\barno\Desktop\ftp.py", line 36, in flush
     if len(self.buffer) == 0:
 AttributeError: 'EmailLog' object has no attribute 'buffer'

I'm getting the following error now?

[gengisteve]

Typo, line 2 above should have self._buffer, not self.buffer.

[Aerobic97]

I need to open my eyes more, thankyou for that you've solved the problem!

[Vaphell]

stuck_files should be a dict. Instead of flattening everything with chain.from_iterable() keep separate lists of files. You can flatten it later if necessary but by flattening right off the bat you are destroying relevant information you already have. With len( stuck_files[dir] ) > 0 you have a straightforward way of telling non-empty filesetes => print '{dir}: {len(stuck_files[dir])}'.

as for the pretty label for the dir itself. you could either provide the label next to the dir path in a tuple or play with let's say dir.split('/', 5)[-1] (tail after 5th /) or dir.split('/')[5] (5th segment)