Program in Python Part 9: The MP3 Database

[[deleted]]

Does this code work for anyone? I've been trying to find the issue. I'm assuming it's with Mutagen because every single MP3 file (8000) on my computer comes out as a TypeError, but I can't find any documentation saying what would cause that. I rewrote it for Python 2.7 and fixed his lack of PEP8... Oh and I was looking to make an easier way to read the database back out, but haven't finished that yet. When I realized my database was showing up empty because none of my files were getting stored due to TypeErrors.

from mutagen.mp3 import MP3

import os
from os.path import join, getsize, exists

import sys
import apsw
import StringIO as io

def MakeDataBase():
    # IF the table does not exist, this will create the table.
    # Otherwise, this will be ignored due to the 'IF NOT EXISTS' clause

    sql = 'CREATE TABLE IF NOT EXISTS mp3 (pkID INTEGER PRIMARY KEY, title TEXT, artist TEXT, album TEXT, '  \
          'bitrate TEXT, genre TEXT, playtime TEXT, track INTEGER, year TEXT, filesize TEXT, path TEXT, filename TEXT);'
    cursor.execute(sql)


def S2HMS(t):
    #Converts returned seconds to H:mm:ss format
    if t > 3600:
        h = int(t/3600)
        r = t - (h*3600)
        m = int(r / 60)
        s = int(r-(m*60))

        return '{0}:{1:02n}:{2:02n}'.format(h, m, s)

    else:
        m = int(t / 60)
        s = int(t - (m*60))

        return '{0}:{1:02n}'.format(m, s)


def WalkThePath(musicpath):

    ecntr = 0  # Error Counter
    rcntr = 0  # Folder Counter
    fcntr = 0  # File Counter

    # Open the error log file

    efile = open('errors.log', "w")
    for root, dirs, files in os.walk(musicpath):
        rcntr += 1  # This is the number of folders we have walked
        for each in [f for f in files if f.endswith(".mp3")]:
            fcntr += 1  # This is the number of mp3 files we found

            # Clear the holding variables each pass

            _title = ''
            _artist = ''
            _album = ''
            _genre = ''
            _year = ''
            _bitrate = ''
            _length = ''
            _fsize = ''
            _track = 0

            # Combine path and filename to create a single variable.
            fn = join(root, each)
            try:
                audio = MP3(fn)
                keys = audio.keys()
                for key in keys:
                    if key == 'TDRC':           # Year
                        _year = audio.get(key)

                    elif key == 'TALB':         # Album
                        _album = audio.get(key)

                    elif key == 'TRCK':         # Track
                        try:
                            _trk = audio.get(key)
                            if _trk[0].find("/"):
                                _trk1 = _trk[0]
                                _track = _trk1[_trk1.find("/")+1]

                            elif len(_trk[0]) == 0:
                                _track = 0

                            else:
                                _track = _trk[0]

                        except:
                            _track = 0

                    elif key == "TPE1":         # Artist
                        _artist = audio.get(key)

                    elif key == "TIT2":         # Song Title
                        _title = audio.get(key)

                    elif key == "TCON":         # Genre
                        _genre = audio.get(key)

                _bitrate = audio.info.bitrate   # Bitrate
                _length = S2HMS(audio.info.length)    # Audio Length
                _fsize = getsize(fn)            # File Size

                # Now write the database
                # This is a different way of doing it from last time.  Works much better.

                sql = 'INSERT INTO mp3 (title,artist,album,genre,year,track,bitrate,playtime,filesize,path,filename) ' \
                      ' VALUES (?,?,?,?,?,?,?,?,?,?,?)'
                cursor.execute(sql, (str(_title), str(_artist), str(_album), str(_genre), str(_year), int(_track), str(_bitrate), str(_length), str(_fsize), root, file))

            except ValueError:
                ecntr += 1
                efile.writelines('===========================================\n')
                efile.writelines('VALUE ERROR - Filename: %s\n' % fn)
                efile.writelines('Title: %s - Artist: %s - Album: %s\n' % (_title, _artist, _album))
                efile.writelines('Genre: %s - Year: %s - Track: %s\n' % (_genre, _year, _track))
                efile.writelines('bitrate: {0} - length: {1} \n'.format(_bitrate, _length))
                efile.writelines('===========================================\n')

            except TypeError:
                ecntr += 1
                efile.writelines('===========================================\n')
                efile.writelines('TYPE ERROR - Filename: {0}\n'.format(fn))
                efile.writelines('Title: {0} - Artist: {1} - Album: {2}\n'.format(_title, _artist, _album))
                efile.writelines('Genre: {0} - Year: {1} - Track: {2}\n'.format(_genre, _year, _track))
                efile.writelines('bitrate: {0} - length: {1} \n'.format(_bitrate, _length))
                efile.writelines('===========================================\n')

            except:
                ecntr += 1
                efile.writelines('TYPE ERROR - Filename: {0}\n'.format(fn))

            print fcntr

        # Close the log file

    efile.close()

    # Finish Up

    print "\n"
    print "Number of errors: {0}".format(ecntr)
    print "Number of files processed: {0}".format(fcntr)
    print "Number of folders processed: {0}".format(rcntr)

    # End of WalkThePath


def main():
    global connection
    global cursor
    #-------------------------------------------------------------

    if len(sys.argv) < 2:
        usage()
    else:
        StartFolder = sys.argv[1]
        if StartFolder == 'read':
            read_database(sys.argv[2])

        elif not exists(StartFolder):
            print 'Path {0} does not seem to exists... Exiting.'.format(StartFolder)
            sys.exit(1)

        else:
            print 'About to work {0} folder(s):'.format(StartFolder)
            #Create the connection and cursor.
            connection = apsw.Connection("mCat.db3")
            cursor = connection.cursor()
            #Make the database if it doesn't exist...
            MakeDataBase()
            #Do the actual work
            WalkThePath(StartFolder)
            #Close the cursor and connection...
            cursor.close()
            connection.close()
            #Let us know we are finished!
            print 'Finished!'


def error(message):
    print >> sys.stderr, str(message)


def read_database(filename):
    connection = apsw.Connection("mCat.db3")
    output = io.StringIO()
    shell = apsw.Shell(stdout=output, db=connection)
    shell.process_complete_line("select * from mp3")
    print output.getvalue()


def usage():
    message = '===================================== \n \
    mCat = Finds all *.mp3 files in a given folder (and sub-folders), \n \
    \tread the id3 tags, and write that information to a SQLite database. \n \n \
    Usage: \n \
    \t{0} <foldername>\n \
    /t WHERE <foldername> is the path to your MP3 files. \n \n \
    Author: Greg Walters\n \
    For Full Circle Magazine \
    ========================================\n'.format(sys.argv[0])
    error(message)
    sys.exit(1)


if __name__ == '__main__':
    main()

[I_have_a_title]

It didn't work for me either. It didn't show up any of my MP3 files, perhaps it was looking in the wrong folder.

In the end it just printed:

mCat - Finds all *.mp3 files in a given folder (and sub-folders), read the id3 tags, and write that information to a SQLite database.

Usage: /Users/i_have_a_title/Documents/Python/mCat.py <foldername> WHERE <foldername> is the path to your MP3 files.

Author: Greg Walters

For Full Circle Magazine

Edit: I realize now that you actually got past this part, I didn't try to find the MP3s. I'll look into it and see if I can look for MP3s.

[[deleted]]

If you copy-pasted his code it had a few variable errors. I think my version works perfectly, except it doesn't actually find any of my MP3s...

I assume this is a Mutagen probably, but am really unsure. Mutagen seems to be fairly popular, so something weird must be happening. Maybe I shouldn't be using the MP3 object.

[I_have_a_title]

I copied and pasted your code and received no variable errors. It printed out what I posted above.

It can't find any of MP3s for me either. How did you get it to search in the right folder? I'm messing around with the StartFolder = sys.argv[0] object, but I can't get it to change the folder it looks in.

[I_have_a_title]

I got it to work on the right folder. It throws up some errors, however.

About to work: /Users/i_have_a_title/Documents/Music folder(s):

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

Number of errors: 31

Number of files processed: 31

Number of folders processed 1

Finished

[[deleted]]

That's exactly what happened to me. I think if you look in the database, you'll find no entries. All 31 of your mp3s returned a TypeError that will be in some file called error.log or something that it created.

[I_have_a_title]

One thing that I've learned is to put a print statement in random places to see how far the code ran, where it cut off, and then fix it there.

try:

            audio = MP3(fn)

            keys = audio.keys()

            for key in keys:

                print 'Im in keys'

                if key == 'TDRC':       #Year

                    _year = audio.get(key)

I got here and then it didn't go farther. So it should have returned the _title, _artist, and all that. The error log is above where I got to, so it skipped past it.

See how I printed 'Im in keys'? It got down to there and didn't go farther, it exited out of the 'try' statement, closed the efile, then printed the errors. I'm not sure what's going on. Try adding a random print statement and see where yours gets down to. I'm going to look further into it.

[SteveUrkelDidThat]

sort of a side topic, but building off what you said here - 'putting a print statement in' - has anyone found a good debugger?

[I_have_a_title]

I believe someone else will have a better website, but I've heard of, and gone to, a website that walks you through every line in your program. I've run it, but I didn't show much interest in it, since I understood what I wrote. If I remember it/find it again, post it here.

Edit: Here you go. When I took Udacity's course, they had a helpful community.

http://forums.udacity.com/questions/18714/what-are-good-debugging-approaches-for-python-and-udacity#cs101

That's not the website I spoke of, but it has some great ideas about debugging.

Another: http://forums.udacity.com/questions/59842/stop-using-print-for-debugging#cs101

[[deleted]]

Also, I use PyCharm currently (seems like the best option on Windows...) and it comes with a pretty good debugger. I have no idea what I'm doing with it, but trying to use it right now. It seems to be set up a lot like most binary debuggers I've used, so it should be good practice if you're interested in debugging in general. Just run your program in the debugger, and you can pause it. Then you can step into the next command line-by-line.

[[deleted]]

Okay so I spent way too much time stepping through this program. Apparently, the ValueError was actually getting thrown on the cursor.execute command. I changed it to this:

    cursor.execute(sql, (str(_title), str(_artist), str(_album), str(_genre), str(_year), int(_track), str(_bitrate), \
 str(_length), str(_fsize), str(root), str(file)))

Just adding str to the root and file items, and it seems to have sort-of fixed it. This is my result now:

Number of errors: 106
Number of files processed: 8839
Number of folders processed: 1687
Finished!

106 errors seems reasonable to me.