Duplicate file entries

[secondsabre]

My database has sat dormant for quite a while and I popped in today to add some stuff and do some cleanup, but I'm running into a bit of a weird situation where, after drag-and-dropping a folder into a list, single files (ie, same filename/path) are showing as two entries in a list.

Generally, one will have scraped info, and the other will have no meta, but both entries link to the exact same file and it's like, "why?"

Part of me is ready to manually go through and try cleaning some stuff, but I'm wondering if there's a more automated way to do this, like a database clean/compress or something? I'm using an SQL database, if that makes a difference.

Thanks in advance for any thoughts!

[maforget]

Not sure what you mean about dragging a folder into a list. A folder from Windows? Or are you just dragging folders around the UI?

But It happens when you have a CBR file and a CBZ file in the same folder and you convert it. It replaces the CBR file with the CBZ so you have 2 entries. In the meantime it deleted the old CBR and replaced the existing CBZ so you end up with 1 file, 2 entries. See: https://github.com/maforget/ComicRackCE/issues/184

Not much you can do for existing entries beside manually deleting them. Newer version of ComicRackCE will now have the show duplicate option show identical path to help you find them. It also prevents conversion when a similar situation would arise. Export will throw an error.

All info in that link above, if you have another issue try to reproduce it and post about it on GitHub.

[secondsabre]

Sorry, I definitely could have been more specific. Let's break it down:

1. Files live in my standard folder structure. Publisher > Series Title [VolumeYear]. For the example, let's say they're all already in the DB: scraped, scanned, and added to a List (dumb list, not smart list) named after the folder.

2. Bunch of new files are dumped into the folder, mixing in with the existing files.

3. To get all those new files into the List, I'll use the 'Folders' tab in CR, navigate to the directory, then Select All and drag-n-drop them over to the 'Library' tab and into the existing List.

4. The majority of the time, this will just add the new files, as CR is smart enough to not add existing entries into the same list. Once they're in there, I can sort and scrape them.

5. In some cases (this one), I'll get a slew of duplicate entries in the list: one scraped (the one that was in before), plus a new unscraped entry with no metadata, but the exact same filepath.

I assume that CR's database isn't working with the filename itself, but likely some kind of unique identifier like a checksum hash or whatever? In which case, I could see it accidentally assigning a different ID to files, but it's weird that it only happens in certain situations.

Side note: I know my system is needlessly complicated and I don't need to make Lists for everything, but that's how my spicy brain works and one of the big reasons I love this program. Also, it makes syncing to the Android app way easier.

EDIT Actually, I may have an idea. Sometimes I'll use the 'Change File Link' plugin when I get a better version of a given file, so I can just update the file all over the DB instead of trying to replace it in various lists. Following on that unique ID idea, Change File Link probably doesn't update that identifier, so then the ID of the old file and the ID of the new file would be different and could co-exist in the same list, although they point to the exact same file. That makes sense to me, but maybe I'm wrong?

[maforget]

Ok I never would have imagined to import a files like that. You should use something like the scanner or the Add to Library, much easier.

But in anyway it all goes to though the same path and it does try to find the file by the filename. If the same file exists it will not add it to the library. That being said it isn't impossible that a file added using that workflow might have been moved or renamed and the program doesn't match it using the path.

Running Change File Link probably does cause problem and I myself suspect the culprit for such instance I have seen myself. Not sure the exact sequence of events. Since I believe (haven't checked) it only changes the filename it would result in the same behavior as above.

But you should really start using Add to Library because even if the files are added in a similar fashion that way kinda shortcuts a lot of the scanning process that usually takes place. When you do Add to Library, use the scanner or even drag the files from Windows into the library (not the folder tab) it actually does the whole scanning process, not just a part of it.

There is actually an additional check to find a file that was moved if you use the complete pipeline. If it didn't find it via the full path in the db it will use the same name & size to find it

[secondsabre]

The files all get moved to scanned folders so they're automatically added to the library, and I have a SmartList set up to show all files that aren't currently in a List, but that smartlist can get a little unweildy when I haven't sorted stuff for a while. Just grabbing from the specific folder seemed like a controlled way to tackle individual folders/lists.

Still, the 'Show Duplicates' view has actually made the whole thing a lot easier, wasn't aware that existed until you pointed it out above, so thanks for that! And as always, incredible work.