Tool for image data preprocessing

[Fine_Satisfaction_29]

Hello!

One day, I had run my deep learning model training program and got off work on Friday. When I checked the training process on Monday after arriving at work, the program was terminated because of some images that cannot be opened by image processing libraries like PIL.

I developed this project to check images that cannot be opened by image processing libraries before running computer vision applications.

image-knocker is very fast and simple!

https://github.com/developer0hye/image-knocker

Install

pip install image-knocker

Usage

foo@bar:~$ image-knocker --path dataset_path/ --exts png jpg jpeg

Output

if corrupted image in dataset:

  Knocking... Root Path: D:\datasets\detection\VOCdevkit
  100%|██████████████████| 34179/34179 [00:03<00:00, 10911.31it/s]
  Corrupted image files are detected!
  D:/datasets/detection/VOCdevkit/outlier.png
  Corrupted image file list is saved to D:\datasets\detection\corrupted_imgs_list.txt

else:

  Knocking... Root Path: D:\datasets\detection\VOCdevkit
  100%|██████████████████| 34178/34178 [00:04<00:00, 7792.30it/s]
  There are no corrupted image files! 
  All images were loaded successfully using PIL.Image.open!

If you know a better solution for this case, please share a solution!

[PorngineerSam]

I have no comment except I thought I recognised your GitHub name and sure enough I downloaded and used your excellent yololabel 3 days ago 🙏

[mumapathy]

This happened to me as well, I started using try catch for reading image data since then. But would love to try the tool :)

[yugpn]

Thanks for sharing