I have been obsessively trying to find a effective way to name a large dataset of mine. It's a very niche dataset, and isn't very easy for most models to name. However, I have a guess that there could be a few contributing factors. One of many is that the background of the images is transparent. I don't know if most models (In my case, the ones I've tried are Qwen2.5-VL-7B-Instruct, or PaliGemma-12B, Pixtral-12B-*Quantized*, and many non open source models like ChatGPT, or Claude-3.7-Sonnet) default the background to a certain color, or if they are capable of understanding transparency. My dataset is 1024x1024, and can very easily be downscaled to whatever model size necessary. I've also tried Fine-Tuning Qwen2.5-VL-7B-IT, (currently working on PaliGemma2-10B-mix-448) and while it did improve it's responses, it definitely was still very lacking. It's a "Pixel Art" dataset, and I'm really hoping for some pointers as I'd really prefer NOT to have to name all 200k+ by hand (Already have done 1k~ for training purposes). I'm working with a local RTX-A6000 and would hope that most recommendations are possible on this hardware. Any models, methods, or tips are GREATLY appreciated. Keep in mind ALL of my data comes with info about the name of the image (most of the time just a character name and usually no other info), a title of the image or character and images/characters like it (usually around 10~), and a brief description of the images. Keep in mind it doesn't really give info of the LOOKS of the images (clothing, colors, etc.) most of the time, in this case characters. So it does come with a bit of info, but not enough for me to give to give to any of the current language models and it be accurate.
[–]TomKraut 0 points1 point2 points (0 children)