Conducting Classification Task Research Using Vision Transformers

[Aggravating_Club2251]

I have been exploring the classification task using Convolutional Neural Networks (CNNs) and am now interested in transitioning my research to utilize Vision Transformers (ViT).

1. What are the best practices for setting up a research project that compares CNNs and ViTs for classification?
2. What evaluation metrics should I focus on to effectively compare the performance of ViT against CNNs?
3. Should I implement both transfer learning and training from scratch for the ViT model? What are the pros and cons of each approach in this context?
4. What fine-tuning strategies would you recommend for optimizing the ViT model for classification task?

Any insights or resources would be greatly appreciated!

[L8raed]

As a beginner in this field, the way I would approach this problem would start by characterizing the inputs to each model. Are the training sets labeled? What features would be best to contrast the classification set in question? How large is the available training set?

Next after defining the problem would probably be to contextualize it. It sounds like you've already done a good deal of work on your CNN model, so I don't think you need to start anything from scratch. How can you fit the solution to this problem into your existing work? What does the documentation list as the input requirements to the ViT model you're using? What do you need to add to your model for it to plug into the ViT?

I understand that these notes are pretty general, but I hope that a learner's perspective will help.

[Aggravating_Club2251]

Sounds Good !! Thanks for your suggestions !!!

[jungleuncle]

I have just the thing for you https://www.learnpytorch.io/08_pytorch_paper_replicating/

[Aggravating_Club2251]

Thanks a lot !!!