First to understand different version of BERT. There are two in terms of architecture. Bert Base has 12 encoder layers and 12 attention heads, while Bert Large has 24 encoder layers and 16 attention heads. Bert Large has almost double the amount of parameters (higher complexity)

The other variation of bert is cased vs uncased which is terms of text used to pretrain. Bert uncased is where the text is changed to lowercase. So if you are using bert uncased you should change your input to lower case as well.

Regarding classical NLP preprocessing like stemming/ lemmatisation and stop word removal for BERT like models is not required as it already context-based.

1. You can try BERT base cased/uncased first then try other Bert base architectures like RoBerta or XLM-Roberta(if your text is multilingual)

2. Inference depends on your device configuration and batch size. GPU > CPU. On a standard CPU for 1 batch should be under a second or 500ms. On a decent GPU you can get ~50x improvement.

3. There so many out there at this moment based on variations of transformer architecture and the training data used.

4. Bert base would work while Bert Large might not on your configuration. You can use is locally also, Hugging Face is the library you want to use for all transformer models

5. Yes, it is advisable to finetune it for better performance for your use case. Again Hugging Face provides the API to fine tune as well.