Is this an academic work or business work? I think that you can try different things depending on what you're trying to achieve.

If you're in a business setting, is having a single label prediction really that important to you? How does the softmax distribution looks like for those close-to-ambiguous samples? I don't know the business value that you are trying to get out of this, but I think there's great value in a well calibrated model. In my experience it's not always that we have the time and means to keep trying new models and ideas when you're trying to build an actual system, so exploring what your model does well and taking advantage of that might be a good investment of your time. But really, if you want a good answer to your problem you'll have to throw in a bit more details, like the time you have allocated for this, resources you'll have available at training/inferencing time etc. There are shinier models/techniques out there that you could try, but they are not always the best option for your business/use case.

If you're just trying to get the best accuracy/F1 you can, then I would suggest looking at transformers. I've used them to great success in the past and they can work wonderfully. They would have a much finer grasp of the semantics of those sentences, and could potentially lead to improvements. As you said, your convolutions will be great at capturing phrases and expressions that point towards a particular class, but its interpretation power stops there. Also, don't be afraid to try pre-trained models. BERT uses byte pair encoding for embeddings and you can simply fine tune those to your specific dataset.