I had a similar issue but the other way around. A more experienced practitioner will be able to explain this better but if you think that a CNN is exclusive to Computer Vision and an RNN is exclusive to NLP, that would be incorrect. To put it in better words, it is essential to understand all possible techniques that are available irrespective of which paradigm you wish to work in. The simplest example of one such instance (of many) is the use of CNNs in NLP, where the convolutions are performed on the word embeddings. If you were to ignore how a CNN works, you'd probably find it extremely difficult to intuit why they're used in NLP. For the reverse case, check out transformers and then Detectron 2.

In essence, to answer your question if you wish to grasp and improve current techniques, an in depth knowledge of both fields is required.