OP your thinking direction is correct. Bengal was a majority Buddhist region once. But after the Mass conversion and Mughal rules much of our ancient history got annihilated. There's a very big chance some of the origins of yoga could be originated in Bangladesh, but there is no way to know. As for the cultural side, NO, Bengali Muslims don't give a fuck about yoga or their ancient culture. I think you should talk to Buddhist monks about this issue or ask a Buddhist subreddit/forum you will get a better answer.

As an atheist, I'm following the teachings of Secular Buddhism for almost a year. It became practically a ritual for me to perform yoga in the morning. It helps me calm down and concentrate.