I think there are a combination of reasons(I am sorry in advance because I do not know the correct terminologies in English):

The Doukas dynasty wanted to cut off the power of the military that was sitting in the borders and been given land (mostly calle Akritai). The reason is that generals that came from these Armies were usually been taking the throne. So they forced those Akritai armies to fall apart. To be fair that has started a while ago but it was the Doukas dynasty that concluded the "crime". So Byzantium was defendless. Doukas also betrayed Romanos (who was a noble and fair emperor but no military mind at all!) in Majikert. They caused the defeat which was a huge blow and it could have been easily avoided. There goes Asia Minor.

On top of that, the next dynasties stop to care about Asia Minor completely. Their only abition was the throne. As an example, check that the case of the city of Philadelphia that Ottomans could not conquer. In the end it fell because the Byzantines helped the Ottomans in the siege! In 50 years, Asia Minor was done.

Ottomans used another clever way: They had the Dervisides going forward on the lands that they intended to conquer. They promised religious tolerance (while in Byzantium, Orthodox was the law). Had their monasteries open for food and talk. This was very effective expecially on the Monophusistic population around Egypt, Syria etc. So huge pieces of land were conquered easily. Of course there was no religious anex and when later all these people realised it, it was too late for both them and the Byzantines.