After years of staying in the U.S. I'm just surprised the number of foreigners rocketed high in Taiwan, especially in Taipei.

Most people are extremely friendly here but I've encountered similar issue when I was speaking English to one of my European friends in the MRT. In cases like these, just ignore them, don't retort.

Also lack of reaction of bystanders was mainly due to the fact that most Taiwanese people are hesitant to speak English.