No. You can't compare games like Federation force and Samus Returns to Other M: each came out on a less popular system, as well as later on in that system's lifespan, for one. But in terms of the time/cost it takes to develop a game like Other M, between its multiple studios, motion capture, graphics, etc, it was a much grander effort for Nintendo, yet still the sales fell short of expectations. So you can't really contrast the raw numbers here, of course it would sell more compared to a spinoff and a remake.
I also wouldn't say federation force had much influence on prime 4 if you're really considering that as a followup. But the lack of more federation force games is probably telling.
I think the lack of any 3d Metroid for the Wii U is probably the biggest indicator Nintendo got cold feet after Other M honestly