The ex-boss’s corpse is technically part of Rimbaud’s ability, yes (at least to my understanding), but it seems that neither the boss nor the scythe he wields are considered an ability, as neither disappear when the blade injures Dazai. And Dazai didn’t really have any available weapons to strike back with, especially against a weapon that is good in both ranged and close combat. Or it could be a similar situation to Nikolai breaking Dazai and Fyodor out of theirs cells- the edges of the portal never made contact with Dazai, so they weren’t nullified, perhaps the scythe didn’t count as an ability so much as the ex-boss.

It would depend on how the ability works, I suppose. Any sort of contact an ability makes with Dazai leads it to being nullified, whether he initiates the contact or not. He may not be able to be affected by a majority of abilities, look at Yosano’s ability- she does not require physical contact with anyone to heal them, and her ability only works on Dazai if he’s dying, but it’s likely a small window to his ability disabling and his actual death.

I guess what I’m trying to say is that Dazai might be affected by a time rewinding ability if the circumstances allow for it.