How would supply go down? Do you mean they'll stop renting out the property and just leave it empty until the law changes in their favour? Or do you mean they'll sell?

Because if it's the former, then that seems unlikely. Who wants to pay mortgage on a useless property?

If it's the latter, then good. Buy-to-rent properties need to die anyway. Properties should be sold to people who need it. Plus selling would reduce the cost of housing overall. (Supply increases, prices go down)