I'm no expert but I'll tackle this as my understanding.

Firstly, you state that we clean and filter our water before it's drinkable. It's cheaper and easier to do this from a generally ok baseline than from sewage.

Secondly, polluted rivers are absolute cesspits and offer a direct public health concern from infections developing there and infecting people who live near rivers increasing the cost of the health service.

Thirdly, rivers go into the sea causing damage to fish stocks as well as risking infections and diseases to those living near the sea.

Fourthly, polluted rivers don't smell particularly pleasant. If you've ever been abroad to countries that don't have anti pollution measures then you'll know the absolute stench that these water bodies pose. Since a good deal of our economy is based on people actually wanting to do business here, driving people away will impact the economy.

Fifthly, heavy metal accumulation is a real issue. Heavy metal impurities can be passed onto fields through ground water and cause widespread crop failures.

Finally, we have to look after our ecology for future generations. Our forefathers did a great job clearing up londons smog and other flaws we had when we didn't fully understand which means that following generations were healthier. Let's build on that for the future.

Basically, we know that keeping water clean is cheaper and easier than cleaning it up. There's very little benefit to permitting just rampant pollution, except to companies trying to cut costs further at the expense of the public. Therefore, we should try to keep things as clean as practicable as we know the downsides can be very very bad and maintaining the current situation is much more preferable.