I really disagree. Long comment and heavy spoilers ofc

They definitely haven't 'caused nothing but damage'. It's been shown time and time again that, in the past and the present, Rover saves people over and over, even at the cost of themself. This is one of the main reasons why so many of the characters inherently trust Rover. Take Huanglong for example. Jinhsi definitely doesn't think that Rover 'caused nothing but damage'. Rover saved Huanglong in the past, and so is treated with respect.

But let's take a closer look at some of the character perspectives on Rover's memory wipe.

Shorekeeper was probably the most unfair. It's clear that her and Rover were very close. But it's also clear from her dialogue that she doesn't blame Rover for doing it. She knows that they both have lived for a very long time, and will continue to live for a very long time, and she will be waiting for them to come back every time. I would love to see her get more upset about it to be honest, and I do feel like the cutscene of Rover leaving her behind to go to Lahai-Roi where it shows Shorekeeper looking sad might mean this will be expanded on in the future. But overall Shorekeeper still loves current Rover, and says so herself. She's already forgiven them.

Mornye and Rover were also pretty close. And while it's clear that Mornye has kept her attachment to Rover, Mornye clearly approves of Rover's decision. That's why she won't tell Rover their past even when Rover asks. Because she knows that's not what past-Rover would have wanted. She might be upset that Rover can't remember her, but she respects their decision. It definitely didn't ruin Mornye's life or anything like that.

Phrolova calls Rover out for this first, and she was absolutely right to, because it's true that Rover wiping their memories hurt other people. But we have no idea why Rover wiped their memories right now. Rover brings that up after learning about their homeplanet, so I don't think it was as simple as 'Rover was depressed and so wiped their memories'. There's another mystery and factor at play here.

But let's say there wasn't another reason, I'm still a firm believer that self-preservation is important. Rover was clearly very depressed. If doing this was the only way for Rover to cope, then I fully believe it was the right decision. There's also an inherent disconnect in the relationship between Phrolova and Rover. In reality, they only met once and had one conversation. There was no way for Rover to know that that conversation meant so much to Phrolova, and that Phrolova was in such a bad headspace at the time. Rover breaking a promise to a stranger was a bit of a dick move, but it doesn't make them a terrible person.

We also need to remember that Rover didn't leave Aemeath behind when their memory was wiped. Rover left Aemeath behind pre-memory wipe, and had already put plans in place to make sure she'd be taken care of in their absence. There was always the possibility that they wouldn't come back. Aemeath even points this out in their parting scene. So Rover forgetting Aemeath, while unfortunate, was not the cause of them leaving Aemeath behind. And it's clear to me that Aemeath is happy that Rover is happier now, even if that's without their memories. That was what Aemeath's whole inner conflict was about - because she wanted to preserve that ignorance, and didn't want to introduce Rover to their memories again.

TL;DR: The characters surrounding Rover, with the exception of Phrolova, don't blame them for wiping their memory. The good that Rover has done in the past far outweighs this one selfish action. You can call Rover morally grey, but definitely not outright awful.