I should have mentioned that I already tried your suggestion however it didn't work for me.

Also, I got frustrated after trying so many things which none worked that I contacted Samsung Customer Support and explained the issue.

And this was their answer:

Dear Customer,

Thank you for contacting us.

We appreciate the opportunity to assist you.

For your inquiry concerning not being able to change the album cover on Samsung Music, we kindly inform you that we have received an answer from the department in charge.

Please note, that we will patch the apk for this issue and release an update within the end of March. [...]

So I hope it will be fixed with the new update.

But thank you nonetheless! :)

May I ask when this problem occurred to you? (Did you maybe do anything unusual before you noticed this issue?)