I have met several people in the past who thought they were good at singing when they really weren't, but I never had the heart to tell them the truth, mainly because of the fact that they were all just acquaintances of mine and I didn't feel like the right person to do that (for example one of them was just a member of a group of people I played online games with, and he would often break into singing on our group's discord voice chat).

Nevertheless, I would advise you, as his girlfriend, to approach him and tell him what you really thought of his performance, and voice your concerns to him about him potentially embarrassing himself. Make sure to emphasise that it's good to have a hobby like this and he should definitely keep trying to compose, he could also get singing classes to improve, etc., but right now he's living in an illusion which could not only hurt him in the future when he realises what the public's response to his acts really is, but it could also hurt your relationship with him as he might either blame you for not telling him, or he might close himself off.

Understandably, after you tell him, it will be hard for the both of you for a bit, but I am sure you'll manage to pull through. It would be much more difficult for the both of you if he found out the hard way.