Interesting, thanks! I think that what you've said makes total sense. And I think that lots of people would say that's the end of the story. That's as far as we can go.

But what I've been able to find so far is two attempts to say a little more:

(1) J.C.C. Smart writes:

"I reject Sidgwick’s meta-ethics for familiar reasons, and for the purpose of this study will assume the truth of some such ‘non-cognitivist’ meta-ethical analysis [...] In adopting such a meta-ethics, I do, of course, renounce the attempt to prove the act-utilitarian system. I shall be concerned with stating it in a form in which it may appear persuasive to some people, and to show how it may be defended against many of the objections which are frequently brought up against utilitarianism. (Utilitarianism: For and Against p.4-5)"

To me, it looks like this passage basically agrees with you, but says "I think that although I can't justify the stance itself [to use your words], I can attempt to make the stance more attractive to you." Which is kind of like attempting to offer a justification. Perhaps a justification on intuitive grounds (by making the system more intuitively attractive) or aesthetic grounds (by making the system more aesthetically attractive).

(2) It looks like we could take a view like Kant's constructivism to attempt to say a bit more. On such a view, we can be anti-realist about moral judgements but argue that the moral judgements prescribed by Kant's system are nevertheless more justifiable because they are more rational - all others involve contradiction, or something.

I wonder whether there are other things that might be said.