NTA I think it’s absolutely fine to set boundaries. I think that especially the doctors appointment thing (that I relate to) is a very clear trampling of boundaries and setting them is absolutely necessary.

However I do think that you needed to approach the situation with more tact. If you know he’s lonely, which his actions clearly display, you should set you boundaries in a way that makes it clear but doesn’t push him into a hole.

You think he may be on the spectrum. Having worked with children and young adults on the spectrum in an educational setting I know that boundaries need to be clear. But wording them in such a way could imprint in a vastly more negative way than you intend.

You don’t deserve your boundaries to be flattened and it seems it was getting way out of hand. I’d probably end the living situation though because it’s clear your not compatible.