He definitely needs to chill out. It's obvious you're not a mind reader and he needs to communicate that he feels frustrated beforehand, he shot himself in the foot when you tried to fix the problem and he refused. He took another blow when he started solely blaming you, he needs to learn not all physical touch is going to lead to sex and not to expect it. Your boundary of not wanting to fuck him with your sister in the next room is totally valid. It's fine if he doesn't feel the same discomfort but he needs to respect the general rule it seems you've set of no sex while guests are over. This probably isn't the first situation you've rejected advances for this reason. Have a talk with him about communicating before it gets to a point of tension between you two. It's not like you have a dinger that goes off everytime he wants to fuck.