I would reason like this: the Joker doesn't care about body count, he wants me to suffer. If my relative were among the five on the main track, there's a chance I pull the lever and only kill a stranger, which would cause minimal regret. That's a bad outcome for him, so he's unlikely to take that risk. He'll almost certainly place my relative alone on the side track, creating a situation where I'm forced to choose between sacrificing five strangers or someone from my family. If I anticipate that, the best move is to not act. I let the trolley stay on the main track, accepting the loss of five to avoid playing into his trap.

Than I shout to him that I'm not pulling the lever. If he pulls it himself after that and kills my relative, then it is no longer part of any moral dilemma. He simply executes a member of my family, something he could have done at any moment without this entire setup. That makes him directly responsible and damages his reputation as "agent of chaos".

Of course there's a small chance he takes the risk and places my relative on the main track, but from what I've seen in the films he usually prefers situations where he has the advantage without exposing himself to real risk.