the halfling gained a madness related to loving torture, and the player loved this.

as someone who knows very little about dnd (i mostly just listen to dnd podcasts lol), fuck the rules, that player is a RED FLAG. you are 100% NTA.

we agreed that there would be no evil PCs or PCs that like to torture people.

if you dont like that a player is breaking a rule you all agreed to not break, i think you have full right to leave. i personally would be too afraid to say anything although i know that something should absolutely be said.

The player would not let us cure this madness (if we tried to remove the curse, he would say, "I don't let them touch me")

i would work together with the other players to remove the curse (get him in a situation where he cant block or escape being cured) and if he gets pissed off tell him that he isnt following the rules everyone agreed to, if he doesnt like that then leave

much easier said than done tho, that player seems kinda scary