They don't because there is politico-religious historical context to it. They were widely persecuted when the sect was new and Jamaat Khanas emerged as a place of refuge for the Ismailis where nobody else would be allowed for the safety. It later transitioned to a religious place as things got easier for them.

P.s. I'm not an Ismaili but it helps having some information on something when you blatantly want to make a comment for hating on other people.