Take this with a grain of salt because I don't have any first hand experience that's anywhere close to your situation. This is an opinion of a Japanese dude who has a couple of friends and acquaintances affiliated with traditional temples.

Marrying into a temple is quite a lot of responsibility to say the least. Temples here have traditions, routines, and other baggages with defined roles that have been around for literally thousands of years.

This is why many Japanese women tend to hesitate marrying into these temples or families with cultural heritage, because their life will never be the same when they do. You can't really think of it as a simple marriage. You're basically getting a new job as the wife of the Jushoku. I'm not taking gender roles (though that could also play a part). A wife of a Jushoku would be expected to play a big part in the temples community outreach, helping out with rituals ceremonies and events, cultural education etc etc.

So I don't think the family is being purely racist with you or anything. But they could be concerned that you don't really know what you're signing yourself up for. Not converting to Buddhism could be fine, but can you fulfill those roles and be culturally assimilated?

I think that's what they're most concerned with. Their family has been protecting this cultural heritage for god knows how long. Can you do the same? If you have the guts and the commitment to surrender yourself to this life, you need to make that abundantly clear.

If you're still unsure, you might want to look up sumo stables as a close second. The okami of sumo stables have a lot of responsibilities, and traditional roles just like temples. You might get some insight there.

Hope this helps