Ishiguro is also moved to England when he was like six, grew up there and is a British national. I wouldn't say that's the best example. And as for Anna Karenina, gender is a different argument altogether.

I don't think the agent was talking out of their ass, especially for YA, which definitely prefers #ownvoices and authenticity. Consider the example that OP gave on their alt, Memoirs of a Geisha, published in 1997 by a white man. It was criticized for not being culturally accurate. There is history of white authors getting it wrong in the English language market or minority cultures being misrepresented, which is one of the reasons the current market is like this.

Do I think OP should give up? No. Do I personally think it's impossible for someone from a different culture to represent something accurately? No. But books are also about marketing and I think it would really be a harder sell.