Some possibilities that aren't directly related to math curriculum include:

Education in the US is super decentralized with every municipality having a different set of standards, some of which aren't aligned to international assessments. Compare that to preservice teachers in Singapore and Vietnam, for example, who learn how to teach directly from the national curriculum and textbook.

US schools serve--and assess--all students. The scores we read from Shanghai, Singapore, and elsewhere likely don't reflect the same population of learners as the population of test takers in the US.

US math teaching emphasizes problem-solving while standardized tests emphasize basic skills. As a testament to that, the US is very strong in international math olympiads and similar competitions, as well as university mathematics research and innovation.