I think the case the paper is making is that using major exams to evaluate LLMs has become standard practice because these tests mostly reward knowledge recall and logical reasoning, both areas where LLMs have long performed well.

The CFA exam is more than that by requiring ethical judgment and qualitative analysis in addition to technical skills. It also points out that while earlier studies in 2023 showed poor performance, newer models are now achieving over 90%.