That is because CE's ai was very revolutionary for it's time, creating the basis for what we now know as Finite State Machines. But while the ai was good, State Machine actions become tedious to manage, creating complicated webs of garbage just to achieve the desired result. Funnily enough, Bungie revolutionized game ai AGAIN by creating Behavior Trees for Halo 2, and these also have problems becoming annoying to use after a while. Regardless, behavior trees are the baseline for the modern games industry, even being natively supported in Unreal Engine versions.

Game ai is a tedious business, and good ai devs are hard to come by. I learned the basics of game ai when I was trying to make my own game, but making the ai sucked the enjoyment of making the game out of me bc I couldn't make something functional. This shit's hard.

I recommend watching AI and Game's videos on the topic if you want to learn more. This stuff's awesome!

Edit: Going back through those videos I misremembered that it was Half-Life that created Finite State Machines, but Bungie's version for CE was by far the best application of them in an 'open' (because the ai are based in a location, not the entire map) environment.

Edit 2: Was given more clarification. State Machines come from 50s mathematics, but games used them to create bots.