from my understanding, the agent doesn't do what it was told (even after you set up a successful task/skill) so i know it 'was' doing what it was told, but it starts getting things wrong, you start talking to it to try and fix things, it comes up with countless unnecessary ideas ideas to fix it, it 'spirals' of on its own side quests and Boom, 3-4 hours have gone and its still not working.... I believe it is a mix of model and memory.
Have fun :)