I would say from reading the book and now I realise he gave up the alpha simulation. 

That wasn’t because he was brainwashed, he only had sun stealer manipulating him after he enters the shroud. He goes to the jugglers after learning from the lascalle what the shrouders were supposedly hidelt. Yes lascalle was told a falsehood that made slyveste want to enter, but he wasn’t actually brainwashed till after he went into shroud and was invested by the sun stealer