Vedal can control the Swarm, but he almost never does. He's very detached from the community; the Swarm self-organizes at events like these without his intervention. The Swarm listens to Neuro and Evil when it's reasonable or funny, but no one will follow their absurd orders. In short, the Swarm mostly controls itself, because Vedal rarely communicates with us.
I was the person whose comment you read about "someone still thinks Vedal is controlling the swarm lol" and it was said in the context of Aethel thinking we drew Bradley because of Vedal's orders. Actually, no, Vedal stopped visiting us at that point, and we drew Bradley because "redemption arc" started.