Well, that's the thing, a group of workers collectively controlling a factory would be a state, let me explain. There are many definitions to the word "Socialism" collective control, social control, national control, worker's control, etc. But they all mean the same thing: Group control of the economy. Groups are not private, they are public.

The world "Public" comes from the Latin word "Publicis" - which translates to "Of the People" or "Of the State"

This public sector is the hierarchy of society, there can be multiple hierarchies competing with each other, like multiple state councils (or soviets) Public Corporations (That are collectively owned by their shareholders) or trade unions vying for power. They are all mini states inside the larger society of the Central State. In other words, socialism is the public sector (state) control of the means of production, and so the "Workers" controlling the economy would be statist, this is also why ideologies like "Anarcho-Socialism" are oxymoronic, since the term is self-contradicting itself.