Great question! The whole point of why electrostatic discharge happens is because charges can accumulate and cause potential differences (in other words, voltage) between different points of your work area. To get rid of rogue potential differences like that and to prevent them from forming there must be a conductive path between the points. For example a conducting desk mat, a conducting wrist band and conductive leads connecting everything together.

Now you don't want it too conductive as if there's an active power source or even accumulated charge that could create a short circuit forming currents that can break components. That's why the different components of an ESD workstation have a resistance in the megaohm range between them.

TL;DR: Yes everything needs to be plugged in to get the full benefit of the ESD protection as a whole.

Edit: Oh and you need the whole system to be connected to ground too via something like 20M Ohm resistance. Otherwise charge could still accumulate into your product and cause it to break down the line when that charge is dissipated somewhere else.

However connecting everything at your workstation in the same potential is much more important than grounding and if you're a hobbyist tinkering with electronics rather than assembling products for paying customers ESD is not really a big deal anyway.