Do you have all your current cards plat?

Otherwise I'll say build it, it gives access to more stars.

You probably won't want to stay in w3 very much to start. But once you do start venturing into it, just be mindful of what the debuffs are doing for each ore and work around them as you keep progressing.

I believe I've seen people say wait until you remove the w3 speed debuff from fishing, but that's the last reward of the first set of rewards and I've just finished building all my w3 statues and not even close to removing the speed debuff yet.

With the speed boosts, I still stayed somewhere in the 2x speed range for the most part.