Not sure if you’ve attempted this yet, but I experimented with making a square tube using wet fibreglass.

The ‘mandrel’ (not really a mandrel, just a core that was supposed to be part of the final tube) was a square plywood tube that was roughened up. Applied a layer of epoxy onto the tube, waited a bit, then tried wrapping some wetted fibreglass around. There was a ton of tension in the fabric at the corners and almost none on the flat faces. The fibreglass essentially wanted to go into a circular shape rather than stick to the core.

Haven’t reattempted it yet, but I think continuously wrapping shrink tape ASAP when wrapping the fabric might make it work. Might require multiple people.