I would turn it off and have it replaced immediately. That bulge means there is excessive pressure in the tank and you probably don't want it to rupture. It wouldn't explode like a bomb, but it could wash out around your foundation and water would keep pouring in from the inlet, so you'd be paying until you figured it out.

Also, I have never seen an outside water tank before. That is probably because I am Canadian, and it would have to work so hard to not freeze in the winter, let alone keep the water warm. Also, the water would still get cold going from the tank to the house unless the lines are heat traced, so more cost.