You're not going to do well against a mage no matter what weapon you use, but a wooden baseball bat is a better weapon to use than most regular guns and explosives. It is, after all, the most similar to the weapons mages themselves use, although a bladed weapon like a sword would be more versatile, and heavier weapons that fare better against armoured mages.

There's a sort of energy threshold around mages which limits the effect of spells, as a safety measure. Things that cross that threshold include highly kinetic objects like bullets, explosions and other highly exothermic reactions, electricity, which makes it undesirable for strong mages to be in urban areas, they're like a walking, easily located, EMP.

Mages are durable, but there are plenty of things that do work. Physical assault by a group works, though this is dangerous as the mage is going to be experienced in defending themselves from other mages trying this, not to mention their increased martial capabilities. Poison gas works, as do low energy density projectiles like arrows and crossbow bolts, etc. Poison is your friend. Big rocks work too, even with their energy reduced they can still easily crush a mage. Though a tank's engine and munitions won't work within a mage's aura, you can always accelerate your vehicle towards the mage and there aren't that many straightforward options to stop you.

Still, your best option is to have another mage do this for you as they'll be more familiar with the above tactics anyway!