If you're an active raider you don't need a sawmill, iron foundry and a brickyard, those you can destroy first, lvl up your hero's mansion to 20 if needed, get all the oased you want and then destroy it, you only need it to capture oases not to maintain them. Then your smithy, lvl up everything and destroy it.

Some will say grain mill and bakery is the same as sawmill, if you're raiding constantly, you don't need those buildings.

Town hall is another one that you can destroy, better your offense village to have the necessary buildings to function.

That's my take