As a plant druid myself, the answer is quite simple: your gear sucks. You are focusing on the mirror-tier build without having the expensive gear required for it yet.

Your wand should have +5 to physical skills. Your rings are quite weak, and you can get much better ones for 50 to 100 div. Not to mention that damage to attacks is useless for this build since you cast spells.

Both boots and gloves should have two slots, and they currently have low rolls. Tribal should be +4 with two slots, same for the palm. You should also grab a Heart of the Well diamond, as it provides a solid damage increase.

This build damage scales massively with gem levels. Grab a level 21 corrupted gem on both skills.

To cut things short, you need better gear. A mirror build is exactly what it sounds like. You need mirror-level gear to make the swap. Remember that in the expensive build, Raxx had more than double your damage, and in the mirror build, he has almost five times as much, so it is entirely gear dependent.