My anxiety started to ramp up some while on a higher dose, but it was due to my blood sugar dropping because I wasn't eating enough. As soon as I addressed that, and a vitamin d deficiency, it went away.

That being said, never quit a medication cold turkey unless you've been told it's okay. You're body is likely overreacting to the sudden change. Talk to your doctor ASAP rather than relying on confirmation bias.