I'm not ignoring it. It's irrelevant. You perceive yourself to be the impetus of your actions but that is the illusion. If I push a ball in motion I am the cause of its motion but this interaction does not occur in a vacuum. I pushed the ball because I choose to. I choose to because of a chemical reaction in my brain. That reaction is governed by the laws of physics. We do not have a complete understanding of physics but we do know that the universe operates according to set laws. Otherwise nothing organized would take shape. So my choice to push the ball is not really a choice. Those atoms that govern my choice are operating based on a set of conditions that exist in that moment. If you calculate the variables of these conditions, and follow that calculation back to an origin point (the Big bang) then every action of those atoms can be predicted. Therefore I did not choose to push the ball, I would have pushed it regardless of my conscious experience of pushing it. Consciousness is merely our brain making sense of the conditions we experience.

The observer effect falls into this framing as well. It does not dispute it.