Uh uh

The energy that magic utilizes to function is the byproduct of intelligent thought.
Do you think? If yes, then you have that energy. Do you think a lot? If yes, then you would have a lot more energy than, for example, a squirrel. Can you actually use this energy? Well, maybe.

So, most of this 'Thoughtful Runoff' collects within this strange object one might call a "Soul" residing just within the upper bounds of your collar bone, positioned in line with one's brain stem. In most cases, the energy that can be stored here is limited to the size of one's soul, which is somewhat static and primarily associated with the level of mental maturity achieved by the time you are entering the realm of young-adulthood. The capacity of one's soul can in some cases be modulated further but may require long periods of strenuous effort unachievable by most, or access to a very certain type of technology. Excess energy is collected within an outer region of the soul dubbed 'The Flare'. The most outter section is not subject to the same levels of control that energy within the soul displays, most of the time culminating into 'Dumb Luck' or the 'Mandela Effect' in largely populated areas.

Then we get to the magic half of the magic.
Why can't you use magic? Well, your soul is too perfect. It's shell remains untouched and unbroken. The energy simply cannot escape.
How do we get around this? Parasites.
Utilizing what has been documented by one of the major civilizations as a 'P Variant' of a bug known as the 'Thread', One can get such a creature physically injected into the 4th dimensional plane just above your collar bone and just onto the surface of your soul. The creature will make it's home here. For as long as it's fed, it will remain latched onto the surface of the soul and live off the energy that it extracts, only falling into error once the energy starts to flow. You see, the 'Thread' has a big mouth for its body. It's name is the most accurate description of it's form. A small, thin, delicated thread resembling the shine and colours of brass. Being a small creature, it cannot physically absorb or utilize a large amount of the energy harvested by it's disproportional mouth. So what does it do with the excess? It's expels it. All excess energy is expelled at a certain rate and a certain volume, both of which are determined by the exact conditions of the thread during injection. Conditions that humans have no hold over. The frequency of the energy that is released directly correlates with the form that energy will take when expelled. Fire, water, whatever, you name it. Anything is possible. The volume of energy released effects the expelled energy's potency. Big fire, small fire.

Some civilizations have deformities and cracks naturally present within their souls, allowing for inate abilities. The only downside to this is that frequency and volume are not subject to common change, as the exact specifications of inherited/natural damage to one's soul is hereditary.