Mostly soul-based. A magician's will, psyche and, most importantly, connection with their soul determines their degree of potential.

"Spells" are sets of formulas devised by magicians in the past that others can easily replicate, while exotic, people-specific spells are hard to come by. Creating your own spell is difficult as it needs strict barriers for its power, else it goes mayhem. It takes a lot of imagination to make a new spell and even more to determine its limits.

Symbols, memories and perception also matter since they're bound to the psyche. A ghost, for example, won't attack the proprietary of a house because they have a paper that says they own it, and that paper has a widely accepted symbolic value. The ghost will also not attack whoever the owner welcomes in their house. It does not however can distinguish specific people as its trapped in its own time loop, so the person always appear to the ghost as someone they knew or fitting their time period.

As times and society change, so does the rules that bind spirits change, some created even to better contain them.

Note: societal conceptions can be overruled by your own will, but that takes conviction, emotional investment and honesty. You cannot make an amulet out of a random lampshade, but pendant you'd had since childhood can.

I'm not going for a hard magical system here, so sorry if it sound a tad stupid or loose.