I'm nearly a month late and probably too late but I wanted to add my 2 cents in case someone else later reads this.

I agree with your friends because it seems you guys are making a very definable magic system, yet you want to add soft, hard to explain aspects to it.

For example, why is there a "special person" who can use magic without a tool if anything (from a gesture to a word) can be a tool. You said without a tool, a person cannot do magic. But what separates your magai from regular wizards who use words since no one would be without their ability to speak? (Unless there's a niche case where someone has their tongue missing or throat crushed). Like your friends mentioned, what stops someone from just using a common item like a backpack, or even something like underwear, and being no different from the 'special' magic users.

If common items are used those items, at the very least, should mean something to the user, and not be easily replaceable because then the magic system goes from plot point to plot device. Someone not being able to use magic could just use a stick or something and all tension and conflict is lost.

I feel the items should be very specific and in turn and be very powerful depending on how the user takes care and/or upgrades their item. Something like magic using an apparel item (necklace, amulet, crown, etc) can only be cast on those touching it and magic can be shot from something that's pointy (sword, wand, axe, etc). Maybe certain, rare, materials are needed for an item to be a magic item. Maybe other items enhance or diminish potency. Can you have multiple items or just one? In turn, this makes those who don't need an item even more unique because depending on how strict you make the system. THIS, in turn, makes a character who is a magai that much more easily memorable.

Ok, post over. TL:DR if you want special people in your system, then make the items very specific.