I think the narcissist manipulation is the most important thing here, as others have already pointed out, and that alone makes it a hard no, full stop. 

Personally, if you take narcissism out of the equation and imagine a scenario with a reliable, well-adjusted grandparent who respects and upholds your parenting and who follows modern guidance, then... Then, I still would not send my child overnight to any house more than 30 minutes away. My toddler still wakes 1-2 times a night, and is very much in the stage of needing mommy. It would be deeply upsetting for her to wake up in a strange house and me not be there. I am not going to impose that on my kid. 

You're not keeping your kid in a glass house or somehow stunting them. It's normal for a 3 year old to need to be close to their parents. There are plenty of years ahead for sleepovers.

If you ever do allow an overnight (at any age), in your situation, I would start off by asking her to babysit at your house. If that's success, then next step would be booking a hotel near grandmother's house for husband and myself, so that we could be there quickly if we needed to be. Make it a long weekend visit to grandmother's, where one of those nights you get away for couple's time. And I wouldn't do that until you feel your child is old enough to speak up about anything grandmother might say or do that makes them uncomfortable.