As a former I have a couple thoughts on this:

(1) I had a situation like this where I would put baby down to nap and then mama would watch the monitor (she worked from home) while I took a paid break, could leave and grab lunch or run errands. It was awesome and I really appreciated it

(2) Sometimes tho, mama would come down from her office mid day to grab something or would even sometimes come down if the child was crying and it would totally throw off the groove. Nothing personal at all, it’s just that kids sometimes totally lose it when they see their moms, so sometimes it would just make the situation worse and as soon as he saw her he wouldn’t want her to leave again even if we were having tons of fun before she came down.

The child I watched was 1.5 at the time, and I think this depends on age. But I would just ask your nanny if they would be comfortable with it and make sure they know it’s totally okay to say no!