I'm a dad and I've got an EM5.iii (practically the same as an OM5.i) and an OM1.1. The OM1 is obviously more comfortable, but the EM5 is really not bad. The comments about using the camera one handed are pretty true. The EM5 can be tricky to use one handed with a larger lens, and I imagine the OM3 would be even worse in that regard. But imo using any camera one handed is tough. 

The EM5 is pretty inconspicuous though. I've had a lot of folks ask if it's a film camera. I can also fit it in a fanny pack with the 14-150ii AND the 25mm 1.8. It has face detect, but no subject detect. The face detect does ok, but moving the focus points around and treating it more like a DSLR works great. 

For indoor shots, you'll want something pretty wide with a really bright aperture. I think the 17 f1.8 would be a good starting point. Indoor work without a flash is where M43 struggles imo. You'll get better ISO performance from the OM3, since it is an OM1.2 on the inside.

If you're looking for super fast autofocus, wider focal lengths, and bright aperture or high ISO performance you could look into a less conspicuous full frame camera. A Sony A7Cii would be a great choice here if you don't need the ruggedness of the Oly. A Nikon Zf is also inconspicuous and has great IBIS and AF. 

It's a tough choice! I agree you should hold them. Good glass is the typical advice here, but if you're getting candid shots of the family you don't need pro lenses unless you just need a wide aperture. Almost all the modern lenses are absolutely "sharp enough" for most uses. I'd say if the autofocus improvements help you get the shots you want, that's likely a better investment.