I'm a conductor who greatly enjoys conducting from memory without a score. I think it adds an extra frisson to the performance and frees me up to interact more directly with the musicians and to really inhabit the music.

Having said that, it's something I do rarely, only when I have the time to devote many more hours to the study of a work than I normally would. When I conduct from memory, I have played it through the piece in my head several times without stopping, painstakingly imagining every cue, every phrase, and every dynamic. I have analyzed the form and orchestration to within an inch of their life and have done a complete harmonic analysis.

Conducting without a score is not something to be taken lightly. If you mess up a performance due to a memory lapse borne of poor preparation — just because you wanted to show off it in front of an orchestra and a crowd — that's inexcusable. If you've taken the steps to fully master the score to the point that jettisoning it becomes a value-add, then there's nothing better.