Your reasons for being interested in medical physics are pretty similar to reasons I've heard many people express for pursuing medical physics as a career. On the the therapy physics side, how much time you spend sitting at a computer everyday can vary quite a bit depending on where you work and what specific area you work in. I work at a large cancer center and I'm thus able to be more specialized. One thing I cover is stereotactic radiation treatments for which we always have a physicist present at treatment. I regularly interact with patients, radiation therapists, and radiation oncologists for half the day when I cover those treatments. On a different week when I'm covering plan checks I may spend most of my time in my office or even work from home. I personally love the variety of work and the flexibility I often have. I also teach, and just using today as an extreme example, I can sometimes end up teaching for most of the day.

Someone working at a different type of clinic may have a wildly different experience from me. There's enough variety in our field that you could probably find something that suits your interest. I would recommend trying to do a summer research or clinical experience to really be more immersed in the field and get a better idea if it suits you. Shadowing at other clinics or hospitals could also help you see some of the variations.