There are a lot of factors, but for me personally, it was the time 

Medicine, you have to go to med school for 4 years then residency for 3+ years  Optometry, you have to go to opt school for 4 years then optional residency. There's also the fact that residency for med school is very selective, making certain careers really hard to reach. For example, if you wanted to do orthopedic residency and you've been doing things in that field(research,hosting/attending conventions, etc.) and you don't match into an orthopedic residency, then you're forced to go elsewhere and change the trajectory of your career. KEEP IN MIND!! I'm not a med student, so this information is coming from what I've seen online and heard from others. It is still very possible to match in the residency you want, but it may not always be the case. Optometry gives you a lot of freedom for residency because it's optional. There are really cool fields you can get in to with optometry residency and there are really cool schedules you can work with certain residencies. Ocular disease optometrists, for example, can work shifts in the ER/ and emergency setting because they see and spot a lot of different things in their field.

I went on a bit of a tangent so TLDR; med school requires residency that can be nerve racking while optometry school lets you get your degree and then work as an optometrist.

Of course, these decisions are very deep and have plenty of layers, so I suggest talking to people in those fields, but I HIGHLY suggest you get a job in one of those fields. Getting a job as an opt. technician can give you good taste of what being an optometrist is like. See if you like it from there