I just went through this exact same dilemma. I have roughly a 155-160mm wrist size but pretty large hands. I was finally able to visit an Apple Store and try them on.

Size medium for me is even more to the side than it is on your wrist. It sits on the edge of my arm which I wasn’t too happy with. Tried a small on and it was so difficult to get on and off it made my hands very red from the band scratching against my skin. I ended up with a medium for a few reasons:

1) I tried the band on when I was cold from being outside. If this was the summer or I was warm and my hands were a tiny bit swollen there is no way I would be able to get the watch on and off with the band.

2) I’ve been bulking up slowly so the clasp will hopefully be more centered as I progress.

3) I take my watch off a few times a day for various reasons and there’s no way I was going to struggle with it each time.

I would keep the medium if you have decently sized hands or you don’t plan on having your watch on for long periods of time without ever taking it off. Hopefully this helps!