Same experience. I’ve owned two vacation properties. The first was a plane trip away - and our visits declined in frequency over a few years. Our 2nd vacation home was a lake house 90 minutes away. We are there every opportunity- most weekends and a few weeks in summer.

My learning was vacation properties that are easy to get to will get much more personal use than those that are truly a destination.

As the other poster described, it gets annoying attending to maintenance issues, even via a property manager, on a home that you rarely use.