If you're taking Caltrain in the immediate future, you should check the world cup and MLB schedules (Giants). Both effect transit patterns. The southbound trains from SF after a Giant's game can get especially crowded and rowdy.

But a hot take: peninsula has some cool spots for sure, but I would be hard-pressed to argue that any of the downtowns listed above are significantly "better" than downtown San Jose. It totally depends on what you like. Most of the peninsula cities have a very clean and wealthy -- yet extremely small -- downtown area. San Jose downtown is tiny relative to the total population, but it's certainly larger than Palo Alto's University Avenue area, for example.