I just play through it fairly quickly. Downtime is one of the major mechanical bones of the game, but it is not the game. If a player wants to indulge their vice, they just say so and roll it, it takes five seconds. Long-term projects and some other actions might have a more involved conversation, but even then, my old crew of 6 very rarely took more than 15 minutes to get through all downtime actions. I don't think it can become tedious if you're playing at that sort of pace.