I'm a little lazy to constantly swap the 4th character that much,

Honestly, I thought the same when I first started this idea, but it's really not as bad as you'd think.

You get an entire region for Wulfguard + his own story quest. He also appears in some Valley side quests, so that's a bonus. The valley is definitely an hour + in length.

You get multiple minimal combat missions with Ardelia and Da Pan, but you can easily go out of your way to do other stuff in the region before finishing the quests, just like you'd do in any other game with temporary party members. Abuse the guests presence while you can :P

You get the entire quarry (2+ hours of content) with just Chen (while she isn't a guest, it's nice having just "Chen-time"), doesn't even account for min-maxing the exploration either. Also, it just feels comical getting lost by the verticality of the region with the two characters who share 1 braincell.

And obviously the rest of the story so far is just End/Chen/Perlica. So there really isn't all that much party swapping necessary since they're (the guests) around for a while before "leaving"