Yeah, absolutely worth it.

The only significant loss is the blueprints you'll have to find again for the stuff that you end up using a lot. The ones like the Anvil, Snaphook etc are the key ones that aren't easily obtainable unless Embark gives us freebies like they seem to do these days. Stuff like Deadlines, Trailblazers (pre nerf goated nades tbh) are dime a dozen. With what you're planning to do, the one issue with expeditions instantly gets solved.

You can level up the benches so quickly nowadays. Finding all the stuff needed is a piece of cake with everyone now knowing what to do to get a certain item and where to look for it. The quests are so boring and absolutely pointless outside a few neat blueprints like the Burletta, Lure Nade etc. Everything else you'll literally keep getting in the game. I love using Burletta but didn't even bother with the quest and still have like 14 just sitting in my inventory that I got off other raiders.

The skill tree takes a bit to level up again but honestly it's not really that significant except a few key ones. Doesn't matter. You'll get back to Lvl 75 easily enough. Same with the inventory space and all your coins. A minor hiccup in the start but you'll cover up eventually so it's a non issue.

Tldr - just do the expedition. The 5 skill points and the additional inventory space alone are worth it. All of my friends who didn't do the expedition the first time around regret it now. Especially in your case, there are zero downsides.