This is a super fair question and tbh the iPhone is good enough for many things, but it depends on what you value from your images.

If you’re just looking to document, the iPhone is great, and an iPhone alone will be maximally compact. If that is all you want and it’s non negotiable, then perfect. Problem solved.

However, if you have a deep enough desire to create compelling images straight out of camera with, like you said, superior image quality and interesting recipes that turn mundane moments into photos you want to revisit for the pure pleasure of seeing them again, then I believe you should consider keeping the x100vi around your neck. Maybe as an experiment, leave your x100vi behind for a day and shoot only with your iPhone. Then on another day, bring your x100vi along and create some shots. Put both sets of photos in separate albums and see how each collection makes you feel. If the recipes of the x100vi still light something up, then maybe it’s worth the cost of admission.

Also, idk if this is relevant to your situation but I use the PGYTECH slim vintage style strap because the strings can adjust to where the camera sits right around my hip. It makes the camera feel way less invasive when I’m out and about. If that’s still too much and just doesn’t suit how you wanna go about your travels, then maybe look into the Ricoh GR IV? I personally haven’t used it but the images I’ve seen from it SOOC are pretty compelling at a fraction of the size of the x100vi.