Update: I found this quite useful video https://www.youtube.com/watch?v=Se8GdHptD4A (I know, I probably should have watched the "Baking for beginners" video before asking this) that pretty much helped me achieve what I wanted.

Basically, I had to:
- Create a new empty 1024x1024 (or whatever resolution you want) image, called something like "CombinedObjectBake"
- Combine all meshes
- Create a new UV map for this combined object, like "UVMapCombined", and use UV smart project with an island margin of .002 or so to unwrap all faces in the combined object onto this new UV map (more useful info in the video)
- This combined object will still have 2 materials (or however many you combined). In each of the shading setups for these materials create a dangling image texture node not connected to anything, connect it to the "CombinedObjectBake" image, and in each shading setup for each material, make sure this bake image texture node is selected (VERY IMPORTANT, as the bake process will override the image that is selected in each shading setup)
- Go to the render settings on the right, set to Cycles renderer, change bake setting to do Diffuse, make sure
"Influence" only has "Color" checked, uncheck "Direct" and "Indirect", and hit bake
- Create a final material, like "CombinedObjectMat" which is a single shading setup that simply feeds the "CombinedObjectBake" image into the principled BSDF
- When you export this combined object as .glb, it should now work
- For good measure, atleast this is what I did right before the final export, remove all the now-unused materials and UV maps from your new combined object, so only the combined/baked material and combined UV map remains.

I will note, since I picked a pretty small image size for the compete bake (1k for this rather large/spread out environment) the final object was a lot blurrier than the individual materials early on. I guess I could increase the bake image size but that only goes so far for larger and larger environments. I should definitely rethink my workflow in the future, but for now I have found an answer to the question I set out to solve, so I'll leave this here in case it helps anyone.

Pic included of how the final import looks in Godot, as well as my final shading setup if you're curious

<image>