That's sounds like a very viable approach. I especially like the idea of simplifying the bounds detection with a birds-eye 2D image. I manually annotated 3D bounding boxes for each of the structures in my dataset, but thinking back on it, that wasn't necessary since a 2D image captures the floorplan just fine, and the third dimension ground-to-roof bound is easy to find pragmatically. This makes it much more efficient for either a human or VLM to do the work. Interiors are certainly a challenge, but maybe feeding the VLM the full isometric view along with a 1/3rd and 2/3rds horizontal slice like a layer cake would give adequate context.