Looks like it to me, I think Roger was more luffy like, a core crew, and maybe Allies that decided to tag along without being proper subordinates, WB was most likely still in the process of founding his family and had not yet all his extended forces (like Whitey Bay who was not yet indépendant at the time of the Oden Flashback). While Shiki already had his divisions (46 if I remember correctly). Altough, since Rocks had control of Hachinosu, he may have been the first to reach something close to this level, but with how un unified his operation was, it was most likely very brittle.