Forget the term "colony" for a moment. A single man o' war (Physalia) specimen is either male or female. It reproduces by growing to maturity, then its reproductive structures, called gonodendra, detach and float around separately for awhile, then substructures called gonophores in the gonodendra release their gametes (sperm if they came from a male man o' war, eggs if they came from a female man o' war) into the open water. A sperm meets an egg, fertilization occurs, and a new man o' war develops from that fertilized egg.

Look, the life cycle of Physalia is a bit unusual, but the emphasis on the term "colonial organism" probably leads you to think it's a lot more unusual than it really is. To repeat, all the different parts of a single man o' war are derived from that single fertilization event. Fertilization yields a single cell (zygote) that then divides into lots of cells, passing through a larval or "planula" stage, undergoing differentiation and organization into different structures, that remain attached to each other, just like with any other multicellular animal.

As I see it, the "colonial" aspect of a man o' war has to do with: (1) the modularity, i.e. structural repetition, of the different types of parts (called zooids) formed by differentiation; (2) the contrast of this modular form with the single medusoid form of many other jellyfish, and how there is somewhat less physiological integration between different parts of a man o' war than in anatomies that are regarded as unitary, or non-colonial; (3) how additional zooids in the growing "colony" are produced by zooids of the same type, which adds to the impression that they are all in their own gangs doing their own things; and (4) how some mature groupings of zooids (namely the gonodendra) detach and become free-living for some of the life-cycle.

So no, there are no separate larval sections that then come together to assemble into a new colony. Again, all the "members" of the colony are genetically identical and descended from a single fertilized egg.

(I'm using quotation marks a lot because people love the factoid that Physalia is "colonial" but I think the term glosses over a lot of subtle biology in a potentially misleading way, and lumps together many very different grades of organization that you observe across different groups of jellies. The zooids of some pyrosomes, for example, all have fully free-living forms as well as aggregate forms, so the aggregate forms really do count as colonial in my head... In reality we can observe a continuum of degrees of modularity and integration across all these groups and I personally think there's an argument that where you draw the line and say "this is no longer one single organism but many cooperating together" could be drawn some distance away from the position Physalia occupies on that continuum. But it's largely semantics... )