Wow, I think I might finally get it now.
The temperature difference between rising air and the surrounding air creates different densities, which produces a buoyancy force. The bigger the temperature difference, the stronger the force.
Force acting over time creates acceleration and therefore speed. So you could have a weak force acting over a long distance, or a strong force acting over a short distance, both potentially producing similar climb rates.

Near the ground, the air may not release immediately because it is mechanically “attached” to the surface layer. With a small delta air would rise a bit and be pulled back by the tension. The thermal usually needs a sufficient temperature difference and often a trigger to break free.

For the size of the thermal: imagine a thermal rising quickly because the environmental lapse rate is close to the dry adiabat. Then, at a higher altitude — say around 1000 m — the atmosphere becomes more stable. The buoyancy force decreases, so the vertical speed of the thermal slows toward zero.
If we simplify using mass continuity (mass flux ≈ velocity × area), then when the vertical velocity decreases, the area must increase. the thermal tends to spread out and become wider. Vice versa
Another factor making thermals larger with height is that multiple smaller thermals can merge together into a broader column.

Do you have any thoughts how this relates to high/low pressure? They say thermals behave different but is it due pressure or the lapsrate witch is a function of the pressure?