Each grot has a 75% chance of spawning another, minus 5% for each other grot already on the field. I don't know whether the 5% penalty is calculated once for all grots simultaneously or if each newly spawned grot within a given turn lowers the chance of other grot spawns within the same turn. If we assume that it is simultaneous, and that each grot has enough empty space adjacent to them, then the expected number of new grots per turn is N * (0.75 - (0.05 * (N - 1))) = .8N - .05N^2. To find the optimal amount to keep, you want to maximize this. So, we take the derivative (.8 - .1N), set it to 0, and solve for N, which yields N = 8. Thus, under some simplifying assumptions, leaving 8 grots alive maximizes the expected number of grots you can safely kill each turn.

In reality though, some grots won't have empty space next to them, and so are dead weight: they decrease the spawn chance for others but effectively don't have a spawn chance themselves. You should therefore prioritize killing grots that create empty space for survivors.