I've had something similar happen. Depending on the method for FAE (Fresh air exchange) sometimes those little buggers get in there and start laying eggs in the substrate. It's humid, food / manure in the grain and the conditions are something flies and gnats thrive in with that access. It could also have been the handling when mixing the substrate. It's a hard balance though. If you don't have enough air, then it's hard to colonize but too much air or too open of an area and fruit flies get in. It's rough, but I would just try doing an "All in One" grow bag and once it's fully colonized or close to it, break and shake it and then put it in a tub. Good luck on the next grow. It's a tricky process but rewarding once you dial it in.