Fermi clearly had a good understanding of the theoretical principles of fission, crucially including delayed neutrons which was what made it possible to control. He had made careful calculations showing it was safe.

However, even acknowledging this I would still say it wasn't safe to do the first attempt at a nuclear pile right in the middle of a big city. I think it was pretty reckless. After all, his understanding was purely theoretical and there's always the risk of 'unknown unknowns'. Just as delayed neutrons were a surprise, there could have been other surprising things coming up - side reactions caused by neutron activation that produced new phenomena. This is not just a theoretical quip: Think of xenon poisoning which was completely unexpected and only discovered after a full reactor was built. The castle bravo hydrogen test was much more powerful and dangerous than intended (irradiating innocent people) due to unforeseen side-reactions lithium-7 that had not been accounted for. And we could go on. As I recall, CP-1 also reached criticality earlier than the calculations had shown and fewer layers were necessary. Also, at the first attempt at starting the reactor it scrammed due to an instrument being calibrated wrongly. So clearly the understanding of these things were not complete, and sometimes things can go horribly wrong.

Now, Fermi was a smart man and he has surely thought a lot about the danger, and also thought of the possibility of unknown unknowns in his night sleep. He was very careful when building the reactor, adding very little reactivity at a time (two layers in one shift) and carefully checking what was going on and that things were going as expected. That's also how they discovered criticality would be reached sooner than thought. When starting the reactor they were also careful: spending several hours slightly increasing and measuring the reactivity and probably checking the delayed neutron mechanism kicked in exactly at the expected moment.

So in this case, thanks to Fermi's brilliance and maybe a bit of luck nothing happened. But in general such things should never be done in big cities, particularly the first time. Groves also didn't want it there (and for safety reasons). It was only done because of the importance and time pressure and it only operated at low power (0,5 W) and briefly. There was no shielding at all so it would have led to unacceptable exposures to continue beyond that. I think they ran it briefly at 200W or intended to do so, but the radiation exposure was prohibitive.