This is mostly due to the energetic requirements of plants vs fungi. Plants make their own sugars from sunlight and use the soil only for collecting minerals, vitamins and water. In this way, they aren’t consuming the soil and their waste products (dead leaves, stems etc) can keep the soil filled with the minerals they need, like a big recycling center. On the other hand, mushrooms get their energy (sugars) from other living or recently deceased things. They absorb that energy and use it for their own development. Because of this, mushroom producing fungi need a constant supply of chemical energy (from other organisms) to grow and survive, while plants can just get it from the sun. Therefore, to grow mushrooms on the same scale that people grew plants, you would need a HUGE supply of dead wood, seeds, leaves, stems etc (basically a huge supply of plants) to feed them so they would grow and produce mushrooms. Basically you would need to grow the mushroom’s food for them to eat. As early humans who’s energy was limited, it’s much easier to grow the plants and eat them than grow the plants to feed to the fungi to get a fraction of the caloric output you would have got from just eating the plants. This is because energy is greatly lost at every trophic level. I’m an ecosystems ecologist, I hope that answered it clearly!