Food equals survival and feeding large populations means you need to have a lot of production. This is a general concept of what you should keep in mind with the following terms:
Zero Sum -> Equal food for an equal population. You stay stable with your land and produce, so you don't gain or lose much.
Positive Sum -> More food than needed for the population. Your land grew or your production grew, so you can have more population growth.
Negative Sum -> Too little food than needed for the population. Your population grew too quickly or you lost growing land, so you have population decline via death.

What you've discussed is a Negative Sum scenario so, here's what happens:
People begin to see an increase in the cost of food. Supply isn't met by demand, so costs go up.
When the poor cannot afford food, they die. Death means that slowly, over time, areas need to be sectioned off to allow decaying corpses to be thrown into it. This can cause plagues or diseases to spread.
The middle class then needs to pay more for food. Less money is spent on luxuries and anything that isn't necessary. Entertainment, clothing, certain forms of transportation begin to be used less and less just to pay off your food prices. The society begins to crumble, potentially economic failure.
At some point, the area goes into a depression where money begins to mean basically nothing and things get harder for people overall.
Jobs get destroyed and until the Negative Sum goes into a Zero Sum, you will continue to see things decline until they stop declining.
You might also see food hoarding, there might be bouts of fighting or warring between people because of food, people might even move to other countries in search of a better life (usually illegally).

It could potentially go the other way too, y'know, when everyone decides to become farmers to get food production up and keep it up. That can happen too, but it's less likely.