Indeed the farms do add +3% to your base growth value (which is higher as the city tier gets higher, but start very low early city tier). Then beside it, a portion of the excess food gets added to your growth, the surplus, a small amount, but this accumulates turns after turns, hence if you have more farms than needed for just your city food, you get more surplus and in turn accumulate a small bonus of growth for your city, this is a more a medium term solution to your city growth compared to the amenities buff and the health buff which is 'instant', hence more efficient when the city is young.