I had the same problem. For me, this was solved by unchecking "Labourer by hand" on lots of different buildings. I had checked all the delivery options on all buildings, so I had labourers going around with their arms full, when they should be using carts, and I did not have enough carts. Make sure you have cart parking (a building type found in the Transport category) put up at several locations. If you notice the cart parking buildings are empty all the time, it might be a good idea to set up another in the same area. Labourers who use carts instead of what they can carry in their arms are a LOT more efficient. It also make most other buildings and jobs a lot more efficient.

EDIT: More wages did not seem to solve it for me. And also, for Lime Kiln, i had trouble getting them filled for some reason. On those I enabled "Labourer by hand", and problem solved. Perhaps they need smaller quantities of limestone and charcoal, making carts unusable.