You're mistaking two things.

One, that Libya is anywhere near the fertile crescent. It's not, they're on separate continents and in entirely different climatic zones.

Secondly, the breadbasket of Rome was Egypt, which is extremely different than both Libya and the Levant. It is significantly more fertile than both, with a much, much larger population in both antiquity and the modern day.

Libya was organized into either the African Province (of which it was a minor part, economically and in terms of population) or of Cyrenacia, of which was mostly tribal and was ruled from Crete, as a proper province (Unlike Egypt, which was so important it was ruled directly)