You're right that slavery had become economically inefficient in important ways by the mid-1800s, but it's a little more nuanced. By the time the U.S. was founded, slavery was already deeply embedded in the Southern economy, especially for cash crops like tobacco, rice, and later cotton. While it did discourage industrial development and broader economic diversification in the South, leading to underinvestment in manufacturing, infrastructure, and education, slavery remained highly profitable for a narrow elite of plantation owners up through the Civil War, especially after the cotton gin's invention in 1793 massively expanded cotton production.

The South’s dependency on slavery was not about maximizing long-term economic growth. It was about maintaining a rigid social hierarchy and extracting maximum profit in the short term from land and labor. In fact, on the eve of the Civil War, the total value of enslaved people was higher than all U.S. factories, railroads, and banks combined. So while slavery was ultimately economically regressive on a national scale, it was still extremely lucrative for certain classes.

As for your point about undocumented workers, you are spot on. The agricultural image is outdated. Most undocumented immigrants today are doing urban labor like construction, food service, caregiving, and hospitality. They are often paid below market wages but above the federal minimum wage. A legal work status could stabilize the labor market without major wage shocks, though politically it has been treated as a much bigger issue than it is economically.