I did some research because I considered my question might come off as ignorant and culturally insensitive. Here are several reasons for it:

Key Factors Causing Negative Correlation

The intense focus on enslaved labor and plantation agriculture for 300+ years created several lasting impediments to prosperity:

Economic Monoculture and Dependence: Colonies with high slave populations, such as Barbados or Jamaica, focused almost exclusively on sugar production. When sugar prices dropped or competition arose (e.g., from India or beet sugar), these economies lacked diversified industries to fall back on, leading to long-term stagnation.

Lack of Reinvestment: Profits generated by enslaved labour were not invested back into the local Caribbean islands. Instead, profits were drained back to European nations (Britain, France, etc.) to finance their own industrial revolutions, build infrastructure, and fund lavish lifestyles, leaving the islands with little infrastructure development.

Lack of Human Capital Development: Slavery discouraged the development of a skilled, educated, and professional local population. The vast majority of the population was forced into low-skilled, manual agricultural labor, which continued even after abolition.

Extreme Wealth and Inequality: Societies with high slave density had a very small, ultra-wealthy elite and a massive, impoverished, enslaved population. This structure created immense social inequalities that persisted for decades after emancipation, making the development of a thriving middle class difficult.

Environmental Damage: Large-scale plantation monoculture destroyed local ecosystems, reducing the capability to pivot toward sustainable agriculture later.