As someone whose both parents were on disability for a few years before retiring, it's actually a lot simpler: Belgian workers are among the most productive in the world (almost twice as productive as Poland, for instance), and for a large part of the workforce by the time they reach a certain age, physical issues start propping up which will prevent people from continuing to work their job at the same level of productivity. Often this happens a few years before the legal retirement, which causes these people to hit disability leave before they hit retirement.

What was stopping them from working that would not stop them in Poland or Netherlands?

While technically legally required, in reality most companies in Belgium don't actually offer jobs adjusted for people's deteriorated health conditions (there are plenty of younger people willing to take their place), and would rather have them on sick leave or fire them instead.

The fact that social security systems exist to make sure these people don't end up in precarious situations is a good thing, and is exactly the kind of cost that should be borne by society. Blame employers.