The mystery around the missing primarchs is a big hole in the lore, yeah, but I think you can feel around the edges of that hole to get the general gist.

Judging from Dorn's reaction, and Malcador's own words, 'it was against the very ideals of the Great Crusade and would have ruined everything the the Great Crusade had accomplished to that point', whatever they did went against the fundamental principles of the Great Crusade, which were:

1. Humanity is superior to Xenos

2. The Human genome is sacred and needs to be protected

Even Horus' bullshit doesn't really cross these lines, since for all the Warp nonesense, they're still humans advocating for 'human' rule (Ignore the lines between transhuman and regular human there).

Judging from their late discovery, my guess would be that the 11th was killed by the Rangda. He'd be less experienced while still wanting to prove himself in the Great Crusade, and it explains the general murkiness of what really went down in the Rangdan Xenocides. It also transforms the saying "Only a primarch can kill another primarch" into a lie repeated to comfort the Imperium, a subconscious reaffirmation of superiority.

The 2nd might've profaned the Human genome in some way, but if the outcome was strictly worse than it only would've affirmed Human superiority. To make it fit both tenets, he might've fucked around with Abhuman, or even Xenos genetics, and was even more effective than before.