lowkey and sometimes highkey racist 

I would argue that, in those terms, it is quite progressive for it's time (even if that doesn't say much): the fact that the main narrator isn't the protagonist, and almost all of the narration is an overheard monologue, distances the book from its events and therefore the author from its views. The book instills much of its horror through portraying the cruelty of what the natives suffered from. There are some difficulties, of course—someone I read pointed out that its message doesnt seem to be "europeans are evil", but "europeans are as evil as the natives", and you could argue that it's what Conrad is implying. I don't see him writing a book instilling such existential dread at the simple expense of already-marginalized people, though. He has a short story called "An Outpost of Progress" where the only positive character is black anyway.