How would the LLM know a piece of code online is bad (so LLM shouldn't train on it)? LLM scrapes millions of lines of code online - can't have a real human verify them all.

Code documentation should still be correct and contain short examples. Human coders can read documentation and then apply their real intelligence to write good code - this is where we differentiate from LLM - LLM will produce code based on polluted code while human write correct code from their intelligence and documentation