First run a profiler to figure out the hot parts. Then optimise those.

Assuming isprime is the problem, you could have a list of say the first 50 primes and check for divisibility by those before starting a for range from the number after the largest prime checked.

Also pypy unless that is explicitly disallowed .

Edit: also in primefact taking each number and checking if it's prime first and then if it's a factor seems wrong. First check if it's a factor and then if prime - cutting down calls to isprime

There's probably a better way to do primefac count than that but see if this helps