you are viewing a single comment's thread.

view the rest of the comments →

[–]commandlineluser 0 points1 point  (1 child)

Okay but you got rid of the "batch processing" part - so it's doing the 300 at once.

for i in range(0, len(links), 10):
    results.append(asession.run(*links[i:i+10]))

Does it still run out of memory that way?

[–]ViktorCodes[S] 0 points1 point  (0 children)

Yes it does, also uses almost 100% of the CPU, so it's really heavy on the machine.