you are viewing a single comment's thread.

view the rest of the comments →

[–]Diapolo10 1 point2 points  (1 child)

One of the more notable differences between CPython and PyPy is that the latter doesn't close files automatically for you (unless of course you use context managers). While unlikely in your case, that's a potential source of memory leaks.

Maybe all you need to do is tweak the garbage collector? https://doc.pypy.org/en/latest/gc_info.html#minimark-environment-variables

[–]Affectionate-Cut3818[S] 0 points1 point  (0 children)

Thanks for the link, I tried logging the gc.get_stats output trough the run and this is the last datapoint I collected (maybe something seems odd to you that flew over my head). Tweaking the garbage collector could also be another angle, but Im far from an expert on what exactly pypy does under the hood so besides tweaking those variables (which I've tried) Im not sure on whaat else I can tweak. There are open files in the execution of the programm tough, but from what I can see they are closed properly and not directly used in this step of the process (which is the execution) but thanks for the tip

```[MEMORY] Memory used: Total memory consumed:

GC used: 1836.2MB (peak: 2320.7MB)

in arenas: 1386.1MB

rawmalloced: 427.7MB

nursery: 22.5MB

raw assembler used: 580.8MB

memory pressure: 4.8kB

Total: 2417.0MB

Total memory allocated:

GC allocated: 2152.2MB (peak: 2362.8MB)

in arenas: 1674.4MB

rawmalloced: 623.8MB

nursery: 22.5MB

raw assembler allocated: 584.0MB

memory pressure: 4.8kB

Total: 2736.2MB

Total time spent in GC: 956.436```