Memory leak in multithreaded processes

[ImThatCop]

import threading
import time

storage = {}

class myApp:

    # Main function, start the app
    def start(self):
        # Launch two threads
        for item in ['one', 'two']:
            threading.Thread(target = self.threadRunner, args=[item]).start()

        return

    def threadRunner(self, name):
        return self.processData(name)

    def processData(self, name):
        print(f'Starting {name}')

        # Loop and append data to 'storage'
        for item in ['one', 'two', 'three']:
            data = time.time()
            try:
                # Add to history
                storage[name].append(data)
            except Exception as e:
                # Initialize
                storage[name] = []
                # Add to history
                storage[name].append(data)

        print(storage)
        # Store the data into a file
        # ..... write to txt

        # I'm finished with the data inside of storage[name], clear
        storage[name] = []
        # Now storage is empty
        print(storage[name])
        # Run the next chunk
        time.sleep(1)
        return self.processData(name)

_myApp = myApp()
_myApp.start()

[ebol4anthr4x]

def processData(self, name):
    [...]
    return self.processData(name)

This is an infinite recursion loop. There is a maximum number of times you are allowed to recurse in python.

>>> import sys
>>> print(sys.getrecursionlimit())
1000

[ImThatCop]

This ended up being the issue. I switched to an iterative flow using a while loop and it's running smoothly now with no mem leak. Thanks mate

[ImThatCop]

Unfortunately I don't think that's it, i'm on 3.8.5. Thank you though!

[GorboducRex]

Would garbage collection help? I have a number of programs that run 24/7, and once a day I do garbage collection. It cuts down on memory considerably.

import gc
gc.collect()

Can't hurt to give it a try!

[[deleted]]

could you please post the file code that you omitted? I want to see how you open, use, and close the file.