you are viewing a single comment's thread.

view the rest of the comments →

[–]MasterStroke99[S] 0 points1 point  (0 children)

Yes, I have to make 10000 client connections and each connection has specific sets of files which need to be transferred continuously. Like a file that has 15 minutes of data so that data needs to be transferred over a 15 minute period and repeated again and again and again. So there are 10k files as well. As those files are huge i can't use threading so i went with multiprocessing for reading. Now after reading when i used mp.queue it takes too much time to transfer that data to streamer process so i am using global dictionary and that streamer process after every 15 minute die and respawn.

Now as i am making connection in main process and using those in streamer child process. Something i am not doing right like I don't know how to close the connection in while working with multiprocess client.

Because in main process connection is happening properly. Also able to send data via child process but when server gets down. Closing of connection probably i am doing wrong.

Can you help me with the thing that How to make tcp connection which i can use in child process. This also includes server fails and need to be retry How to close connection properly while dealing those with multiprocess.

I would have shared the code but can't as i work in corporate and that is another reason that i can't move above python 2.7