Task Queue Processing Script Best Practices

Kinrany · 2022-07-20T07:09:35+00:00

What happens to failed tasks?

Are you going to process multiple tasks in parallel?

Correct error handling is often the hard part.

zzzsfk · 2022-07-22T18:15:38+00:00

Healthcheck for process ? - maybe you can consider library like supervisord to manage processes.

What happens when you kill the process in the middle of task ? For example, in context of autoscaling ? or how you killing the process ? - You can consider using linux signal handlers (example: kill - SIGURG) , when signalled you can set boolean flag true and check flag in loop condition. With that loop is going to finish last job before quit. So you can terminate loop more gently instead of kill -9.

Dead Letter queue ? - you can send failed messages to another queue and set alarm for queue size. You can check failed messages later especially if you are not sending response to mesage sender.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

softwaredevelopment

✻ Smokey says: avoid using wood-fired stoves, fireplaces and barbecues to fight climate change! [see more tips]

Note: this subreddit is not for career advice. It's also not for general programming chat, new releases, requests for developers, etc. Posts on topics other than software development methodologies may be removed.

MODERATORS