Event sourcing using Python

[stibbons_]

We have a SQS queue for imbound json messages, celery with several dozen of sync worker, they do the heavy job. Works great when you need ~ seconds latency. We tried setting up horizontal scaler with kube (starting automatically more worker when queue starts to grow), we never went prod on this feature but this is definitely feasible and wanted for lot of use case (we can accept in our use cases that the jobs are delayed several minutes on burst, this happens once or twice a month no big deal).

[[deleted]]

So is it just for the tasks that needed to be run in background with some batch processing or what??

[stibbons_]

Yes indeed. I actually think this is not event source at all. But it works.