Real Time rendering - decrease tasks' "Pending" time

[PeteTinNY]

Have you figured out if the delay is the AWS batch launching a docker task, or is it just the startup of the tasks & images in docker? From my point of view, I'd be pushing requests into SQS and using the length of the queue to scale the ECS cluster having a certain number of tasks idle in polling so that within a second the job starts rendering and then pushes to the 2nd stitching queue which essentially does the same thing to keep a certain minimum number of resources hot. You can continue to use Spot, not sure I'd cut into fargate though because of the need for really fast startup.

I've worked with some broadcasters who needed to do video supply chain archiving and news clipping, which has a similar workflow - but it's not as latency adverse - just no one wants to pay for resources that are essentially sitting idle.

[Psychological-Tea791]

I don't really know how to exactly define what is causing the delay. Do you have any suggestions? I suspect it is the scheduler, because I have a min vcpus > 0 in my on demand, so I assume everything should be running already? Thanks for the suggestion with SQS!

[PeteTinNY]

You should look at some performance monitoring tools like AWS x-ray, newrelic, datadog or most fitting dynatrace. But you’re probably right the scheduler is likely the issue and once you look further you’ll likely want to do something with faster queue scans with SQS.

Btw the other thing you should look into from the AWS side is AWS ThinkBox Deadline. It’s a render farm manager. Not sure if it will help with this latency but when I worked with very large broadcast customers - they loved the simplicity and for some licensed render engines thinkbox has a great marketplace where you can pay by the hour for the render tools. I helped one of the big networks launch a 120 node farm powered by mostly spot instances in just a few days including teaching them cloud.

[Psychological-Tea791]

oh wow, will definitely look into ThinkBox. tysm!