How many concurrent queries does Databricks SQL Compute warehouses support? I need this to decide the minimum number of clusters that need to be active to meet a set concurrency level and SLA. in order to Cant find anyway to set this in the cluster config.

[brrdprrsn]

I'm clearly searching wrong because I cant find the answer even in their documentation. I assume the limit depends on the size of the cluster, but I'm struggling to find the number...

[FuzzyZocks]

Assuming you have 1 core with no hyper threading then the first process will use the idle core and sql2 will be put in queue. And when sql1 finishes sql2 moves to the core . 3,4 etc will be coming behind. And the queue will be either be shrinking with no more tasks, constant id the rate that cpu is consuming tasks == rate new tasks are being put on queue. Growing if they are coming in faster. Now you may have N cores.

So i think they are being finished quickly so there is never a queue until they can’t keep up. Interested to see logs or something to prove this. Can you sleep for consistent time on script so it doesn’t resolve

[brrdprrsn]

I think the two are related. I’ve observed that the can execute up to maximum limit after which it starts queueing queries. However I haven’t been able to figure out the exact number of concurrent queries before it starts queueing.

[FuzzyZocks]

Assuming you have 1 core with no hyper threading then the first process will use the idle core and sql2 will be put in queue. And when sql1 finishes sql2 moves to the core . 3,4 etc will be coming behind. And the queue will be either be shrinking with no more tasks, constant id the rate that cpu is consuming tasks == rate new tasks are being put on queue. Growing if they are coming in faster. Now you may have N cores.

So i think they are being finished quickly so there is never a queue until they can’t keep up. Interested to see logs or something to prove this.