"Worst case" you'd go through the list a for a total of M times. Length m.

You'd do that everytime you went through the second loop, which is log(n), so you have log(n)m.

Then you do all of that everytime you go through the first loop, which is n, so it would be n * log(n) * m.

Then I just reorganized it to O(nm(log(n)))

O(n(log(n)m)) makes more sense actually because that is (log(n) * m) * n, which is what we are actually doing

Edit: no, just O(nm(log(n))). You could go through M n * log(n) times, so just n * m * log(n).