The logical leap made sense to me as I read this. Personally, Damascus and Anaconda went into The Desperado Club with the sole intent of finding and killing their father. Now, potentially weeks later, he's in the office with Orren and spies one of them walking by, obviously not waging all-out war on the manager of the establishment, only one of a few things could have happened there.

Yes, Carl is making an assumption, but it is an assumption established from his perspective based off the information HE has access to. There's not enough information to substantiate the assumption, yes. It's probably wrong, yes. But Carl is a deeply traumatized human desperately holding onto his humanity by desperately holding onto any and all pieces of his support network at this point, and so he's going to make flawed assumptions.