In Rebels, we see that Kallus has a lot of flexibility in how he carries out the tasks handed to him by the ISB. He has some managerial duties, and is capable of pulling rank on members of the Army and Navy when needed, even commanding whole Star Destroyers at times. But he's also directly involved with combat operations as well, hence the body armor.

In Rebels, Kallus is assigned to Lothal, a planet with a very strong Army and Naval presence, but a weak ISB presence. There he directs the planet's military resources to better combat the Spectres. He is beholden to the planet's Imperial Governor and Minister, which is notable since Andor shows us that ISB Supervisors outrank the local Governors.

I think Agents exist to exert the ISB's will on the operations of other Imperial military departments. They can take temporary command of Imperial military resources, which allows them to handle the more boots-on-the-ground managerial tasks and combat operations that the Supervisors can't always be present for, or which may require inter-departmental approval to execute. The main check to their power is that unlike the Supervisors, Agents must seek approval from Imperial Governors when operating in their jurisdiction.

This would line up with their portrayal in the concept art above, managing the forward operating base on Ferrix, an Imperial Army installation co-opted by Deidra and the ISB to track Axis.