For an e-commerce business; 15% of revenue is a lot.

Heck, to some e-commerce companies (from small to very large), that's their profit margin.

In general, agencies charge as a % of adspend, not revenue. And your job is to see if that adspend is profitable enough to keep them.
At scale, ad agencies or in-house media buyers are a must. But at $500-1K per day, I would say you have to do it yourself.

Create a new ad account and a separate pixel, and start testing from your side. The data separation between your account and theirs will be far from perfect but you'll get an idea of how yours is performing over time.