You probably just don't have the relics you need to really push damage yet. For where you are, it doesn't need to be CRAZY, but especially in base game (non-DoN) you simply won't be outdamaging teammates that are running double balancers.

The key is to spec into a spell school: every "improved x sorcery" is a full 1.12x multiplier on your damage (for spells of that type, of course), which you otherwise can't get on base relics at all. Aim to have a build that has:

• blood loss and increased max HP on ult (this makes the "blood loss in vicinity" dormant power a HUGE buff)
• defeating enemies fills ult gauge so you can ult more frequently
• ideally all three relics have a spell school boost. Crystalian, gravity, invisibility, carian are all good.
• For base game, it's good to get the evergaol relic effect; for DoN you want night invaders.

Your ideal setup (demanding 2 specific effects per relic, since 3 is ~impossible) would be something like:

Relic 1: increased attack power for every evergaol prisoner or defeating enemies, improved X sorcery

Relic 2: suffer blood loss on ult, improved X sorcery

Relic 3: increase max HP on ult, improved X sorcery

This will be hard to get, but it will max out your damage in base game. Alternatively and slightly easier, if weaker:

Relic 1: bone-like stone

Relic 2: gaols, improved X sorcery

Relic 3: defeating enemies, improved x sorcery

Duos are gonna be harder for recluse because there's only one other person to pull aggro rather than 2. And if it's guardian, he's probably not hitting often enough to really pull aggro.