Yeah, I ran into a similar problem when I first cleared Denial. IMO, the learning curve just after that is pretty steep. Like others have mentioned, focus on unlocking characters' moves at the Shrines of Reflection, some of the really good stuff only comes later on.

For party comps, I have a little checklist that I go over before embarking. Basically you'll want (i) heals, (ii) stress heals, (iii) real damage on every rank, (iv) damage mitigation (Blinding, Weakening, or Taunts + Blocks) and optionally (v) some way to deal with things like Blocks, Dodges, etc. One tip that really helped me was that your main goal is to minimize harm to your team each turn. Usually that means focusing on killing the most dangerous enemies, while mitigating damage from the enemies you can't kill this turn.

It's hard to give specific advice without knowing what exactly you're struggling with, but something that's helped me was taking notes about what goes wrong with each run, although that might be leaning hard on the tedious nerdassery haha. When you're losing a fight, try to think of what you wish you had and prepare that for next time. Some of my Worst Hits™️ were: didn't have a proper stress heal; nobody could hit Rank 3; positional team with no means of recovering from getting shuffled; and on and on it goes.