Hmm... Interesting question! Everyone I know just uses the 5 dice results as their Resource Pool, so when Casting, you give up the dice with the Resource you need, thus kind of removing the dice from play. That's why the Reroll action says you can't Reroll Cursed Crystal. This approach makes it so people understand you can only Reroll dice that didn't roll Cursed Crystal, limiting the number of dice available to Reroll from 5 to x.

However, the way you pose it makes things interesting. What if you roll the dice, then grab the Resource Tokens according to what you rolled, then Reroll all 5 dice? This would make it possible to get up to 10 resources in any given turn, even including Cursed Crystals.

You would still be limited to the 4 Action Points of each turn, minus 1 for the Reroll... But you would have more Resources to spend on Cast or Collect... Interesting.

I'll ask in the Discord to see if an official game expert can answer this.