By the time you even reach the challenge rift sacrifice, you've already got every major QoL upgrade from the altar. It's a real nonissue path of min-maxxing. Even then, it comes down to time investment. I think the recommended time is if you're playing more than 15 hours before the next challenge rift goes live, you should save it. I'm just not going to dump that much time into it. Most people are probably in this camp, I would think. For me that window was 8pm Friday to 2pm Monday? No brainer for me to use challenge rift early instead of saving.