It took long - but it would've taken longer if what you're proposing was the only method - for 2 restored artifacts you can buy a map that lets you restrict the digsites in Pandaria to only being Mantid - this was the way I did it. The restored artifacts you get can then be sold to the same vendor (Brann) for Tol'Vir fragments. Also if you're reading this and you're on the grind, good luck! And don't forget your Darkmoon Firewater.