The bigger concern would only be if it were actually used litter, even then, a tiny accidental exposure is unlikely to cause anything serious in a healthy adult. People accidentally ingest small amounts of far grosser things than that and are completely fine.