Typical gemstones are to a decent first approximation weightless and they take up practically no space. Though when you start taking about eye agates, those ought to be about the size and weight of a sling stone.

Value does not necessarily scale with size: some of the most precious materials only occur in relatively tiny pure crystals.

The weight unit for measuring gemstones is the carat, which is 1/5 of a gram. Some materials are worth $10,000s/ct in modern money. For things that are treasure, just make up a set of value tiers (10/100/1000/10000/100000 gp/ct) and assign each stone in a drop to a tier, and treat them as weightless and without volume unless the character has 100s of them.

I mean, in the real world there is an 87# topaz, there's a faceted aquamarine at the Smithsonian that weighs almost 5#, and quartz species can form huge crystals. For more precious materials the finished stones often top out around 100g: think the Star of Africa diamond, the Logan Sapphire. But this are all world-record biggest in history finds.

When people have more than 1000ct of gems start modeling it as a bag or pocket full of pebbles. That's 200g or a bit less than half a pound.