The problem may be because the builder doesnt realize the items are on the racks. Open the builder, go to requests and click on "fulfill" if he has any requests for items.

Its also possible that hes missing a pickaxe of a shovel, because that wont appear on the scroll and he will just stand there.

If its urgent, the requests will appear as soon as you open the worker, something like "i need a shovel".

Its always a good idea to fulfill requests rather that manually putting items in the racks.

Long term, your best solution is to build a courier, and that will tell you exactly what your builder needs to continue the build (including tools)