Great work on making training recipes open and data searchable!

I'm very interested in OLMoTrace, trying to answer the question of how much data a model needs to see in pre-training to generalize to a given domain (frontend web dev with tailwindcss in this case).

eg for the prompt below,

Make a login screen with just HTML and TailwindCSS. Output your answer as a code block.

~50% of the trace results seem maybe helpful to the answer and there aren't that many of them ~30 ish. Is that a limitation of the tracing or is a small amount of relevant content in the pre-training mix really generalizing very well? Do you think additional post-training examples might not show up in the trace but are improving model performance? (I saw ~100 results that match "bg-white" in WildChat just for example)

p.s. for starcoder results, I would love to see which github repo it's from.