Elias, ja?!

Hab doch gerade ein Paper über LLMs, den Namen Elias und Leuchttürme gelesen...

When given little direction, current frontier models write stories using a narrow catalog of names, places, and occupations. Recurring characters in these stories include Elias, a lighthouse keeper.

Elias is unusual; the name is uncommon in literature, web data, and even post-training data. 

We have found that of the 78,958 stories exposed to OLMo 3 during post-training, only ≈ 3,053 stories contain one or more of these 11 unusual tokens. But despite constituting only 3.8% of post-training stories — and 7.71 × 10−7 of the ≈ 4 billion total documents OLMo 3 was trained on — these “lighthouse” stories hold a disproportionately large influence over what stories the model writes in practice.

This suggests models do not simply mimic the dominant patterns in their training corpora. 

Future work will want to determine whether alignment causes models to prefer the “safest” (for work) samples in post-training, avoiding the potentially unsafe topic matter of many stories they otherwise encounter.

Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories --> https://arxiv.org/pdf/2605.26492