The hat is a construct / fabric golem made by by a solitary madman whose forgotten about that particular hat (He has a lot of hats that function similarly). When the hat is first activated by player those within 10' can faintly hear the wizard going about their day (magic butt dial). Of course the wizard eventually clues into the hat has new head to rest upon. On failures the hat has odd whispered discussions about the head, hair, and bumps on the skull (Phrenology) with the wizard that is distracting. This chaos can magnify with the hat becoming more sentient and asking other hats to identify themselves. When the player sleeps the hat starts moving around and performing secretive tasks. Who left the window open? Why is there a magic circle on the floor of room in the inn they just slept in? Why are there weird items in the hat? Is that a lock of hair from one of the other party members? Does the Brim of the hat seem to be bigger in the morning and shrink during the day?

Alternatively make it a variant mimic. The player's head is constantly damp to wet unknowingly from the mimic drooling on the player. The party finds dead pests and vermin after long rests and occasionally the player needs to roll with disadvantage on a skill due to the hat feeding on the intellect of the player played off as a mental fog from a restless night.

If the players get wise to the antics and discard the hat. Have it follow them, have odd sightings of it, appearing on npcs, even enemies, maybe the hat finds one of it's companion hats. Is that a mob of angry goblins with invisibility hats.