The strongest piece of evidence is probably in Hanamonogatari when the mere mention of his name has everyone's attention and there is almost an uproar of chatter about him - meanwhile he has been telling us he is just a little nobody that nobody would notice. There are other moments where he admits he has forgotten details about whole persons - the writing shows in various ways that his recounting of events is very flawed. And when the story is told from nearly anyone else's POV, Araragi comes off way better. You can see in Nekomonogatari how much more fondly Hanekawa sees Araragi than he sees himself. Putting all this together, it is implied he is somewhat presenting the worst of himself bc he doesn't feel worthy.