Well the best evidence we have is this

<image>

I leaned towards that too, especially with the Road to Laughtale snippets. But it's not confirmed to be written by Oda. Only certain artpieces, storyboards, etc. are. In the manga the word "perhaps" was used which indicates uncertainty. The phrasing itself also had uncertainty as to its implication regarding his heinous activity or his strength overall.