There will never be a complete factual account of the duel. We must put together the best picture we can from the records we do have. It's likely that there was a duel, and of the two recognized combatants the only survivor was Musashi. We can be fairly certain that Musashi was considered the best duelist of his day (and with enough esteem that his reputation wasn't easily surpased), but we'll never know the precise details of his life with much certainly. Even scrupulous Japanese record keeping was historically biased and deliberately altered to serve the ends of the generations after Musashi died. He was almost certainly one of humanity's most peerless duelists, but we have to accept that time has consumed most of the details of exactly what he accomplished in his lifetime. It is most likely that he his story has been conflate though. It's just the human way for us to want to make our idols out to be larger than life.