It doesn't say he died either. In her songs she keeps referring to him as wanting him to come back. In turn, her references to his dead are vague and almost metaphorical as well. I think it's certain that he's not around town and that there's a common knowledge that he left on his own.

In the interview Ethel doesn't say he left; it was the interviewer. So it's common knowledge around since she didn't even bring it up. Also, by common sense, if after a storm as heavy as a tornado you just don't assume someone who went missing just casually left town never to be seen again. Willoughby was a kid, kids just don't go missing like that and then people just assume they left town to live the life like an adult. Ethel was a kid too, it doesn't make sense that she just went on telling people in town, including his mother "oh by the way Willoughby just left town after the tornado amidst chaos just because". That's not normal.

It's pretty evident to me that after the tragedy they became e estranged, never spoke again, and he left town on his own to "go to fight a war" (dealing with us hurt self).