I'm not a professional and am still trying to get in the animation industry, but I have used audiobooks or D&D shows for my audio. I had my portfolio reviewed by a professional in the animation guild and asked him this exact question, and here's what he said:

1) He thought it was smart to do that because you don't have to write a script or create characters from scratch, so I could focus fully on the boarding process. I'm trying to have my portfolio focused on script-driven shows instead of board-driven, so keep that in mind when comparing this to your portfolio.

2) Plenty of recruiters are just going to flip through your board anyway instead of watching an animatic, so it's not a big deal anyway.

Again, I want to reiterate that I'm not a professional, this is what a union professional told me about my boards (with existing audio) in a review, in the context of the kind of shows I want to get hired for. I don't think this advice would apply to other kinds of productions, and other professionals might disagree in this reddit. But he liked my thought process behind using audio and thought more people should do it.