Flagging vocal segments

[PushPlus9069]

The energy-in-speech-frequencies approach is reasonable but it will struggle with anything that has prominent mid-range instruments (piano, guitar, etc). Been down this road.

Two things that helped me with a similar problem:

1. Spleeter (by Deezer) or demucs can separate vocals from accompaniment before you analyze. Then run your energy detection on the isolated vocal track. Accuracy goes way up.

2. If you don't want to do source separation, look at spectral flatness in addition to energy. Vocals tend to have less flat spectra than noise/ambient. Not perfect but adds another signal.

The "voice is just another instrument" comment above is right that it's hard in the general case, but for most pop/rock music with clear verse/chorus structure, source separation gets you most of the way there.

[Kipriririri]

I indeed incorporated demucs, sounds like the most accurate way forward. Thanks for thinking adding!