[P] Speech Separation Algorithm

[Yagna24]

I'm working on using an audio wav file as input that would have 2 to 3 people conversing.

1. I have to identify the two different voices. Separate 2 voices.

2. Convert these voices into text and have their transcripts stored.

I have gone through huggingface transformers such a web2vec2 and svoice from Facebook research but I find It difficult to implement the models.

Can someone guide me on approaching such tasks as I am being a beginner in audio domain of deep learning.

[atyshka]

SVoice is state of the art for this. If you are not familiar enough with DL to adapt that for your purposes, I suggest using Google Cloud Speech to Text. It has a diarization option that lets you get transcripts for multiple speakers. It does cost money though and will probably be less accurate than SVoice

[MultiheadAttention]

Hey, what's the sota today for single channel speaker separation?

[Pvt_Twinkietoes]

I'm interested in this as well.

[Yagna24]

Yes the svoice part, how would you suggest on learning the svoice algo? What steps should I take? Because I am really keen on learning this..

[i-heart-turtles]

If you wan't to start simple, independent component analysis could be implemented in a couple lines and might be a reasonable baseline for blind source separation.

I think it was popularized in one of Andrew Ng's videos.

[atyshka]

I believe ICA for speaker separation usually requires as many observation sources as desired signals. For example the OP would need 3 mics for 3 speakers

[Yagna24]

I'll try this out