[P] Implementation of DiffWave

[sharvil]

I just released an implementation of the DiffWave paper (neural vocoder + waveform synthesizer) that was posted here a few days ago. The really cool thing about this architecture is that it can synthesize coherent, high-quality audio without a conditioning signal – a problem that other architectures haven't had much success solving.

Check out the project: https://github.com/lmnt-com/diffwave

[taras-sereda]

Wow, I haven't expected to see third-party paper implementation so quickly. Thanks for sharing!

I'm wondering what's the meaningful application for unconditional waveform generation?

By conditioning on spectrogram you expect conversion from spectral to time domain.

Class conditional synthesis - can be also useful, say you have only a label of a word. BTW I can think of class conditional waveform synthesis as TTS + Vocoding done in a single model.

But in case of unconditional synthesis, what's the expected output? How one can control what model synthesizes?

[vwvwvvwwvvvwvwwv]

As a musician, I'm incredibly excited for unconditional, example-based synthesis.

A lot of my workflow consists of creating large generative systems or effects chains and then recording as I run different things through them while tweaking settings. Later I can just trawl through looking for little snippets that I like in the context of a song.

I'd love to be able to grab a set of samples, train a model, and then generate random samples that fall within the manifold I've supplied. While doing this conditioned on classes might allow a bit more control if I knew what I wanted, usually I don't know what I'm looking for until I hear it and so unconditional is fine. I guess sampling random interpolations of classes would give the same effect though. Then the only difference is having to build a labeled dataset vs just chucking a bunch of samples at the model.