A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet

[jmvalin]

This is a follow-up on the first LPCNet demo. In this new demo, we turn LPCNet into a very low-bitrate neural speech codec (see submitted paper) that's actually usable on current hardware and even on phones. It's the first time a neural vocoder is able to run in real-time using just one CPU core on a phone (as opposed to a high-end GPU). The resulting bitrate — just 1.6 kb/s — is about 10 times less than what wideband codecs typically use. The quality is much better than existing very low bitrate vocoders and comparable to that of more traditional codecs using a higher bitrate.

Read More

Comments

Opus

[(Anonymous)]

In your left to do paragraph, you mention maybe this could be used in Opus. Could this technology also be used to enhance music at lower bit rates, say 20kb/s and achieve similar quality to a higher bitrate of say 64kb/s? Also, is there any samples at to what music would sound like (I know this is for speech so far, but would be interesting to see what happens to music at 1.6kb/s. Cheers, Kirk

Re: Opus

[jmvalin]

This work is not directly applicable to music because music is much more complex (speech has much fewer variability). There could be something for music eventually, but we're not there yet.

Re: Opus

[(Anonymous)]

I was thinking the same, but thought I would double check. It would be great if this ends up in Opus and can be used for say a podcast where speech / music detection takes place to mix music with current opus WB codec with higher bitrate in with the LPC net stream at 1.6kb/s. I hope that makes sense. Should yield very small files for distribution. Great work by the way. Extremely impressive result so far.