Representations for speech synthesis

01 / MOTIVATION

The question

How should a synthesis system represent linguistic context and the spectral envelope of speech?

02 / METHOD

The approach

The SSW 2016 study augments conventional text features with hidden states from an RNN trained to predict acoustic features. It evaluates this acoustically guided context representation in DNN-based synthesis for Hindi and Telugu using Blizzard Challenge 2015 data.

The 2015 spectral-envelope study uses maximum spectral amplitudes in sub-bands (MSASB) to parameterize the envelope directly in the spectral domain. The representation is incorporated into the STRAIGHT vocoder and explored in a deep-neural-network synthesis framework.

03 / EVIDENCE

Findings & scope

The contextual-representation study reports improvements over its baseline DNN system in both objective scores and subjective listening tests for the evaluated Hindi and Telugu data.

The MSASB study reports analysis-by-synthesis performance comparable to STRAIGHT in objective and subjective evaluation. Its direct spectral parameterization offers a different representation from cepstral-domain alternatives.

These are complementary representation studies, rather than a single end-to-end benchmark. The publications below describe the individual methods and evaluations.