Analysis/synthesis is a paradigm of audio processing. In this paradigm, an audio signal is analyzed to produce a model of the signal. Then this model is transformed according to our wishes. Finally, the modified model is synthesized to produce the transformed signal. This paradigm can be used not only for transforming signals, but also for synthesizing them from scratch by means of selecting prerecorded samples from a database. Also, another method is omit the analysis stage and generate the model using statistics or neural networks.
In analysis/synthesis, for any real (i.e. non-synthetic) signal, there will always be a portion of it which cannot be represented well by the model. The exact composition of this component depends on exactly what the model itself models, but in this post I will be focusing on what can not be modeled by any model, that is, stochastic components.
Broadly, even if they are not specifically implemented this way, analysis/synthesis techniques that consider the stochastic part due so by first separating out the part that cannot be modeled by the model. Although the remaining component is stochastic, it is not strictly random. Usually, this means it is either tied to the model, or a separate model that describes a non-random aspect of the stochastic component. For example, a typical case is that the spectrum of then stochastic part is not flat as it would be in white noise, but follows an envelope. In that case, the extraction of just the random part would be done by inverse filtering the stochastic component by that envelope, a process known as spectral whitening.
From there, there are basically two ways to proceed. Either the random signal is preserved, or it is discarded and replaced with white noise at synthesis. The former has the advantage that, depending how the rest of system is implemented, it may be possible to reconstruct the original signal perfectly when no transforms are applied. Additionally, in practice, the 'random' component will in practice always include some amount of signal, so discarding loses some aspect of the original signal. On the other hand, the latter has the advantage of flexibility. Often, transforming the model requires that we now have more or less samples of the random part.
One option would be to interpolate the random component. However, when more samples are required than there were originally (i.e. upsampling), the new signal is smoother than the original. This can be a significant problem, for example if we are slowing down a signal by large factor.
From signal theory, we know that all other interpolation methods approximate the ideal sinc-convolution interpolation. In the frequency-domain, that method corresponds to there being no new spectral components above the old Nyquist frequency when upsampling. Given that our random component is supposed to be white noise, i.e. that it should have a flat spectral envelope, we can now see where the smoothness problem comes from.
Thus, from this observation, I propose a hybrid model. We preserve the original random component, with all the advantages that brings. When requiring a different amount of samples than there were originally, we resample it. However, in the case of upsampling, we replace the new spectral bins with random values instead of zeros.
(continued in next post)