diffsptk

diffsptk is a differentiable version of SPTK based on the PyTorch framework.

Requirements

• Python 3.10+
• PyTorch 2.3.1+

Documentation

• See this page for the reference manual.
• Our paper is available on the ISCA Archive.

Installation

The latest stable release can be installed through PyPI by running

sh pip install diffsptk

The development release can be installed from the master branch:

sh pip install git+https://github.com/sp-nitech/diffsptk.git@master

Examples

Running on a GPU

```python import diffsptk

stftparams = {"framelength": 400, "frameperiod": 80, "fftlength": 512}

Read waveform.

x, sr = diffsptk.read("assets/data.wav", device="cuda")

Compute spectrogram using a nn.Module class.

X1 = diffsptk.STFT(**stft_params, device="cuda")(x)

Compute spectrogram using a functional method.

X2 = diffsptk.functional.stft(x, **stft_params)

print(X1.allclose(X2)) ```

Mel-cepstral analysis and synthesis

```python import diffsptk

fl = 400 # Frame length. fp = 80 # Frame period. n_fft = 512 # FFT length. M = 24 # Mel-cepstrum dimensions.

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Compute STFT amplitude of x.

stft = diffsptk.STFT(framelength=fl, frameperiod=fp, fftlength=nfft) X = stft(x)

Estimate mel-cepstrum of x.

alpha = diffsptk.getalpha(sr) mcep = diffsptk.MelCepstralAnalysis( fftlength=nfft, ceporder=M, alpha=alpha, n_iter=10, ) mc = mcep(X)

Reconstruct x.

mlsa = diffsptk.MLSA(filterorder=M, frameperiod=fp, alpha=alpha, taylororder=20) xhat = mlsa(mlsa(x, -mc), mc)

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error)

Extract pitch of x.

pitch = diffsptk.Pitch( frameperiod=fp, samplerate=sr, fmin=80, fmax=180, voicingthreshold=0.4, outformat="pitch", ) p = pitch(x)

Generate excitation signal.

excite = diffsptk.ExcitationGeneration(frame_period=fp) e = excite(p) n = diffsptk.nrand(x.size(0) - 1)

Synthesize waveform.

xvoiced = mlsa(e, mc) xunvoiced = mlsa(n, mc)

Output analysis-synthesis result.

diffsptk.write("voiced.wav", xvoiced, sr) diffsptk.write("unvoiced.wav", xunvoiced, sr) ```

WORLD analysis and synthesis

```python import diffsptk

fp = 80 # Frame period. n_fft = 1024 # FFT length.

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Extract F0 of x, or prepare well-estimated F0.

pitch = diffsptk.Pitch( frameperiod=fp, samplerate=sr, fmin=80, fmax=180, voicingthreshold=0.4, outformat="f0", ) f0 = pitch(x)

Extract aperiodicity of x by D4C.

ap = diffsptk.Aperiodicity( frameperiod=fp, samplerate=sr, fftlength=nfft, algorithm="d4c", out_format="a", ) A = ap(x, f0)

Extract spectral envelope of x by CheapTrick.

pitchspec = diffsptk.PitchAdaptiveSpectralAnalysis( frameperiod=fp, samplerate=sr, fftlength=nfft, algorithm="cheap-trick", outformat="power", ) S = pitch_spec(x, f0)

Reconstruct x.

worldsynth = diffsptk.WorldSynthesis( frameperiod=fp, samplerate=sr, fftlength=nfft, ) xhat = world_synth(f0, A, S)

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

LPC analysis and synthesis

```python import diffsptk

fl = 400 # Frame length. fp = 80 # Frame period. M = 24 # LPC dimensions.

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Estimate LPC of x.

frame = diffsptk.Frame(framelength=fl, frameperiod=fp) window = diffsptk.Window(inlength=fl) lpc = diffsptk.LPC(framelength=fl, lpc_order=M, eps=1e-5) a = lpc(window(frame(x)))

Convert to inverse filter coefficients.

norm0 = diffsptk.AllPoleToAllZeroDigitalFilterCoefficients(filter_order=M) b = norm0(a)

Reconstruct x.

zerodf = diffsptk.AllZeroDigitalFilter(filterorder=M, frameperiod=fp) poledf = diffsptk.AllPoleDigitalFilter(filterorder=M, frameperiod=fp) x_hat = poledf(zerodf(x, b), a)

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Mel spectrogram analysis and synthesis

```python import diffsptk

fl = 400 # Frame length. fp = 80 # Frame period. nfft = 512 # FFT length. nchannel = 128 # Number of channels.

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Compute STFT amplitude of x.

stft = diffsptk.STFT(framelength=fl, frameperiod=fp, fftlength=nfft) X = stft(x)

Extract log-mel spectrogram.

fbank = diffsptk.FBANK( fftlength=nfft, nchannel=nchannel, sample_rate=sr, ) Y = fbank(X)

Reconstruct linear spectrogram.

ifbank = diffsptk.IFBANK( nchannel=nchannel, fftlength=nfft, samplerate=sr, ) Xhat = ifbank(Y)

Reconstruct x.

griffin = diffsptk.GriffinLim( framelength=fl, frameperiod=fp, fftlength=nfft, ) xhat = griffin(Xhat, out_length=x.size(0))

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Subband decomposition

```python import diffsptk

K = 4 # Number of subbands. M = 40 # Order of filter.

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Decompose x.

pqmf = diffsptk.PQMF(K, M) decimate = diffsptk.Decimation(K) y = decimate(pqmf(x))

Reconstruct x.

interpolate = diffsptk.Interpolation(K) ipqmf = diffsptk.IPQMF(K, M) x_hat = ipqmf(interpolate(K * y)).reshape(-1)

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Gammatone filter bank analysis and synthesis

```python import diffsptk

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Decompose x.

gammatone = diffsptk.GammatoneFilterBankAnalysis(sr) y = gammatone(x)

Reconstruct x.

igammatone = diffsptk.GammatoneFilterBankSynthesis(sr) x_hat = igammatone(y).reshape(-1)

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Fractional octave band analysis and synthesis

```python import diffsptk

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Decompose x.

oband = diffsptk.FractionalOctaveBandAnalysis(sr) y = oband(x)

Reconstruct x.

x_hat = y.sum(1).reshape(-1)

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Constant-Q transform

```python import diffsptk import librosa # This is to get sample audio.

fp = 128 # Frame period. K = 252 # Number of CQ-bins. B = 36 # Number of bins per octave.

Read waveform.

x, sr = diffsptk.read(librosa.ex("trumpet"))

Transform x.

cqt = diffsptk.CQT(fp, sr, nbin=K, nbinperoctave=B) c = cqt(x)

Reconstruct x.

icqt = diffsptk.ICQT(fp, sr, nbin=K, nbinperoctave=B) xhat = icqt(c, outlength=x.size(0))

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Modified discrete cosine transform

```python import diffsptk

fl = 512 # Frame length.

Read waveform.

x, sr = diffsptk.read("assets/data.wav")

Transform x.

mdct = diffsptk.MDCT(fl) c = mdct(x)

Reconstruct x.

imdct = diffsptk.IMDCT(fl) xhat = imdct(c, outlength=x.size(0))

Write reconstructed waveform.

diffsptk.write("reconst.wav", x_hat, sr)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

Vector quantization

```python import diffsptk

K = 2 # Codebook size. M = 4 # Order of vector.

Prepare input.

x = diffsptk.nrand(M)

Quantize x.

vq = diffsptk.VectorQuantization(M, K) xhat, indices, commitmentloss = vq(x)

Compute error.

error = (x_hat - x).abs().sum() print(error) ```

License

This software is released under the Apache License 2.0.

Citation

bibtex @InProceedings{sp-nitech2023sptk, author = {Takenori Yoshimura and Takato Fujimoto and Keiichiro Oura and Keiichi Tokuda}, title = {{SPTK4}: An open-source software toolkit for speech signal processing}, booktitle = {12th ISCA Speech Synthesis Workshop (SSW 2023)}, pages = {211--217}, year = {2023}, }