This course provides a collection of fundamentals tools for audio and speech signal analysis and processing.
A wide range of applications will be considered, including musical sound synthesis, sound classification and music information retrieval, sound localization and tracking, 3D audio, speech processing and coding, etc.
Introduction
Motivation of digital audio processing. Course and exam description.
Review of the basic concepts of DSP
Review of the Fourier Transform for Discrete Time Signals.
Review of the Zeta transform.
Introduction and main properties of the DFT.
FIR and IIR filters.
Elements of acoustics and psychoacoustics
Introduction to acoustics and sound propagation.
Elements of psychoacoustics. The hearing system.
Critical bands. Consonance and dissonance.
Notes, harmony, pitch, timbre.
Tuning systems.
Introduction to room acoustics.
Sound analysis tools
A/D conversion of the audio signal. The Discrete Time Fourier Transform (DFT). Windowing. The Short-Time Fourier Transform (STFT).
Sub-band decomposition and STFT.
Sound synthesis and modeling
Resolution of sinusoidal signals and windowing, Spectral interpolation,
Additive synthesis, sinusoidal peaks tracking, sinusoidal modeling.
Sound synthesis: additive and subtractive synthesis, granular synthesis, wavetable synthesis, nonlinear distortion (phase/frequency modulation).
Introduction to physical sound modeling for timbral synthesis.
Feature extraction
Low-level descriptors (Time Domain, Spectral and Timbral), Sound
Classification and similarity.
Audio features. Time-frequency localization. Pitch features. Chroma features.
Digital audio effects and 3D audio
Modulated digital delay lines, sound effects, reverberation and spatialization algorithms, binauralization, head-related transfer function, multi-channel processing, ambisonics.
Equalization: typical structures for equalization.
Fundamentals of microphone array signal processing.
Introduction to the speech signal
Acoustic theory of speech production. Analysis and modeling of the speech signal. Introduction to speech recognition, synthesis and coding.
Concluding remarks and examples
Audio retrieval. Cover song identification. Music representation. Introduction to audio and speech coding (MP3, AAC, ...). Concluding remarks.
David M. Howard, Jamie A. S. Angus, Acoustic and Psychoacoustics, Elsevier - Focal Press, 2009
Udo Zölzer et al., DAFX - Digital Audio Effects, John Wiley & Sons, 2002
Meinard Muller, Information Retrieval for Music and Motion, Springer, 2010
M. Kahrs, K. Brandenburg, Applications of digital signal processing to audio and acoustics, Kluwer, 1998
Class notes, lecture overheads, readers, tutorials. Recommended supplementary reading.
Further readings
Hugo Fastl, Eberhard Zwicker, Psycho-Acoustics: Facts and Models, Springer, 2007
Arthur H. Benade, Fundamentals of Musical Acoustics, Dover Books,1990
M. Brandstein, D. Ward, Microphone arrays, Springer Verlag, 2001.
P. Stoica, R. Moses, Introduction to Spectral Analysis, Prentice Hall, 1997.
Oral examination.
No.