Head-related transfer functions (HRTF) describe the sound transmission from the free field to a place in the ear canal in terms of linear time-invariant systems. Due to the physiological differences of the listeners' outer ears, the measurement of each subject's individual HRTFs is crucial for sound localization in virtual environments (virtual reality).

Measurement of an HRTF can be considered a system identification of the weakly non-linear electro-acoustic chain from the sound source room's HRTF microphone. An optimized formulation of the system identification with exponential sweeps, called the "multiple exponential sweep method" (MESM), was used for the measurement of transfer functions. For this measurement of transfer functions, either the measurement duration or the signal-to-noise ratio could be optimized.

Initial heuristic experiments have shown that using Gabor multipliers to extract the relevant sweeps in the MESM post-processing procedure improves the signal-to-noise ratio of the measured data even further. The objective of this project is to study, in detail, how frame multipliers can optimally be used during this post-processing procedure. In particular, wavelet frames, which best fit the structure of an exponential sweep, will be studied.

Systematic numeric experiments will be conducted with simulated slowly time-variant, weakly non-linear systems. As the parameters of the involved signals are precisely known and controlled, an optimal symbol will automatically be created. Finally, the efficiency of the new method will be tested on a "real world" system, which was developed and installed in the semi-anechoic room of the Institute. It uses in-ear microphones, a subject turntable, 22 loudspeakers on a vertical arc, and a head tracker.

The new method will be used for improved HRTF measurement.

Gabor multipliers are an efficient tool for time variant filtering used implicitly in many engineering applications in signal processing. For these operators, the result of a Gabor transform (the sampled version of the Short Time Fourier Transform) is multiplied by a fixed function, called the time-frequency mask or symbol. The result is then synthesized.

While Gabor multipliers are widely and practically used, some of their theoretical properties are not well known. The goal of this project is to improve the mathematical knowledge about Gabor multipliers, in order to optimize their use in applications.

The problem will be approached using modern Gabor theory, harmonic analysis tools, and numeric tools. Formulation and demonstration of analytical statements will be conducted jointly with systematic numeric experiments to study the properties of Gabor multipliers.

The following topics will be investigated in the project:

- Eigenvalues and eigenvectors of Gabor multipliers and their localization
- Invertibility and injectivity of Gabor multipliers
- Reproducing kernel invariance
- Connection of irregular Gabor multipliers and irregular frames of translates
- Discretization and implementation of Gabor multipliers
- Best approximation of operators by Gabor multipliers and identification of Gabor multipliers.

The applications of Gabor multipliers in signal processing are numerous, and include any application requiring time-variant filtering. Some applications of Gabor multipliers will be investigated further in the following parallel projects:

- Mathematical Modeling of Auditory Time-Frequency Masking Functions
- Improvement of Head-Related Transfer Function Measurements
- Advanced Method of Sound Absorption Measurements

The implementation of a Gabor multiplier in the software system STx has already proceeded quite far, see Stx-Mulac.

- Monika Dörfler and Bruno Torrésani, “Representation of operators in the time-frequency domain and generalized Gabor multipliers”, J. Fourier Anal. Appl., 2009 (in press)
- Yohan Frutiger: "Multiplicateurs de Gabor pour les transformations sonores" (Gabor Multipliers for sound transformations) Master thesis under the supervision of R. Kronland-Martinet, June 2008
- F. Jaillet, P. Balazs, M. Dörfler and N. Engelputzeder, “On the Structure of the Phase around the Zeros of the Short-Time Fourier Transform”, NAG/DAGA 2009, International Conference on Acoustics, March 2009, Rotterdam, Nederland
- F. Jaillet, P. Balazs and M. Dörfler, “Nonstationary Gabor Frames”, SampTA'09, 8th International Conference on Sampling and Applications, May 2009, Marseille, France

Gabor multipliers are an efficient tool for time-variant filtering. They are used implicitly in many engineering applications in signal processing. For these operators, the result of a Gabor transform (the sampled version of the Short Time Fourier Transform) is multiplied by a fixed function (called the time-frequency mask or symbol). Then the result is synthesized.

Other transforms beyond the Gabor transform, for example the wavelet transform, are more suitable for certain applications. The concept of multipliers can easily be extended to these transforms. More precisely, the concept of multipliers can be applied to general frames without any further structure. This results in the introduction of operators called frame multipliers, which will be investigated in detail in this project in order to precisely define their mathematical properties and optimize their use in applications.

The problem will be approached using modern frame theory, functional analysis, numeric tools, and linear algebra tools. Systematic numeric experiments will be conducted to observe the different properties of frame multipliers. This observations will support the analytical formulation and demonstration of these properties.

The following topics will be investigated in the project:

- Eigenvalues and eigenvectors of frame multipliers
- Invertibility, injectivity, and surjectivity of frame multipliers
- Reproducing kernel invariance
- Generalization of multipliers to Banach frames and p-frames
- Connection of frame multipliers to weighted frames
- Discretization and implementation of frame multipliers
- Best approximation of operators by frame multipliers and identification of frame multipliers

The applications of frame multipliers in signal processing are numerous and include any application requiring time-variant filtering. Some applications of frame multipliers will be investigated further in the following parallel projects:

- Mathematical Modeling of Auditory Time-Frequency Masking Functions
- Improvement of Head-Related Transfer Function Measurements
- Advanced Method of Sound Absorption Measurements

- P. Balazs, "Matrix Representation of Bounded Linear Operators By Bessel Sequences, Frames and Riesz Sequence", SampTA'09, 8th International Conference on Sampling and Applications, May 2009, Marseille, France
- P. Balazs, J.-P. Antoine, A. Grybos, "Weighted and Controlled Frames: Mutual relationship and first Numerical Properties", accepted for publication in International Journal of Wavelets, Multiresolution and Information Processing (2009), preprint
- A. Rahimi, P. Balazs, "Multipliers for p-Bessel sequences in Banach spaces", submitted (2009)
- D. Stoeva, P. Balazs, "Unconditional convergence and Invertibility of Multipliers", preprint (2009)

Time-variant filters are gaining importance in today's signal processing applications. Gabor multipliers in particular are popular in current scientific investigations. These multipliers are a specialization of Bessel multipliers to Gabor frames. These operators are interesting in regard to both theory and application:

- Bessel and Frame Multipliers in Banach Spaces: In this project, the concept of frame multipliers should be generalized to work with Banach spaces.
- Theory of Wavelet Multipliers: The concept of multipliers can be easily extended to wavelet frames. The influence of the special structures of these sequences will be investigated.
- Basic Properties of Irregular Gabor Multipliers: Here multipliers for Gabor frames on irregular lattices are investigated.

- Time Frequency Masking: Gabor Multiplier Models and Evaluation: The symbol for the Gabor multiplier is calculated adaptively and the resulting model incorporates both time and frequency masking components. The goal is to obtain an algorithm using 2-D convolution.
- Improving the Multiple Exponential Sweep Method (MESM) using Gabor Multipliers: The MESM is an efficient system identification method. Initial tests have shown that this method can be improved with a Gabor multiplier applied as a mask for the original sweep.
- Wavelet Multipliers and Their Application to Reflection Measurements: One method to calculate the absorption coefficient of a sound proof wall requires separation of the impulse responses of different reflections. They can be easily separated in a scalogram and they can be extracted using a wavelet multiplier.
- Mathematical Foundation of the Irrelevance Model: In this project, the theoretical foundation of the irrelevance algorithms implemented in STx is being developed.

- H.G. Feichtinger, K. Gröchenig et al., NuHAG, Faculty of Mathematics, University of Vienna
- R. Kronland-Martinet, S. Ytad, T. Necciari, Modélisation, Synthèse et Contrôle des Signaux Sonores et Musicaux of the LMA / CRNS Marseille
- S. Meunier, S. Savel, Acoustique perceptive et qualité de l’environnement sonore of the LMA / CRNS Marseille

- P. Balazs, B. Laback, G. Eckel, W. Deutsch, "Introducing Time-Frequency Sparsity by Removing Perceptually Irrelevant Components Using a Simple Model of Simultaneous Masking", IEEE Transactions on Audio, Speech and Language Processing, Vol. 17 (7) , in press (2009) , preprint
- P. Majdak, P. Balazs, B.Laback, "Multiple Exponential Sweep Method for Fast Measurement of Head Related Transfer Functions", Journal of the Acoustical Engineering Society , Vol. 55, No. 7/8, July/August 2007, Pages 623 - 637 (2007)

This project ended on 01.01.2010; most subprojects ended on 28.02.2008 and are incorporated into the 'High Potential'-Project of the WWTF, MULAC.

Another project has investigated the basic properties of frame and Bessel multipliers. This project aims to generalize this concept so that it will work with Banach spaces also.

As the Gram matrix plays an important role in the investigation of multipliers, it is quite natural to look at the connection to localized frames and multipliers. The dependency of the operator class on the symbol class can be researched.

The following statements will be investigated:

- Theorem: If G is a localized frame and a is a bounded sequence, then the frame multiplier Ta is bounded on all associated Banach spaces (the associated co-orbit spaces).
- Theorem: If G is a localized frame and a is a bounded sequence, such that the frame multiplier Ta is invertible on the Hilbert space H, then Ta is simultaneously invertible on the associated Banach spaces.

The applications of these results to Gabor frames and Gabor multipliers will be further investigated.

Although Banach spaces are more general a concept than Hilbert spaces, Banach theory has found applications. For example, if any norm other than L2 (least square error) is used for approximation, Banach theory tools have to be applied.

- K. Gröchenig, NuHAG, Faculty of Mathematics, University of Vienna

This project ended on 28.02.2008 and is incorporated into the 'High Potential'-Project of the WWTF, MULAC.

The most basic model for convolution algorithms is an extension of the simultaneous irrelevance model. A triangle-like function describes the masking effect in the frequency and time direction. Combined, they result in a 2-D function, which is used as convolution on the time-frequency coefficients of the given signal. The resulting information is then used to calculate a threshold function. This can be implemented as a Gabor multiplier. This very simple function should be exchanged for a more elaborate 2-D kernel. A more elaborate 2-D kernel can be developed from the first time frequency masking effect measurements of a Gaussian atom.

An extension of the simultaneous irrelevance model is used as the most basic model for the convolution algorithm under investigation. A triangle-like function describes the masking effect in the frequency and time direction. Combined, they result in a 2-D function, which is used as convolution on the time-frequency coefficients of the given signal to calculate a threshold function. This can be implemented as a Gabor multiplier. This very simple function should be exchanged for a more elaborate 2-D kernel developed from the first time-frequency masking effect measurements of a Gaussian atom.

After thoroughly testing this algorithm in psychoacoustic experiments, it will be implemented in STx.

- R. Kronland-Martinet, S. Ytad, T. Necciari, Modélisation, Synthèse et Contrôle des Signaux Sonores et Musicaux of the LMA / CRNS Marseille
- S. Meunier, S. Savel, Acoustique perceptive et qualité de l’environnement sonore of the LMA / CRNS Marseille

This project ended on 28.02.2008 and is incorporated into the 'High Potential'-Project of the WWTF, MULAC.

An irrelevance algorithm based on simultaneous masking is implemented In STx. In the years following its first development by Eckel, the efficiency of this algorithm has been clearly shown. In this project, this irrelevance model will be based on modern mathematic and psychoacoustic theories and knowledge.

This algorithm can be described as a Gabor multiplier with an adaptive symbol. With existing related theory, it becomes clear that a high redundancy must be selected. This guarantees:

- perfect reconstruction synthesis
- an under-spread operator for good time-frequency localization
- a smoothing-out of easily detectable quick on/off cycles

Furthermore, it can be shown that the model used for the spreading function here is mathematically equivalent to the excitation pattern.

This algorithm has been used for several years already for things such as:

- automobile sound design
- over-masking for background-foreground separation
- improved speech recognition in noise
- contrast increase for hearing-impaired persons

- G. Eckel, Institut für Elektronische Musik und Akustik, Graz

- P. Balazs, B. Laback, G. Eckel, W. Deutsch, "Introducing Time-Frequency Sparsity by Removing Perceptually Irrelevant Components Using a Simple Model of Simultaneous Masking", IEEE Transactions on Audio, Speech and Language Processing, Vol. 17 (7) , in press (2009) , preprint

This project ended on 01.01.2010, and leads to a sub-project of the 'High Potential'-Project of the WWTF, MULAC.

Time-variant filters are gaining more importance in today's signal processing applications. Also, there wavelet analysis has numerous applications. The goal of this project is to investigate time-variant systems based on wavelet analysis.

The concept of multipliers can be easily extended to wavelet frames. This means the coefficients of a wavelet analysis are multiplied by a fixed symbol and then resynthesized. The influence of the special structures of these sequences on the resulting operators will be investigated.

The theory of Pseudo-Differential Operators (PDO) can be translated to the wavelet case. How operators of interest in the investigation of multipliers, like the Kohn-Nirenberg correspondence, are translated to this case is of particular interest. Natural starting points for the research are:

Use dilations in the definition of the spreading function instead of modulation.

Define a special wavelet kernel function by using a weak formulation:

< K f , g > = < k , Wg f >

A very useful application for this project is an analysis-modification-synthesis system based on the wavelet analysis. With some language manipulation, this could be called a "Wavelet Phase Vocoder".

The application investigated in this project is the measurement of reflection coefficients. The wavelet analysis is preferable for signals containing transient parts. It is essential to separate the impulse responses of different reflections in order to calculate the absorption coefficient of a sound-proof wall. The impulse responses can be easily separated in a scalogram, and they can be extracted by using a wavelet multiplier.

This project ended on 28.02.2008 and is incorporated into the 'High Potential'-Project of the WWTF, MULAC.

The Multiple Exponential Sweep Method (MESM) is an optimized method for the semi-simultaneous system identification of multiple systems. It uses an appropriate overlapping of the excitation signals. This leads to a faster identification of the weakly nonlinear systems that are retrieving the linear impulse response only. Using a Gabor multiplier in the post-processing procedure of the system identification may reduce the measurement noise. This may further improve the signal-to-noise ratio of the measured data.

A Gabor multiplier is used to cut the interesting parts out of the measured signals in the time-frequency plane. This allows a specific optimization of signal parts, independent of the frequency. Initial tests applying a Gabor multiplier to simulated data showed that the depth of spectral notches could be raised. A systematic investigation of this method is the main goal this project.

This method may improve the signal-to-noise ratio in system identification tasks of any weakly nonlinear system, such as those involving acoustic measurements with electric equipment.

- P. Majdak, P. Balazs, B.Laback, "Multiple Exponential Sweep Method for Fast Measurement of Head Related Transfer Functions", Journal of the Acoustical Engineering Society , Vol. 55, No. 7/8, July/August 2007, Pages 623 - 637 (2007)

This project ended on 28.02.2008 and is incorporated into the 'High Potential'-Project of the WWTF, MULAC (q.v.).

This project line has the goal of finding efficient algorithms for signal processing applications. To apply the results of signal processing, Gabor or wavelet theory, the algorithms must be formulated for finite dimensional vectors. These discrete results are motivated by the continuous setting, but also often provide some insight. Furthermore, the efficient implementation of algorithms becomes important. For the consistency of these algorithms, it is useful to incorporate them into a maintained software package.

- Double Preconditioning for Gabor Frames: This project develops a way to find an analysis-synthesis system with perfect reconstruction in a numerically efficient way using double preconditioning.
- Perfect Reconstruction Overlap Add Method (PROLA): The classic overlap-add synthesis method is systematically compared to a new method motivated by frame theory.
- Numerics of Block Matrices: In some applications in acoustics, it is apparent that block matrices are a powerful tool to find numerically efficient algorithms.
- Practical Time Frequency Analysis: This project evaluates the usefulness of a time-frequency toolbox for acoustic applications and STx.

- H.G. Feichtinger et al., NuHAG, Faculty of Mathematics, University of Vienna
- B. Torrésani, Groupe de Traitement du Signal, Laboratoire d'Analyse Topologie et Probabilités, LATP/ CMI, Université de Provence, Marseille
- P. Soendergaard, Department of Mathematics, Technical University of Denmark
- J. Walker, Department of Mathematics, University of Wisconsin-Eau Claire

- P. Balazs, H.G. Feichtinger, M. Hampejs, G. Kracher; "Double Preconditioning for Gabor Frames”; IEEE Transactions on Signal Processing, Vol. 54, No.12, December 2006 (2006), preprint
- P. Balazs, H.G. Feichtinger, M. Hampejs, G. Kracher; "Double Preconditioning for the Gabor Frame Operator”; Proceedings ICASSP '06, May 14-19, Toulouse, DVD (2006)