5 : Fusion of Audio and Vision
Descriptif
The fifth and last part of the video lectures will address the fusion of auditory and visual data. We will start with the motivation behind audio-visual fusion, followed by a short overview of the visual features that are likely to be used.We will describe audio-visual fusion in the temporal and in the spectral domains and we will present a few examples using the audio-visual head of the NAO robot. This part will complete the course that described the main methodologies needed to perform hearing with a binaural robot head.
Vidéos
Audio-visual processing challenges
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Representation of visual information
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
The geometry of vision
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Audio-visual feature association
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Audio-visual alignment
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Visually-guided audio localization
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Audio-visual event localization
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Audio-visual clustering
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Conclusions
Part 5 : Fusion of Audio and Vision 5.1. Audio-visual processing challenges 5.2. Representation of visual information 5.3. The geometry of vision 5.4. Audio-visual feature association 5.5. Audio
Intervenants
Auteur d'une thèse de docteur-ingénieur en automatique (Grenoble INPG, 1981). - Directeur de recherche au CNRS, puis à l'INRIA, Laboratoire d'informatique fondamentale et d'intelligence artificielle de l'Institut national polytechnique de Grenoble (en 1993). - Directeur de thèse à l'Université Joseph Fourier de Grenoble et à Grenoble INPG (-1990-1994-). - Consultant