<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="static/style.xsl"?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T11:40:23Z</responseDate><request verb="GetRecord" identifier="oai:uwspace.uwaterloo.ca:10012/3065" metadataPrefix="dim">https://dspacemainprd01.lib.uwaterloo.ca/server/oai/request</request><GetRecord><record><header><identifier>oai:uwspace.uwaterloo.ca:10012/3065</identifier><datestamp>2017-03-21T21:38:19Z</datestamp><setSpec>com_10012_9900</setSpec><setSpec>com_10012_1</setSpec><setSpec>col_10012_9908</setSpec><setSpec>col_10012_6</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:doc="http://www.lyncode.com/xoai" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="author" authority="0fac6d8f-f6e1-44a5-85d0-aaa30baf830f" confidence="600">Makkook, Mustapha</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2007-05-22T14:15:43Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2007-05-22T14:15:43Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2007-05-22T14:15:43Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="submitted">2007</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">http://hdl.handle.net/10012/3065</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract" lang="en">A key requirement for developing any innovative system in a&#xd;
computing environment is to integrate a sufficiently friendly &#xd;
interface with the average end user. Accurate design of such a&#xd;
user-centered interface, however, means more than just the&#xd;
ergonomics of the panels and displays. It also requires that&#xd;
designers precisely define what information to use and how, where, &#xd;
and when to use it. Recent advances in user-centered design of&#xd;
computing systems have suggested that multimodal integration can&#xd;
provide different types and levels of intelligence to the user&#xd;
interface. The work of this thesis aims at improving speech &#xd;
recognition-based interfaces by making use of the visual modality&#xd;
conveyed by the movements of the lips.&#xd;
&#xd;
Designing a good visual front end is a major part of this framework.&#xd;
For this purpose, this work derives the optical flow fields for &#xd;
consecutive frames of people speaking. Independent Component&#xd;
Analysis (ICA) is then used to derive basis flow fields. The&#xd;
coefficients of these basis fields comprise the visual features of&#xd;
interest. It is shown that using ICA on optical flow fields yields &#xd;
better classification results than the traditional approaches based&#xd;
on Principal Component Analysis (PCA). In fact, ICA can capture&#xd;
higher order statistics that are needed to understand the motion of&#xd;
the mouth. This is due to the fact that lips movement is complex in &#xd;
its nature, as it involves large image velocities, self occlusion&#xd;
(due to the appearance and disappearance of the teeth) and a lot of&#xd;
non-rigidity.&#xd;
&#xd;
Another issue that is of great interest to audio-visual speech &#xd;
recognition systems designers is the integration (fusion) of the&#xd;
audio and visual information into an automatic speech recognizer.&#xd;
For this purpose, a reliability-driven sensor fusion scheme is&#xd;
developed. A statistical approach is developed to account for the &#xd;
dynamic changes in reliability. This is done in two steps. The first&#xd;
step derives suitable statistical reliability measures for the&#xd;
individual information streams. These measures are based on the&#xd;
dispersion of the N-best hypotheses of the individual stream &#xd;
classifiers. The second step finds an optimal mapping between the&#xd;
reliability measures and the stream weights that maximizes the&#xd;
conditional likelihood. For this purpose, genetic algorithms are&#xd;
used.&#xd;
&#xd;
The addressed issues are challenging problems and are substantial &#xd;
for developing an audio-visual speech recognition framework that can&#xd;
maximize the information gather about the words uttered and minimize&#xd;
the impact of noise.</dim:field>
   <dim:field mdschema="dc" element="format" qualifier="extent">1133785 bytes</dim:field>
   <dim:field mdschema="dc" element="format" qualifier="mimetype">application/pdf</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso" lang="en">en</dim:field>
   <dim:field mdschema="dc" element="publisher" lang="en">University of Waterloo</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">Multimodal fusion</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">Visual speech recognition</dim:field>
   <dim:field mdschema="dc" element="subject" qualifier="program" lang="en">Electrical and Computer Engineering</dim:field>
   <dim:field mdschema="dc" element="title" lang="en">A Multimodal Sensor Fusion Architecture for Audio-Visual Speech Recognition</dim:field>
   <dim:field mdschema="dc" element="type" lang="en">Master Thesis</dim:field>
   <dim:field mdschema="dc" element="pending" lang="en">false</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="department" lang="en">Electrical and Computer Engineering</dim:field>
   <dim:field mdschema="uws-etd" element="degree" lang="en">Master of Applied Science</dim:field>
   <dim:field mdschema="uws" element="typeOfResource" lang="en">Text</dim:field>
   <dim:field mdschema="uws" element="peerReviewStatus" lang="en">Unreviewed</dim:field>
   <dim:field mdschema="uws" element="scholarLevel" lang="en">Graduate</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>