<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="static/style.xsl"?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T19:57:34Z</responseDate><request verb="GetRecord" identifier="oai:uwspace.uwaterloo.ca:10012/10609" metadataPrefix="dim">https://dspacemainprd01.lib.uwaterloo.ca/server/oai/request</request><GetRecord><record><header><identifier>oai:uwspace.uwaterloo.ca:10012/10609</identifier><datestamp>2025-03-26T23:38:12Z</datestamp><setSpec>com_10012_9924</setSpec><setSpec>com_10012_1</setSpec><setSpec>col_10012_9930</setSpec><setSpec>col_10012_6</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:doc="http://www.lyncode.com/xoai" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="advisor">Smucker, Mark</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="advisor">Clarke, Charles</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="author" authority="15da09d2-335e-441d-9dce-f12362456df5" confidence="600">Baruah, Gaurav Makhon</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2016-08-04T15:53:01Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2016-08-04T15:53:01Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2016-08-04</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="submitted">2016-08-02</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">http://hdl.handle.net/10012/10609</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract" lang="en">Events like hurricanes, earthquakes, &#xd;
or accidents can impact a large number of people. Not only are people in the &#xd;
immediate vicinity of the event affected, but concerns about their well-being are&#xd;
shared by the local government and well-wishers across the world.&#xd;
The latest information about news events &#xd;
could be of use to government and aid agencies in order to make informed decisions on&#xd;
providing necessary support, security and relief. The general public &#xd;
avails of news updates via dedicated news feeds or broadcasts, and lately,&#xd;
via social media services&#xd;
like Facebook or Twitter.&#xd;
Retrieving the latest information about newsworthy events from the world-wide web&#xd;
is thus of importance to a large section of society.&#xd;
&#xd;
As new content on a multitude of topics is continuously being published on the web, &#xd;
specific event related information needs to be filtered from the resulting&#xd;
stream of documents. &#xd;
We present in this thesis, a user-centric evaluation measure for&#xd;
evaluating systems that filter news related information from document streams.&#xd;
Our proposed evaluation measure, Modeled Stream Utility (MSU), models&#xd;
users accessing information from a stream of sentences&#xd;
produced by a news update filtering system. &#xd;
The user model allows for simulating a large number of users with different &#xd;
characteristic stream browsing behavior. Through simulation,  &#xd;
MSU estimates the utility of a system for an&#xd;
average user browsing a stream of sentences.&#xd;
Our results show that system performance is sensitive to a user population&amp;apos;s&#xd;
stream browsing behavior and that &#xd;
existing evaluation metrics correspond to very specific types of user behavior.&#xd;
&#xd;
To evaluate systems that filter sentences from a document stream, &#xd;
we need a set of judged sentences. This judged set is   &#xd;
a subset of all the sentences returned by all systems, and is&#xd;
typically constructed by pooling &#xd;
together the highest quality sentences,   &#xd;
as determined by respective system assigned scores for each sentence.&#xd;
Sentences in the pool are manually assessed and &#xd;
the resulting set of judged sentences is then used to compute system performance metrics.&#xd;
In this thesis, we investigate the effect of including duplicates of &#xd;
judged sentences, into the judged set, on system performance evaluation. We also develop an  &#xd;
alternative pooling methodology, that given the MSU user model, &#xd;
selects sentences for pooling based on the probability of a sentences being read by &#xd;
modeled users.&#xd;
&#xd;
Our research lays the foundation for interesting future work for utilizing&#xd;
user-models in different aspects of evaluation of stream filtering systems.&#xd;
The MSU measure enables incorporation of different &#xd;
user models. Furthermore, the applicability of MSU could be extended through &#xd;
calibration based on user&#xd;
behavior.</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso" lang="en">en</dim:field>
   <dim:field mdschema="dc" element="publisher" lang="en">University of Waterloo</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">information retrieval</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">system evaluation</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">user modeling</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">streams and filtering</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">pooling</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en">duplicates</dim:field>
   <dim:field mdschema="dc" element="title" lang="en">Filtering News from Document Streams: Evaluation Aspects and Modeled Stream Utility</dim:field>
   <dim:field mdschema="dc" element="type" lang="en">Doctoral Thesis</dim:field>
   <dim:field mdschema="dc" element="pending">false</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="department" lang="en">David R. Cheriton School of Computer Science</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="discipline" lang="en">Computer Science</dim:field>
   <dim:field mdschema="uws-etd" element="degree" qualifier="grantor" lang="en">University of Waterloo</dim:field>
   <dim:field mdschema="uws-etd" element="degree" lang="en">Doctor of Philosophy</dim:field>
   <dim:field mdschema="uws" element="contributor" qualifier="advisor">Smucker, Mark</dim:field>
   <dim:field mdschema="uws" element="contributor" qualifier="advisor">Clarke, Charles</dim:field>
   <dim:field mdschema="uws" element="contributor" qualifier="affiliation1" lang="en">Faculty of Mathematics</dim:field>
   <dim:field mdschema="uws" element="published" qualifier="city" lang="en">Waterloo</dim:field>
   <dim:field mdschema="uws" element="published" qualifier="country" lang="en">Canada</dim:field>
   <dim:field mdschema="uws" element="published" qualifier="province" lang="en">Ontario</dim:field>
   <dim:field mdschema="uws" element="typeOfResource" lang="en">Text</dim:field>
   <dim:field mdschema="uws" element="peerReviewStatus" lang="en">Unreviewed</dim:field>
   <dim:field mdschema="uws" element="scholarLevel" lang="en">Graduate</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>