INEX 2005 Workshop on Element Retrieval Methodology Saturday, 30 July 2005 - University of Glasgow http://www.cs.otago.ac.nz/inexmw INTRODUCTION The INitiative for the Evaluation of XML Retrieval (INEX), and the University of Glasgow invite participation in the INEX Workshop on Element Retrieval Methodology. The workshop is to be held in Glasgow as part of an information retrieval research festival. The annual INEX workshop examining element retrieval has shown improvements in retrieval precision, while at the same time raising new questions about element retrieval methodology. The annual evaluation workshop focuses on retrieval performance; this mid-year workshop will focus on methodology. PARTICIPATION Opinion papers discussing XML element retrieval are sought. Topics include but are not limited to Theory, Application, Measurement, Judgment, and Experience. Note that relevance ranking algorithms are excluded (they are the focus of the annual INEX workshop (inex.is.informatik.uni-duisburg.de)). Theory What is the perfect collection and what is the ideal way to interact with it? Is a heterogeneous collection diverse in DTDs or diverse in content? How should queries of this collection be specified in either a formal or natural language? When a user provides feedback are they providing information about the element (e.g. too large) or the content of the element? Application Is there an existing application of element retrieval either commercially or academically? Can user modeling of this application be used to identify unaddressed problems or formally define existing problems? How is element retrieval used, and what is it used for? What is a relevant element in (a given) context, and how can performance be measured within this context? Measurement What are current element retrieval metrics actually measuring? What would the ideal metric measure? There already exists a plethora of metrics so new metrics are not of interest, what is of interest is the identification of what should be measured. Judgment What can be determined from the existing INEX relevance judgments? Can an E3S3 element validly be the child of an E3S3 element, and if so then what does it mean? Is the 10 point relevance scale understood by the judges? How and when does a relevant element have no relevant descendants and what are the implications of this? How much information is needed for an element to become relevant (can a reference make an element relevant)? Note that the existing INEX online assessment software is not under discussion. Experience What can the experiments of TREC (trec.nist.gov), CLEF (www.clef-campaign.org), and NTCIR (research.nii.ac.jp/ntcir) bring to element retrieval and INEX? How (if at all) is XML element identification like spoken document retrieval, video document retrieval, or passage retrieval? Is element retrieval a form of topic distillation, and how might experience in topic distillation be used for element retrieval? Others Any other topic related to XML element retrieval methodology is welcomed. Discussion of relevance ranking algorithms is not - they are the focus of the end of year INEX workshop. SUBMISSION Opinion papers discussing any of the above topics, or any other topic related to XML retrieval methodology (excluding relevance ranking) are sought. Contributions formatted according to the ACM SIG Proceedings guidelines (www.acm.org/sigs/pubs/proceed/template.html) and not exceeding 8 (eight) pages should be submitted by email to Mounia Lalmas (mounia@dcs.qmul.ac.uk) and Andrew Trotman (andrew@cs.otago.ac.nz). Submissions should be formatted in PDF. WORKSHOP FORMAT On-topic contributions will be combined into a publicly-available proceedings. This will form a discussion document for the workshop (so please read it). From this the program committee will choose the most fiercely debated topics for discussion at the workshop. The workshop will consist for 4 (four) sessions, each discussing one topic. Sessions will commence with a 30 minute discussion-raising presentation (by invitation) followed by a 90 (ninety) minute open discussion. Notes will be taken and made available. SCHEDULE 30 May 2005 Deadline for submissions 15 June 2005 Discussion Topics Announced 30 July 2005 INEX Element Retrieval Methodology Workshop ORGANIZERS Norbert Fuhr (fuhr@uni-duisburg.de) Mounia Lalmas (mounia@dcs.qmul.ac.uk) Andrew Trotman (andrew@cs.otago.ac.nz) For program committee and other details, visit http://www.cs.otago.ac.nz/inexmw