TITLE: Towards Standards and Tools for Discourse Tagging DESCRIPTION: Discourse tagging assigns labels from a tag set to discourse units in texts or dialogues. The discourse units range from words or referring expressions to multi-utterance units identified by criteria such as speaker intention or initiative. Since the emergence of syntactically annotated corpora has resulted in major advances in sentence-level natural language processing, the hope is that corpora of tagged discourse may lead to similar advances in the area of discourse processing. Work on discourse tagging has gained momentum in the last 3-4 years. Three major initiatives in this area are: the Discourse Resource Initiative (http://www.georgetown.edu/luperfoy/Discourse-Treebank/), that has organized yearly international workshops addressing the standardization of discourse tagging schemes for coreference, for dialogue acts, and for higher level discourse structures; MATE (http://mate.mip.ou.dk/), a project co-funded by the European Union, whose aim is to develop tools and standards for tagging spoken dialogue corpora at different levels, including the discourse level; the Global Document Annotation initiative, that aims at having Internet authors annotate their documents with a common standard tag set which allows machines to recognize the semantic and pragmatic structures of documents (http://ww.etl.go.jp/etl/nl/GDA). Even with these three initiatives in place, there is still much work to be done before there are widely accepted (standardized) tagging schemes for various discourse phenomena that could be shared across sites; moreover, there has not yet been an open forum to which researchers working in this area could participate and contribute. This workshop will provide such a forum. Submissions are invited on, but not limited to, the following topics and issues: 1. How can standardization for discourse tagging concretely be achieved? by developing a single coding scheme, or more likely, a set of coding schemes, one for each phenomenon of interest? or rather, by developing some specification guidelines and a way of mapping from one scheme to another? in some other way? 2. Cross-level coding: all the initiatives mentioned above promote an approach in which coding schemes are developed at different levels, rather than an approach in which a monolithic scheme addresses all phenomena. Given this methodology, the issue of cross-level coding arises, namely, how can coding schemes for different levels take advantage of each other and allow coding of cross-level relationships? is it possible to relate corpus annotations at different annotation levels to examine the interdependence of linguistic phenomena? 3. Coding schemes and theories of discourse: is it possible to develop coding schemes that faithfully reflect a discourse theory? if yes, is it desirable? conversely, can corpora coded for discourse issues help advance our theoretical understanding of discourse phenomena? 4. Coding schemes and applications: is it possible to design discourse coding schemes independently from the applications tagged corpora are supposed to be used for (eg, to train a speech act recognizer)? 5. Coding schemes and reliability: discourse categories are difficult to code for reliably. Whatever the reason (e.g., lack of an overarching theory for discourse, or genuine ambiguity and misunderstandings in real dialogue reflected in the coding), how can we devise reliable coding schemes? What reliability measures should be used: are widely used measures (Kappa, Alpha, precision and recall) appropriate in this case? If not, what other measures can we use? Is reliability affected by whether naive or expert coders are used? 6. Tools for discourse tagging: what specific features of a tool does discourse tagging require? can we just extend tools developed eg for syntactic tagging? do we need to develop new tools? 7. Some paradigms for evaluating dialogue systems take advantage of the use of tagged corpora: how are tagging for evaluation purposes and discourse tagging related? Are there some discourse tags that may be used as evaluation tags or is it advisable to introduce another dimension of tagging? In addition to papers, prospective participants may be asked to do a small homework before the workshop to test out various tagging schemes. Prospective participants who have developed tools are welcome to bring a demo with them. Submission Procedure: Authors are requested to submit one electronic version of their papers OR four hardcopies. Please submit hardcopies only if electronic submission is impossible. Send your electronic submission to both Marilyn Walker (walker@research.att.com) and Morena Danieli (morena.danieli@cselt.it). If electronic submission is impossible, please contact the organizers to arrange for hardcopy submission. Maximum length is 6 pages including figures and references. Please conform with the traditional two-column ACL Proceedings format. Style files can be downloaded from ftp://ftp.cs.columbia.edu/acl-l/Styfiles/Proceedings/. Timetable: Deadline for submissions: March 20, 1999. Notification of acceptance: April 16, 1999. Camera ready copies due: April 30, 1999 WORKSHOP CHAIRS: Marilyn Walker, Morena Danieli, Johanna D. Moore, Barbara Di Eugenio.