Call for Papers:  Special Issue of the
Journal of Information Technology & Politics (http://www.jitp.net)
\u201cText Annotation for Political Science Research\u201d

Text is an important data source for political science research.
Large, digitized text collections are becoming increasingly common.
Yet most political scientists have little familiarity with the
language-processing methodologies available to support research using
these collections.  Specifically, we are interested in methodologies
from the fields of information retrieval, natural language processing,
and machine learning.  These techniques facilitate the automatic
searching, organizing, categorizing, and extracting of information
from digitized text.

At a high level, the goal of language-processing is to provide one or
more semantic annotations on the text.  The political science question
of interest can then be explored using these annotations.  Text
annotation techniques vary not only according to the type of semantic
annotation required, but also according to the degree of manual
intervention involved in the annotation process: text annotation tasks
can be accomplished entirely manually (i.e., via human content
coding), entirely automatically (e.g. via keyword-based search or text
clustering algorithms), automatically after a manual training period
(i.e. via "supervised" machine learning methods), or
semi-automatically (e.g. via "weakly supervised" machine learning
methods that acquire automatic annotation systems from very small
amounts of manually labeled text).

Although the potential of text annotation methods for political
science research is enormous, it is understandably difficult for
researchers to know where to start.  In addition, in contrast to other
research methodologies in the social sciences, the criteria for
evaluating social science results that rely on automatic text
annotation systems are not widely known, accepted, or appreciated.
Keyword searches, for example, are commonly used to trace changing
political attention across time, but rarely is attention given to
their reliability or accuracy, raising doubts about the validity of
researcher inferences.

The aim of the special issue is to solicit and publish papers that
provide a clear view of the state of the art in text annotation and
evaluation, especially for political science.  How do the techniques
map onto major questions addressed by political scientists?  What
kinds of problems have been addressed in existing work and what text
annotation methods have proven most successful?  Are standard
statistical measures of accuracy, recall, and precision adequate for
evaluating the performance of the text annotation technique, or are
new evaluation procedures needed that simultaneously consider the
social science question being investigated?

Given these interests, we therefore encourage submissions in the following areas:

·         tutorial-style surveys of state-of-the-art techniques in human language technologies and text annotation;

·         surveys of the state-of-the-art in the application of language-processing techniques in the social sciences,  particularly in political science;

·         comparisons of competing text annotation methodologies on the same corpus/corpora;

·         innovative evaluation and diagnostic methods;

·         studies of text annotation methods that try to limit the amount of costly, manually annotated data for training automatic annotators, e.g. active learning;

·         specific applications and evaluations of language-processing and text annotation techniques;

·         applications of the text-processing techniques on non-social science problems that point the way to innovative social science applications; and

·         surveys of the available language-processing tools and resources with guidance for when to use them.

All submissions must be prepared according to the submission guidelines available at: www.jitp.net.  Authors must submit via: http://www.criticalmath.com/method/sm.php?org_id=12789

The initial submission is due by November 1, 2007

The guest editors for the special are:

 Claire Cardie, Professor
Computer Science and Information Science
4130 Upson Hall
Cornell University
Ithaca NY 14853-7501
cardie@cs.cornell.edu

 

John Wilkerson, Associate Professor
Department of Political Science
101 Gowen Hall
University of Washington
Seattle WA 98195-353530
jwilker@u.washington.edu