[Air-L] "Text Annotation for Political Science Research"

[Stuart Shulman]

Call for Papers

A special issue of the Journal of Information Technology & Politics

"Text Annotation for Political Science Research"

Text is an important data source for political science research.  Large,
digitized text collections are becoming increasingly common.  Yet most
political scientists have little familiarity with the language-processing
methodologies available to support research using these collections.
Specifically, we are interested in methodologies from the fields of
information retrieval, natural language processing, and machine learning.
These techniques facilitate the automatic searching, organizing,
categorizing, and extracting of information from digitized text.

At a high level, the goal of language-processing is to provide one or more
semantic annotations on the text.  The political science question of
interest can then be explored using these annotations.  Text annotation
techniques vary not only according to the type of semantic annotation
required, but also according to the degree of manual intervention involved
in the annotation process: text annotation tasks can be accomplished
entirely manually ( i.e., via human content coding), entirely automatically
(e.g. via keyword-based search or text clustering algorithms), automatically
after a manual training period (i.e. via "supervised" machine learning
methods), or semi-automatically ( e.g. via "weakly supervised" machine
learning methods that acquire automatic annotation systems from very small
amounts of manually labeled text).

Although the potential of text annotation methods for political science
research is enormous, it is understandably difficult for researchers to know
where to start.  In addition, in contrast to other research methodologies in
the social sciences, the criteria for evaluating social science results that
rely on automatic text annotation systems are not widely known, accepted, or
appreciated.  Keyword searches, for example, are commonly used to trace
changing political attention across time, but rarely is attention given to
their reliability or accuracy, raising doubts about the validity of
researcher inferences.

The aim of the special issue is to solicit and publish papers that provide a
clear view of the state of the art in text annotation and evaluation,
especially for political science.  How do the techniques map onto major
questions addressed by political scientists?  What kinds of problems have
been addressed in existing work and what text annotation methods have proven
most successful?  Are standard statistical measures of accuracy, recall, and
precision adequate for evaluating the performance of the text annotation
technique, or are new evaluation procedures needed that simultaneously
consider the social science question being investigated?

Given these interests, we therefore encourage submissions in the following
areas:

• tutorial-style surveys of state-of-the-art techniques in human language
technologies and text annotation;
• surveys of the state-of-the-art in the application of language-processing
techniques in the social sciences,  particularly in political science;
• comparisons of competing text annotation methodologies on the same
corpus/corpora;
• innovative evaluation and diagnostic methods;
• studies of text annotation methods that try to limit the amount of costly,
manually annotated data for training automatic annotators, e.g. active
learning;
• specific applications and evaluations of language-processing and text
annotation techniques;
• applications of the text-processing techniques on non-social science
problems that point the way to innovative social science applications; and
• surveys of the available language-processing tools and resources with
guidance for when to use them.

All submissions must be prepared according to the submission guidelines
available at:
http://www.jitp.net

The initial submission is due by November 1, 2007

The guest editors for the special issue are:

Claire Cardie, Professor
Computer Science and Information Science
4130 Upson Hall
Cornell University
Ithaca NY 14853-7501
cardie at cs.cornell.edu

John Wilkerson, Associate Professor
Department of Political Science
101 Gowen Hall
University of Washington
Seattle WA 98195-353530
jwilker at u.washington.edu


-- 
Dr. Stuart W. Shulman
Director, Sara Fine Institute
School of Information Sciences
Director, Qualitative Data Analysis Program
University Center for Social and Urban Research
University of Pittsburgh