[Air-L] Quantifying Political Alignment

Deen Freelon dfreelon at u.washington.edu
Mon Nov 22 08:35:44 PST 2010


Coding political characteristics is a common task in the field of 
political communication. There are no broadly established metrics as far 
as I am aware, but here are a few recent references:

    * Freelon, D. G. (2010). Analyzing online political discussion using
      three models of democratic communication. New Media & Society,
      12(7), 1172-1190.
    * Janssen, D., & Kies, R. (2005). Online forums and deliberative
      democracy. Acta Politica, 40(3), 317-335.
    * Kelly, J., Fisher, D., & Smith, M. (2005). Debate, division, and
      diversity: Political discourse networks in USENET newsgroups.
      Online Deliberation 2005/DIAC-2005.
    * Papacharissi, Z. (2004). Democracy online: Civility, politeness,
      and the democratic potential of online political discussion
      groups. New Media & Society, 6(2), 259-283.
    * Stromer-Galley, J. (2007). Measuring deliberation's content: A
      coding scheme. Journal of Public Deliberation, 3(1), 1-37.

When coding your texts, you will want to follow standard content 
analysis practices, one of which is the assessment of intercoder 
reliability. In this you may find my online intercoder reliability 
application ReCal of some value: http://dfreelon.org/utils/recalfront/ . 
It calculates not only percent agreement but also chance-corrected 
reliability coefficients such as Scott's pi, Cohen's kappa, and 
Krippendorff's alpha.

Hope some of this helps, ~DEEN

On 11/22/10 10:36 AM, Michael Conover wrote:
> Our research group (made up of physicists and computer scientists) needs to
> evaluate the political content of short pieces of text.  To this end we've
> constructed a rubric for classification, ranging from strong left, lean
> left, and neutral, to lean right, strong right, and unclassifiable.  While
> the criteria of the rubric are rather clear cut (a 'strong' classified
> sentence would contain attacks on a person or group's character, hyperbolic
> language, or intense / fanatical expressions of support for an issue), we
> all suspect that this is a solved problem.
> I've heard this referred to by political scientists as 'coding' text, and it
> definitely falls within the domain of qualitative content analysis, but I'm
> yet to find anything specifically on evaluating political text.  This in
> mind, can anyone point me towards an established metric for evaluating the
> political content of bodies of text?
> Thanks kindly,
> Michael Conover
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> Join the Association of Internet Researchers:
> http://www.aoir.org/

Deen Freelon
Ph.D. Candidate, Dept. of Communication
University of Washington
dfreelon at uw.edu

More information about the Air-L mailing list