[Air-L] New Web-based Intercoder Reliability Calculator for Content Analysis

[Deen Freelon]

Hello internet researchers,

Those of you who use content analysis in your research might be 
interested in a recent project of mine: ReCal ("reliability 
calculator"), an online utility that calculates several intercoder 
reliability coefficients for nominal data. It is available at 
http://dfreelon.org/utils/recalfront/ . Here is a short feature list:

* Calculates the following reliability coefficients:
* percent agreement
* average pairwise percent agreement (for 3+ coders)
* Scott’s Pi
* Cohen’s Kappa
* Fleiss' Kappa
* Krippendorff’s Alpha
* Accepts any range of possible variable values
* Can calculate reliability for multiple variables at a time (2 coders only)
* Displays all possible pairwise percent agreements, making it easier to 
root out rogue coders (3+ coders only)
* Results should be valid for nominal data coded by any number of coders

There are other solutions available, but PRAM is Windows-only (ReCal is 
written in PHP, so it's cross-platform) and the various macros floating 
around the web for SPSS, STATA, R, and the like are only useful if you 
have easy access to those applications. ReCal is fast, free, and 
(hopefully) relatively easy to use. It is currently still undergoing 
testing, but its results have been demonstrated to be robust thus far. 
If you decide to use it, feel free to leave feedback on the web site or 
email me directly. Thanks, ~DEEN

-- 
Deen Freelon
Ph.D. Student, Communication
University of Washington
dfreelon at u.washington.edu