[Air-L] Content Analysis Information

[Laura B Nielsen]

2/25/08
TO:  Political Science Methodolgy-POLMETH 
         Association of Internet Researchers-Air-L
Please find attached the consolidated submissions from my inquiry: 

“I am looking for reference material for use in my work at EPA. We are planning to launch a discussion board with a limited number of comment fields. But we also want to analyze discussion content in order to better understand the needs and interests of our customers--in particular those who come to EPA seeking environmental information.
1) Have any of you been teaching about content analysis methods lately? If so, what's your favorite reference?
2) Has anyone been working on content analysis of discussion boards as a research method?”

Thanks to everyone for all the helpful responses!

Content Analysis Reference
3) Sent by: Harald Klein <intext at GMX.DE>  , Alexander Semenov <semenoffalex at googlemail.com
 if you are looking for an overview on content analysis research methodes, Klaus Krippendorff's book is excellent:
Krippendorff, Klaus (2004): Content Analysis. An Introduction ot Its Methodology. Thousand Oaks. Sage. 411 pages
2) Sent by: "Callahan, Ewa S. Prof." <Ewa.Callahan at quinnipiac.edu>  
Following Alexander's note, I am a big fan of Krippendorff, especially
his method of intercoder reliability testing. For his, and other methods
you can check: 
http://www.temple.edu/sct/mmc/reliability/

followed up by: 
http://www.asc.upenn.edu/usr/krippendorff/dogs.html

Content Analysis Research 
0) Sent by: Muhammad Abdul-Mageed <mumageed at yahoo.com>  
Herring, S. C. (2007). Web(log) content analysis: Expanding the paradigm. Submitted to J. Hunsinger, M. Allen, & L. Klastrup (Eds.), The International Handbook of Internet Research. Springer Verlag.

1) Sent by: James Honaker <tercer at UCLA.EDU>  
There is a project that is quite close to what you want at the ISI at USC. 
Their page, with a number of project papers and abstracts, is:

http://www.isi.edu/division3/discourse/

Their nlp goals are analysis of classroom discussion boards to 
create some degree of automated answering of student questions, direction 
to previous threads, and links between students with similar interests or 
research problems.  The project is specifically about discussion boards 
commonly used for class teaching, but other than the classroom focus it is 
very close to the subject you are seeking.
(Full disclosure, my wife worked on this project.)
2) Sent by : Henry Kim <h27kim at EMAIL.ARIZONA.EDU>  
There is a group called "Dark Web" here at Univ of AZ where computer
scientists are doing content analysis on web sites connected to terrorism
and more.  Some of the techniques they are using may be of use for your
purposes.

Content Analysis Teaching
1) Sent by: Kevin Guidry <krguidry at gmail.com> , Muhammad Abdul-Mageed <mumageed at yahoo.com>  , Alexander Semenov <semenoffalex at googlemail.com>  
You may like to have a look at the course description, and syllabus, of Prof. Susan Herring in Indiana University, for her course titled "Content Analysis for the World Wide Web". Herring has been teaching it for a long time and she has been trying to develop methodologies for analyzing the WWW. 

Web site at http://www.slis.indiana.edu/faculty/herring/.  
Specifically: http://ella.slis.indiana.edu/~herring/web.syll.06.html

The following article is her last written paper on the issue, as far as I know:

Herring, S. C. (2007). Web(log) content analysis: Expanding the paradigm. Submitted to J. Hunsinger, M. Allen, & L. Klastrup (Eds.), The International Handbook of Internet Research. Springer Verlag.

Content Analysis Software
0) Sent by: Harald Klein <intext at GMX.DE>  
An overview on text analysis software gives http://www.textanalysis.info

1) Sent by: Paul Kellstedt <kellstedt at politics.tamu.edu>  
I'm the co-editor of The Political Methodologist. Tyler Johnson wrote a
review for us of the software package WordStat, which he liked. You could
find the back issue of TPM as a .pdf at the polmeth web site.
http://polmeth.wustl.edu/thepolmeth.php
2) Sent by:  Michael Evans <evans.michael at gmail.com
Over the past few years I have experimented with various methods of
computer-assisted textual data analysis, including traditional and new
approaches to content analysis. In the process, I have developed an
expertise in two particularly useful software packages produced by
Provalis Research: QDA Miner and WordStat. 
Provalis' approach is to closely integrate several methods of textual
data analysis that have traditionally (and misleadingly) been thought
of as solely "qualitative" or "quantitative." The result is a
mixed-methods approach that does not merely provide the convenience of
having several quantitative and qualitative functions in one package,
but also allows for creative synthesizing of the functions in order to
accomplish entirely new analytical tasks.  You can learn more about QDA Miner here
(http://www.provalisresearch.com/QDAMiner/QDAMinerDesc.html) and
WordStat here (http://www.provalisresearch.com/wordstat/Wordstat.html).


Related Topic – Analysis of Public Comments on Regulations 
1) Sent by: schrodt <schrodt at ku.edu> 
Contact Stuart Shulman at the University of Pittsburgh -- he has some 
quite sophisticated software that does this sort of thing with public 
input on proposed regulations, and I assume it would work for your project 
as well. In fact I think he has already done contract work for EPA -- or 
at least I know he has analyzed environmental regulations 
2) Sent by: John D Wilkerson <jwilker at u.washington.edu>  
There are a lot of people working on public commenting in agencies in the field of information sciences. you should contact Stu Shulman (shulman at pitt.edu) who works with Jamie Callan. But as far as I know this is a pretty rich area of research in the e-government field.
3) Sent by: Russell Winn <ruwinn at nmsu.edu>  
Last summer I worked with public comments about designating Wilderness Areas in southern New Mexico. The approach we took was to treat each comment (e-mail, letter, etc.) as a separate document and then used NVIVO to code and organize the responses(NVIVO is a nice program for analyzing qualitative data but there are other programs as well). 
Most of the ‘hand coding’ was done by a group of graduate students who were enrolled in an Environmental Policy class I was teaching at the time. As a class we reviewed some of the documents, met with staff from the agency that had collected the comments, and drew up a list of potential codes (NVIVO calls them nodes) along with a brief description of when that node would be appropriate. I had every document separately coded by 2 students. If there was a disagreement about the coding the students met and agreed on a coding. Even with this reliability check there was some confusion about the coding. I ended up recoding many of the documents myself. If I were doing this again I would probably do all the coding myself, or go through a more lengthy training with the students that involved group coding of a number of the documents.
In the end we had too many nodes and the comments were a bit difficult to analyze although we did find some interesting patterns. Your project sounds more limited looking for needs and interests.
4) Sent by: Stephen Purpura <stevepurpura at gmail.com>  
I'm a PhD student at Cornell (and graduate of the Kennedy School of
Government) and I pretty much just work on automating content analysis of
government data.  I have two papers set for publication at dgo.2008 (digital
government research) on automating finding issues in regulatory rule
comments (I can send the drafts) and another in press with JITP  (see
http://www.stephenpurpura.com/20080116_HillardPurpuraWilkerson-revised.pdf).
My other publications are listed at http://www.purpuras.net/publications.html.

Laura B. Nielsen, Ph.D.
Adjunct Professor of Public Policy  
Georgetown Public Policy Institute
3520 Prospect Street, Suite 308F
Washington, DC  20007
WEB http://www.georgetown.edu/faculty/lbn4/            

Program Analyst
Environmental Analysis Division, MC 2842T
Office of Information Analysis and Access
Office of Environmental Information
U.S. Environmental Protection Agency
(202) 566-0621