[Air-L] Downloading online forum postings and analyzing occurence of keywords in the data

[Leurs, K.H.A. (Koen)]

Dear list-members,

I'm a Utrecht University based (Netherlands) researcher working on the European http://www.mignetproject.eu/ project working on transnational digital networks, migration and gender.
More specifically I'm studying knowledge and education processes across formal (university) and informal (forums, wikipedia) knowledge domains. 

I was wondering if anyone on the list has experiences with scraping online forum data and searching it for the occurence of keywords. Especially I'm
 wondering which software best suits my preferences, in selecting for instance a certain timeline to scrape as I would like to compare the curriculum of a year of university courses
and their attention for diversity issues and a year of forum postings and their attention for diversity issues. 

Scraping software that I have used so far are:
Wget

Software that I have been looking at for searching keywords in digital data are:
Provalis Research Wordstat 
NVivo
Digitalmethods.net > issuecrawler 

I'm looking forward to any suggestions, and thank you for your time.

Kind regards,

Koen. 


Koen Leurs | Aio / PhD student Gender Studies  |  Research Institute for History and Culture  | 
 |  Utrecht University, the Netherlands  |  Muntstraat 2a  | 3512 EV Utrecht  |  Room 1.12  | 
 www.koenleurs.net  |  www.uu.nl/wiredup | www.mignetproject.eu/ |