[Air-l] Great Ethical disasters in Internet Research?

[Yvonne Waern]

Sorry to be so late in reacting. The issue caught my interest after 
three weeks on a Swedish Island - Gotland.  Sunny! So I may need some 
intellectual change!  Here is my reaction to the discussion.

>Thanks for putting the homenet study as a warning example!  I have 
>used it myself in a course in methodology. Useful to have flawed 
>studies!  Did you see, for example, that the web-formt of the paper 
>does not allow you to see the figures?  I had to get the original 
>paper!


I also saw that somebody asked for the "flaws" in the study. I have 
seen a couple:
1) the sample of respondents is not random. This is common in 
psychological research, but here, the sample is probably rather 
biased. People did get a computer to try. How many have not already 
got one? Who are the people who got the computer?
2) the data are mostly based on ratings. Ratings have a tendency to 
get biased if they are repeated. There is something that is called 
"regression toward the mean". This means for instance that if you are 
very positive at the beginning, the average will be lower at a 
repeated measure, just because the average was above the (supposed) 
mean from the beginning. Thus, one possible bias can be the 
following: people were happy to get a computer. Later, they, probably 
got disappointed with the utility of the computer. This affects all 
ratings.
3) the actual "significant" differences are VERY small!  You have to 
read the original paper. I usually do not recommend my students to 
talk about effects in this range at all!

Good methodology and ethics go together in research, as many have 
proposed. We are responsible to our subjects as well as to other 
researchers to do as good a research as possible. Why do we else have 
research at all?

Yvonne

>
>1. Carnegie Mellon's HomeNet study
>
>the study:
>http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/copetas/www/public/pr/aug31-98.html
>
>reaction:
>http://www.columbia.edu/cu/21stC/issue-3.4/featherstone.html
>
-- 
Yvonne Waern
Professor emerita
CMC-research group
Computer and System Sciences
Stockholm University /KTH
Forum 100, S-164 40 Kista, Sweden
Phone (home): +46 8 500 307 18