[Air-L] Soliciting opinions about using Amazon's Mechanical Turk (aka Mturk) for survey participant recruitment

[Leticia Bode]

Other articles that address this issue are Berinsky, Huber, and Lenz 2012
(Political Analysis) and Chandler, Mueller, and Paolacci 2013 (Behavioral
Research Methods). This is a worthwhile blog post in thinking through some
of the issues too -
http://www.culturalcognition.net/blog/2013/7/10/fooled-twice-shame-on-who-problems-with-mechanical-turk-stud.html
.
Broadly I agree with others that it depends on the question you're asking.




On Wed, Jan 8, 2014 at 8:09 PM, Tim Muntinga <munt.tim at gmail.com> wrote:

> Additionally, keep in mind the nature of Mechanical Turk; is there an
> incentive for the specific group you are targeting to deliver high quality
> entries? With the amounts of projects executed on a weekly basis, problems
> of satisficing and routine knowledge of internal tests occur. I find this
> for surveys quite an important point to consider.
>
> See for more:
> Kapelner, A., & Chandler, D. (2010). Preventing Satisficing in online
> surveys.
>
> *----*
> Tim
> *W: *www.timmuntinga.com
>
>
>
> 2014/1/9 Aaron S. Veenstra <aaron at etchouse.com>
>
> > If you're doing experiments, MTurk is great, especially if your likely
> > alternative is a sample of undergrads. If you're doing surveys, it's
> > not. The U.S. users are not representative of the American population,
> > and I'd imagine the same is true of other countries' MTurk
> > populations. And even if it were, there's no way to prove that to any
> > kind of satisfaction. Basically, using it for a survey that you want
> > to generalize to the population isn't getting you a sample that's any
> > better than posting the survey to Facebook and Twitter and asking
> > people to spread it around.
> >
> > FWIW, the skew I've seen with MTurk samples compared to the U.S.
> > population is a) slightly too male, b) slightly too white, c) too
> > young (though older and with more age variance than an undergrad
> > sample), d) average income too low, and e) too liberal (though, again,
> > less skew than I'd expect from undergrads).
> >
> > Aaron
> >
> > On Wed, Jan 8, 2014 at 6:40 PM, Robinson,Cory
> > <cory.Robinson at colostate.edu> wrote:
> > > While Qualtrics and SurveyMonkey seem to be generally well respected,
> > they also come with hefty price tags for participant recruitment ($5-10
> per
> > American respondent). Amazon’s Mechanical Turk (aka Mturk), on the other
> > hand, can recruit participants for far less (<$1/respondent).
> > >
> > > What is the AoIR consensus on utilizing Mturk? I’ve seen articles both
> > for and against using the service.
> > >
> > > Thanks in advance for all opinions/insights.
> > >
> > > - Cory Robinson
> > > --
> > > Stephen Cory Robinson
> > > cory.robinson at colostate.edu<mailto:cory.robinson at colostate.edu>
> > > Office: Clark C258A
> > > http://colostate.academia.edu/StephenCoryRobinson
> > >
> > > _______________________________________________
> > > The Air-L at listserv.aoir.org mailing list
> > > is provided by the Association of Internet Researchers http://aoir.org
> > > Subscribe, change options or unsubscribe at:
> > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> > >
> > > Join the Association of Internet Researchers:
> > > http://www.aoir.org/
> >
> >
> >
> > --
> > Aaron S. Veenstra
> > Assistant Professor, Southern Illinois University Carbondale
> > School of Journalism || 1232 Comm Building
> > asveenstra at siu.edu || manytoomany.com
> > _______________________________________________
> > The Air-L at listserv.aoir.org mailing list
> > is provided by the Association of Internet Researchers http://aoir.org
> > Subscribe, change options or unsubscribe at:
> > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> >
> > Join the Association of Internet Researchers:
> > http://www.aoir.org/
> >
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at:
> http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
>