[Air-l] Re: Survey Methodology on the WWW

Fri Nov 2 06:31:22 PST 2001

That would be an interesting collection of stories... it is something that
I hear all the time.  Especially given that the perception is out there
that creating your own web survey is easy to do.  I take calls weekly from
folks who are in a crisis situation, with servers crashing (or data
missing), wondering what single software package they could buy to sweep in
and save the day.  And in such situations, I usually find out that there
are such large data quality issues with their survey, that what they
perceive as their worse problem is actually fairly minor.

I have equated good quality web survey research with good quality
e-commerce site management.... with an added complexity of having a higher
level of interactivity and a more complex data structure in the back end.
So, when someone asks me if they could do web surveys... I ask back if the
would feel comfortable setting up and managing an e-commerce store with
multiple products and shipping possibilities.  If the answer is no, then I
recommend against trying to do a web survey.

As for actually doing a collection of stories... I'd be happy to contribute
one if such a compilation was pulled together.

**********************************
Scott Crawford
Research Director - Social Science Research
http://www.ms-interactive.com
734/542-7796
734/661-0323 (fax)

                    "John White"                                                                                 
                    <John.White at w        To:     air-l at aoir.org                                                  
                    ku.edu>              cc:                                                                     
                    Sent by:             Subject:     Re: [Air-l] Re: Survey Methodology on the WWW              
                    air-l-admin at a                                                                                
                    oir.org                                                                                      

                    11/01/01                                                                                     
                    02:26 PM                                                                                     
                    Please                                                                                       
                    respond to                                                                                   
                    air-l                                                                                        

Scott,

Many Thanks for the reference!!  I hope I speak for many others in asking
for a
copy of manuscript or slides or whatever you are willing to share! :))

In terms of anecdotes, I'm curious about how/why many of us begain using
the
internet as a methodology or even to study.  In my own case, I was
technically
savy in the sense I knew what the internet was, something about the
technical
structure, and particulary the usefulness of coding using web based
technology.
What I wasn't prepared for was the frustration, in terms of learning new
technologies, outages, using resources that belonged (and hence controled)
by
someone else, deletion of files (once by a student running the server, once
by
myself), the disappearnace of a server, updating software on a server and
the
webmaster leaving on vacation the same day, dealing with Human Subjects,
etc...etc....etc...in short, the dirty nasty little details that never get
written up in our formal reports.

Are there any collections of these stories?  And if not, is there any
interest
in begining one?

Just a few thoughts...

--JW

Scott_Crawford at marketstrategies.com wrote:

> Overall... to answer these questions, one great place to start is:
> Couper, Mick P. (2000) "Web Surveys:  A Review of Issues and Approaches"
> Public Opinion Quarterly.  V. 64:464-494.
> It includes over 60 references to other papers/presentations that cover
the
> topic.
>
> But more specifically...
>
> >1)    What are the differences between paper & pencil methods
> >versus web (including but not limited to cost, design, response, etc)
> I was just involved in a large scale mode experiment between web and
mail.
> Results are to be presented at next weeks "Higher Education Center and
the
> U.S. Department's 15th Annual National Meeting on Alcohol, Other Drug,
and
> Violence Prevention" Conference in DC (http://www.edc.org/hec/natl/2001/
).
> For anyone interested, I would be happy to share our slides after the
> presentation is given.  We will also be presenting results from this same
> study as an invited paper at the 2002 International Conference on
Improving
> Surveys (http://www.sfi.dk/).
>
> >2)    What, if any, research exists regarding best practices?
> Not a lot yet... considering.  There are a lot of "statements of best
> practices" and other non scientifically based standards.  However, the
> arena of web survey methodology has just begun to be explored seriously.
> Next year at this time, I would expect to see a considerably more stable
> standard practice guide out there.  But good quality research in methods
> takes time.  Anyone asserting that they have "figured out" web research
at
> this point of the game doesn't understand all of the implications.
>
> >3)    Surprises, anecdotes, and special skills required to become a
> >seasoned 'net researcher.
> I'd be happy to talk and share my thoughts.  From my perspective... it
> takes a good amount of research skills... coupled with technology
saavy...
> coupled with a general understanding of sociology, psychology,
> communications.
>
> **********************************
> Scott Crawford
> Research Director - Social Science Research
> http://www.ms-interactive.com
> 734/542-7796
> 734/661-0323 (fax)
>
> _______________________________________________
> Air-l mailing list
> Air-l at aoir.org
> http://www.aoir.org/mailman/listinfo/air-l

_______________________________________________
Air-l mailing list
Air-l at aoir.org
http://www.aoir.org/mailman/listinfo/air-l