[Air-L] Conference review, “sampling procedure,” and interdisciplinarity

[Huatong Sun]

Dear Terri,

I was Reviewer 3 for your roundtable proposal who asked about sampling
procedure. Thanks for your feedback to my review, and I appreciate having
this chance to engage in a discussion of exploring the interdisciplinarity
of the AOIR. Maybe you don’t remember, I met you and Nicole in the CSCW
workshop on digital identity in Chicago in 2004, and I enjoyed chatting
with you about your book project of camgirls at that time. In this message,
I'd like to clarify about some misunderstanding about my review and share
my thoughts. Hopefully this side of reviewer’s story will help improve the
quality of future AOIR conferences, :).

A disclaimer first: I saw most of the current discussion about the AOIR
review system started from this year’s paper template; however, I didn’t
use that template to review your proposal, and I reread the CFP part
concerning the roundtable session to make sure I understood the expectation
of the conference organizers for this format before reviewing your piece
(since I didn’t get particular review guidelines for roundtable proposals).
And I do support work-in-progress submissions: I regularly submit this type
of work for feedback myself and I organized a review of this type of
submissions for a conference last year.

Let’s go back to the review. A review is just a review, one person’s
opinion about certain research, and I’m upset to see such a review is
misinterpreted in this context of a discussion on how to make an
interdisciplinary conference better. So far I only attended AOIR once (as
many interdisciplinary researchers with small travel budgets on this list,
I have to be highly selective on the conferences I go). While I enjoyed the
fresh ideas at AOIR, I was disappointed to see some presentations only had
the depth of news reporting, which I regarded as an issue of description
vs. interpretation in research. Of course your proposal is much more than
that, but I don’t want to deny that my past experience influenced my
review. I wanted to see more of the details, as I wrote in the comment. I
guess one could understand my feelings if s/he serves on a job search
committee: There are always those moments of disappointment when you see a
stellar job applicant looks glorious on paper articulates his/her research
framework poorly in a phone interview.

By all the means, I’m sorry about the misunderstanding and frustration that
came from my unsophisticated use of “sample” in review comments. I’m a
qualitative researcher, and I don’t do experimental social science research
at all. When I asked about “sampling procedures” in review, I simply wanted
to know how you chose your cases for cross-cultural comparison because I
reviewed many cross-cultural studies that picked up their sites randomly
without justification. The lesson I learned from this case is that I would
be much more careful about my wording in review in the future, particularly
for this type of interdisciplinary conferences. Even though I didn’t intend
to, people could misinterpret the connotation that go with certain words. I
apologize about this misuse.

On the other end, I was wondering whether the sampling question lacked
legitimacy if it had been raised by an experimental social scientist. Isn’t
one part of the joys of attending this kind of interdisciplinary
conferences is to have our ideas collided in different perspectives? Yes,
we are looking for camaraderie in professional communities, but we also
want to see our ideas inquired and challenged by people who share research
interests in similar topics but employ different research methodologies. Or
maybe are we just still so discipline-rooted?

The conference review process is always an interesting and heated topic for
discussion. I’d like to recommend Jonathan Grudin’s recent piece:
“Varieties of Conference Experiences” (The Information Society, 29: 71–77,
2013). Citing Anderson’s research, he wrote: “A selective conference
accepts perhaps 5 percent that most experts would agree are strong,
dismisses about 50 percent that attract no positive reviews, and arguably
conducts a lottery to select among the rest to fill the remaining slots”
(p. 75). For an interdisciplinary conference like AOIR, it is not a
surprise that we have arguments about those selected papers. Yes, my
submission was selected this year, but the one before this was rejected. As
a writing scholar, I saw the problem of the paper template, and I discussed
it in length with my colleague Jim Porter as we both tried hard to fit our
own papers into that template before the deadline. I hope we are able to
find a better system to review papers, and this is why I wrote to share
another side of the story.

Terri, I feel your pain about center and marginality, as my work was
considered marginal. I still remembered how I was stunned to find that I
was the only one who had a different skin color from dozens of attendees in
a big meeting room as I was respectfully nodding my head and earnestly
taking notes of the feedback for my dissertation proposal at a graduate
research network years ago. In retrospect, I’m grateful for the critical
(and sometimes brutal) feedback. It has taught me how to negotiate in a
milieu of diverse perspectives, learn to be open-minded, and not to be
offended by the face value of the words; of course it helped me improve my
project eventually. I always use that experience to remind myself to be
supportive to new work.

I hope this note clarifies some confusions and misunderstanding about my
review. I’m looking forward to reading more exciting work from you!

Best,

Huatong


------------------------------------------------------------------
Huatong Sun, Ph.D.
Assistant Professor of Digital Media Studies
Interdisciplinary Arts and Sciences
University of Washington Tacoma
http://faculty.washington.edu/htsun/

Book: Cross-Cultural Technology Design: Creating Culture-Sensitive
Technology for Local Users
http://global.oup.com/academic/product/cross-cultural-technology-design-9780199744763