[Air-L] Request: Stop sending reviewer scores to authors

[Jill Walker Rettberg]

I would like to make a formal request to the AoIR board that for future conferences, the organization NOT send out reviewer scores to authors.

I get why scores are useful for the selection process but it’s hard to see how they are helpful for the author, and easy to see how they can be harmful and demotivating. I’m in a secure position and can handle bad reviews, but if I had received the scores I received this year as a junior scholar I would have been devastated.

The reviewer comments are useful to authors – thank you to all the reviewers for the hard work, and for many constructive comments that will be helpful for authors developing their ideas. But there is no reason to send the scores to authors.

There is also so much research showing the arbitrariness of peer review (it’s not random but there is a LOT of subjectivity involved) that putting such apparently objective scores out is really misleading. I mean, even national and transnational research funders who pay reviewers and have really robust systems for checking and double checking reviews have problems setting scores. It’s basically impossible to do in a completely fair way, and even more so in a volunteer system like ours where people don’t have much time and reviewers are often assigned to assess papers on topics and using methodologles they’re not familiar with. So let’s not make the scores part of the feedback to authors.

Not sending scores to authors is an easy fix, and we don’t even have to get into all the annual discussions about do 1200 word abstracts even work, are humanities perspectives properly evaluated by reviewers, how big should the conference be, is it cool that some people present multiple things while others are rejected completely etc.

Are there ANY reasons to send scores to authors?

Jill


Jill Walker Rettberg
Professor of Digital Culture
University of Bergen

Principal Investigator of the ERC Consolidator grant Machine Vision in Everyday Life: Playful Interactions with Visual Technologies in Digital Art, Games, Narratives and Social Media. https://www.uib.no/en/machinevision/