[Air-L] Let's talk about AoIR.

[Raquel Recuero]

Hi everyone!

I'd like to make a few suggestions and perhaps add some ideas to the 
discussion:

_1) About the new template:_
If I may talk for us in South America and specially in Brazil... Event 
though I think the new "short paper" (for us it is actually an extended 
abstract) is better than 500 words to allow you to argue a bit more 
about your research, the new model made hard for us to get funding to 
attend the conference because there is no full paper option. Before, we 
could submit full papers (and we actually did and the review process to 
get published in SPIR was *awesome*).

*Suggestion:* I know this will sound very ACM/AAAI/IEEE etc.but... Why 
don't we create short/full paper proposals? Short papers are ideal for 
folks who have ongoing research and want to discuss it (maybe without 
publication) and full papers for those other folks (like me) who need to 
present something more complete in order to get funding. Also perhaps a 
way to differentiate roundtables and panels would be the format of 
papers (and the completeness of the research): roundtable = short 
papers; panel= full papers. We can also have different types of proposal 
for other things that contemplate other types of discussion. SPIR could 
continue on publishing full papers and the proceedings, short ones.

_2) About reviews:_
Although I wasn't a reviewer this year and I think people are really 
generous to volunteer (I forgot to volunteer - shame on me!), it is 
pretty much a fact by now that while some reviews were awesome, some 
were not so good. My panel got through, but with odd reviews that only 
focused on one or two of our six papers. This seems to point out that 
reviewers are having some difficulties in understanding what they are 
evaluating (not one paper, but a panel of six) and how they should do it 
(perhaps the system we used is somewhat flawed). Also, my students that 
got rejected, like many people in this list, got complete opposite 
reviews with also complete opposite marks (like 90 and 24, for example). 
And I felt it was very discouraging for them (as for everyone else) to 
submit again because the process was so weird. It is normal to get 
rejected. But at least it seems fair to have *some* coherence in the 
rejection/approval process.

*Suggestion:* We can create reviewer guidelines for different types of 
proposals. I also like the bidding process someone suggested and the 
idea of creating a program committee or something in order to credit 
properly the people who are doing the hard work. Maybe create meta 
reviewers would also help the conflicts (and I think someone already 
suggested this).

_3) About AOIR__Conference_
That said, I would like to point that I *love* AOIR conferences. I've 
been to several others and still think we get more discussion and more 
debate with our papers in IR than in any other. Also, it is very much 
likely to find similar works and people with similar interests. However, 
I also feel that in the last conferences the abstract model was too 
short and many very very early works that perhaps were not yet polished 
enough to be presented were accepted. I think we need better ways to 
select good work (which doesn't mean it has to be finished) and the 
"short paper" was a step towards it. The conference, for me, is about a 
discussion, about exchanging ideas and we need good work to inspire them.

*Suggestion*: If people feel the conference is becoming too broad, maybe 
we should think about creating tracks that represent the lines of 
research/objects of our associates. That would also help more focus on 
the discussions and would also help the reviewers to know what to expect 
and how to evaluate papers from each track. We could have, for example, 
a methodological discussion track.



Sorry for the long email. :)

Best,
Raquel