[Air-L] Toolkit for collecting instant messaging data and running controlled experiments

[Gregory Mills]

Dear AoIR colleagues,

We have just released a toolkit for collecting naturalistic instant 
messaging data, as well as running controlled experiments on 
algorithmically mediated conversation.

The toolkit runs as a bot on the Telegram network, which participants 
connect to using the Telegram app on their mobile phones.   Since all 
messages between participants are mediated via the toolkit, data is 
instantly available for analysis - there is no need for participants to 
send in their data.  The toolkit allows high levels of experimental 
control: Participants can be dynamically assigned to different groups, 
and experimental instructions (e.g. images, texts, questionnaires) can 
be sent automatically by the toolkit to the participants in their chat 
window.

The toolkit allows high-levels of experimental control over the content 
and timing of participants’ turns. Turns can be selectively reordered, 
or blocked, e.g. to investigate the effects of shadowbanning. The 
toolkit also allows individual words, phrases, emoticons, etc. to be 
filtered from participants' turns. This technique can also be used to 
insert "spoof" turns into the dialogue, that appear, to participants to 
originate from each other - allowing direct testing of hypotheses about 
how participants respond to specific prompts.

In addition, the toolkit provides a set of customizable, (non-Telegram ) 
chat interfaces for investigating how different interfaces affect online 
conversations.


The toolkit is open-source and is downloadable at:

http://cogsci.eecs.qmul.ac.uk/diet/

Best,

Gregory Mills
g.j.mills at rug.nl