[Air-L] CLEF 2023 LongEval Lab on Longitudinal Evaluation of Model Performance

[Elena Kochkina]

Call for Participants

LongEval: Longitudinal Evaluation of Model Performance
CLEF 2023 Lab

18-21 September, Thessaloniki, Greece

--------------------------------------------------------

** Registration open, training data available **

The CLEF 2023 LongEval lab is motivated by recent research showing that the performance of information retrieval and text classification models drops as the test data becomes more distant in time from the training data. LongEval differs from traditional IR and classification shared tasks with special considerations on evaluating models that mitigate performance drop over time. It encourages participants to develop temporal information retrieval systems and longitudinal text classifiers that survive through dynamic temporal text changes, introducing time as a new dimension for ranking models performance.

The lab consists of two tasks:
* Task 1. LongEval-Retrieval: The goal of Task 1 is to propose an information retrieval system which can handle changes over the time. The proposed retrieval system should follow the temporal timewise evolution of Web documents. Contact: longeval-ir-task at univ-grenoble-alpes.fr<mailto:longeval-ir-task at univ-grenoble-alpes.fr>

* Task 2. LongEval-Classification: The goal of Task 2 is to propose a temporal persistence classifier which can mitigate performance drop over short and long periods of time compared to a test set from the same time frame as training. Contact: r.a.a.alkhalifa at qmul.ac.uk<mailto:r.a.a.alkhalifa at qmul.ac.uk>

Registration
------------
Please use the CLEF registration form (http://clef2023-labs-registration.dei.unipd.it/) to register to the tasks. The registration closes on Friday, 28 April 2023.

Timeline
--------
April 2023: Test data release
28 April 2023: Registration closes
10 May 2023: End of Evaluation Cycle [submission of runs]
5 June 2023: Submission of Participant Papers [CEUR-WS]
5–23 June 2023: Review process of participant papers
July 2023: Camera ready paper submission
September 2023: CLEF Conference

Organisers
----------
Rabab Alkhalifa, Iman Bilal, Hsuvas Borkakoty, Jose Camacho-Collados, Romain Deveaud, Alaa El-Ebshihy, Luis Espinosa-Anke, Gabriela Gonzalez-Saez, Petra Galuščáková, Lorraine Goeuriot, Elena Kochkina, Maria Liakata, Daniel Loureiro, Harish Tayyar Madabushi, Philippe Mulhem, Florina Piroi, Martin Popel, Christophe Servan, Arkaitz Zubiaga.