Home | Tasks | CLEF program | Publications | Partners | Contest | Contact Us | Tools | CLEF 2022
CLEF 2023 JOKER Track:
Automatic Wordplay Analysis
Tasks, evaluation setup, and metrics
In the edition, we propose three shared tasks:
- Task 1: Pun detection in English and French. We will use the corpora which were a side result of JOKER-2022  and the augmented data of SemEval-2017  as well as standard evaluation metrics for binary classification (precision, recall, accuracy)
- Task 2: Pun location and interpretation in English and French. We will use the extended corpus of JOKER-2022 Task 1. We will evaluate location based on the accuracy and the interpretation based on paronyms and synonyms.
- Task 3: Pun translation from English to French. We will use the extended version of the corpus of JOKER-2022 Task 3 and we will continue to evaluate the generated translation manually.
 L. Ermakova et al., “Overview of the CLEF 2022 JOKER Task 3: Pun Translation from English into French”, in Proceedings of the Working Notes of CLEF 2022: Conference and Labs of the Evaluation Forum, 2022.
 T. Miller, C. Hempelmann, and I. Gurevych, “SemEval-2017 Task 7: Detection and Interpretation of English Puns”, in Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), Vancouver, Canada, 2017, p. 58‑68. doi: 10.18653/v1/S17-2005.
This project has received a government grant managed by the National Research Agency under the program "Investissements d'avenir" integrated into France 2030, with the Reference ANR-19-GURE-0001.
JOKER is supported by The Human Science Institute in Brittany (MSHB)