Overview

Lexical Semantic Change Detection (LSCD) studies how word meanings change over time. Existing benchmarks have mostly focused on comparing two time periods, but semantic change is often gradual and unfolds across decades. This task extends the evaluation setting from two time periods to ten, enabling the modeling of individual word senses and their dynamics over time.

The task is based on a novel, multilingual benchmark covering Swedish, English, Italian, Spanish, Dutch, and Russian, with manually annotated data across ten time periods and approximately 126,000 human annotations.

Task Description

Subtask 1: Diachronic Word Sense Induction

Given dated usages of a target word, participants must identify which senses the word has in which time periods in an unsupervised manner.

• Subtask 1a: Diachronic sense assignment: determine which usages belong to which sense.
• Subtask 1b: Sense dynamics: track how the distribution of individual senses changes over time.

Subtask 2: Hypothesis-Driven Change Detection

Participants are given a specific sense description and must identify which usages across time instantiate that sense. This subtask reflects common research scenarios in the humanities and social sciences, where researchers are interested in tracing one particular meaning.

Data

The benchmark contains dated usages from historical corpora spanning approximately 1880–2023. Each language is divided into ten time periods of roughly equal length.

• Swedish: 40 target words, plus 10 development words
• English: 30 target words
• Italian, Spanish, Dutch, and Russian: 15 target words each

For each target word, usages are sampled from each time period. Each usage is manually annotated by at least three annotators.

How to partecipate

The data and the evaluation will be hosted in Codalab. The link to Codalab will be provided soon.

Important Dates

• To be announced.

Organisers: Nina Tahmasebi, Pierluigi Cassotti Felix Morger, Lucia Siciliani, Eduardo Calò, Pablo Mosteiro, Stefano De Pascale, and Mariia Fedorova

References:

• Nina Tahmasebi, Adam Jatowt, Lars Borin. Survey of Computational Approaches to Lexical Semantic Change Detection. Nina Tahmasebi, Lars Borin, Adam Jatowt, Yang Xu, Simon Hengchen (eds). Computational Approaches to Semantic Change. Berlin: Language Science Press.
• Francesco Periti and Nina TahmasebiA Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change. (2023) In Proc. of NAACL2024
• Stefano Montanelli and Francesco Periti, Lexical Semantic Change through Large Language Models: a Survey. ACM Computing Surveys (2024)
• Pierluigi Cassotti, Lucia Siciliani, Marco DeGemmis, Giovanni Semeraro, Pierpaolo Basile, XL-LEXEME: WiC Pretrained Model for Cross-Lingual LEXical sEMantic changE. (2023) In Proc. of ACL2023
• Simon Hengchen, Nina Tahmasebi, Dominik Schlechtweg, Haim Dubossarsky. Challenges for Computational Lexical Semantic Change. Nina Tahmasebi, Lars Borin, Adam Jatowt, Yang Xu, Simon Hengchen (eds). Computational Approaches to Semantic Change. Berlin: Language Science Press.