Large-scale computational approaches to evolution and change prospects and pitfalls

To understand how and why a complex system like language works, understanding how it changes is key. Among the multitude of possible approaches to studying language change dynamics, the focus in this workshop is on the empirical study of large collections of linguistic data such as corpora and lexical databases. Scale necessitates computation: no human alone could read through billions of words fast enough. But machines can.

There is no time like now to apply machine learning to language. Advances in the NLP fields of semantic shift and lexical semantic change detection yield increasingly accurate automated inferences (Schlechtweg et al. 2020; Montanelli & Periti 2023, Tahmasebi et al 2021), primarily driven by various (large) language models. Machine-readable diachronic data at both short and long time scales has become abundant thanks to corpus building and digitization efforts, and gold standard test sets are available to objectively evaluate different approaches (e.g. Schlechtweg et al. 2021). Developments in generative LLMs have also come to a point where previous complex NLP pipelines and costly supervised learning architectures can often simply be replaced with zero-shot LLM queries without loss in performance (Ziems et al. 2023; Karjus 2023).

These approaches however don't come without limitations. While the availability of pretrained LLMs opens up new and simplifies previously complex research avenues, it is important to take into account and mitigate their biases (Dubossarsky et al. 2017) (which are inherent to any pretrained model). While research rooted in NLP often focuses rather on the what (has changed), the what can inform the how and why. It is therefore crucial to embed them in theoretically meaningful frameworks, and furthermore, delineate which detected changes may be driven by inherent linguistic mechanisms like selection and drift (Montero et al. 2023) or general cognitive factors like optimal encoding strategies in our mental categories (Dubossarsky et al. 2015), in contrast to those reflecting changes in the socio-cultural aspects of the language communities and their communicative needs (Kemp et al. 2018; Karjus et al. 2020; De Pascale & Marzo 2023).

This workshop seeks to bring together language evolution, cutting-edge NLP, and LLM-driven approaches, to critically discuss novel opportunities of large-scale empirical approaches to language evolution and change (cf. Hartmann 2020), but also the aforementioned issues. Submissions will be short abstracts, assessed by rigor and relevance to the following questions and themes:

• How to combine large-scale computational language change detection with meaningful inference frameworks, and cognitively plausible theories of language evolution mechanisms?
• To what extent are LLMs as zero-shot classifiers and inference engines applicable to the study of language change?
• How to tease apart linguistic and the socio-cultural drivers of change at scale?
• If and how can applied NLP benefit from evolutionary thinking?
• How to evaluate and mitigate bias in pretrained language models? This includes issues of applying models trained on modern data to historical material, as well as potentially harmful social and other biases that a model may propagate from (at times unknown) training data.
• How can experimental methods from cognitive science, psychology or social sciences be used to study machine bias or “behavior”? What are possible pitfalls of such method transfer?

Schedule

References