The origins of large language models: an historical perspective on computational language modeling

Séquence du séminaire les Rencontres de Statistique Appliquée 

Brainpower Meets Machine Power: Chat-GPT synergies with social scientists

Benoît Crabbé (Paris Cité University)

This talk will provide elements on the history of language modeling with the aim to shed light on the nature of current large language models. We start from the original insights of A. Markov, C. Shannon and W.Weaver and contrast them with the insights of N. Chomsky and symbolic AI. We will provide elements explaining the failure of symbolic AI for language and the generalization of statistical methods for language modeling. The last step of this evolution is the so-called deep learning paradigm that gave birth to current Large Language Models.