NLP Seminar Series: Large Generative Models for a smaller language: Finnish GPT-3 – Sampo Pyysalo
Avsnittsinformation
Avsnittsbeskrivning
Title: Large Generative Models for a smaller language: Finnish GPT-3 Speaker: Sampo Pyysalo, Research fellow at the University of Turku Abstract: Large generative language models have demonstrated remarkable capabilities in a broad range of tasks within natural language processing and beyond. While increasing numbers of models are becoming openly available, the most capable models remain closed, and open models are limited in their coverage of smaller languages. This presentation introduces open GPT-3 -like models for Finnish, including monolingual models of sizes up to 13B parameters trained from scratch and a 176B-parameter multilingual model created by continuing the training of the largest BLOOM model. The talk will cover challenges and solutions in collecting sufficient data for training, data preprocessing and cleaning, training on the LUMI supercomputer, and the introduction of new evaluation datasets, and conclude with discussion of ongoing and future work building on the models. May 10, 2023 The NLP Seminar Series at AI Sweden is a bi-weekly forum for people who work with or are interested in Natural Language Processing (NLP). It is organized by AI Sweden and RISE NLP Group. AI Sweden is the Swedish national center for applied artificial intelligence. Our mission is to accelerate the use of AI for the benefit of our society, our competitiveness, and for everyone living in Sweden. Website: https://ai.se Contact: info@ai.se Follow us Newsletter: https://www.ai.se/en/AI-Sweden-newsletter LinkedIn: https://www.linkedin.com/company/aisweden Medium: https://medium.com/ai-sweden