Skip to Main content Skip to Navigation
Conference papers

Choisir le bon co-équipier pour la génération coopérative de texte

Abstract : Language models (LM) generate texts by successively predicting probability distributions for next tokens given past ones. In order to generate texts with some desired properties (eg. being more natural, non toxic, or having a specific writing style...), recent approaches use a classifier to guide the decoding of the LM distribution towards relevant texts with the expected property. In this paper, we examine three families of (transformer-based) discriminators for this task of cooperative decoding : bidirectional, left-to-right and generative ones. We evaluate the pros and cons of these different types of discriminators for cooperative generation, exploring their respective accuracy on classification tasks, their impact on the resulting sample quality and their computational performance. We also provide the batched implementation of the powerful cooperative decoding strategy used for our experiments, the Monte Carlo Tree Search, working with each discriminator for Natural Language Generation.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03701506
Contributor : Yannick Parmentier Connect in order to contact the contributor
Submitted on : Friday, June 24, 2022 - 4:42:12 PM
Last modification on : Friday, August 5, 2022 - 2:54:52 PM
Long-term archiving on: : Sunday, September 25, 2022 - 9:36:59 PM

File

6393.pdf
Publisher files allowed on an open archive

Identifiers

  • HAL Id : hal-03701506, version 1

Citation

Antoine Chaffin, Thomas Scialom, Sylvain Lamprier, Jacopo Staiano, Benjamin Piwowarski, et al.. Choisir le bon co-équipier pour la génération coopérative de texte. TALN 2022 - 29e conférence sur le Traitement Automatique des Langues Naturelles, Jun 2022, Avignon, France. pp.12-26. ⟨hal-03701506⟩

Share

Metrics

Record views

58

Files downloads

20