Skip to Main content Skip to Navigation
Conference papers

Investigating associative, switchable and negatable Winograd items on renewed French data sets

Abstract : The Winograd Schema Challenge (WSC) consists of a set of anaphora resolution problems resolvable only by reasoning about world knowledge. This article describes the update of the existing French data set and the creation of three subsets allowing for a more robust, fine-grained evaluation protocol of WSC in French (FWSC) : an associative subset (items easily resolvable with lexical co-occurrence), a switchable subset (items where the inversion of two keywords reverses the answer) and a negatable subset (items where applying negation on its verb reverses the answer). Experiences on these data sets with CamemBERT reach SOTA performances. Our evaluation protocol showed in addition that the higher performance could be explained by the existence of associative items in FWSC. Besides, increasing the size of training corpus improves the model’s performance on switchable items while the impact of larger training corpus remains small on negatable items.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03701511
Contributor : Yannick Parmentier Connect in order to contact the contributor
Submitted on : Friday, June 24, 2022 - 4:42:29 PM
Last modification on : Friday, August 5, 2022 - 11:39:55 AM
Long-term archiving on: : Sunday, September 25, 2022 - 9:38:02 PM

File

7675.pdf
Publisher files allowed on an open archive

Identifiers

  • HAL Id : hal-03701511, version 1

Citation

Xiaoou Wang, Olga Seminck, Pascal Amsili. Investigating associative, switchable and negatable Winograd items on renewed French data sets. Traitement Automatique des Langues Naturelles (TALN 2022), Jun 2022, Avignon, France. pp.136-143. ⟨hal-03701511⟩

Share

Metrics

Record views

9

Files downloads

5