g-vamp: Improving sequence labelling frameworks for multiword expressions identification

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Annotated corpora such as the parseme collection frequently contain Multiword Expressions ( mwe s) that share tokens within a sentence, meaning that a single token may belong to two or more mwe s. This phenomenon is widespread across languages and poses a significant challenge for machine learning models that rely on standard sequence-labelling formulations. Token sharing increases training complexity and often leads to systematic prediction errors. As a result, most existing approaches resolve these conflicts by retaining only one of the overlapping mwe s and discarding the others, which entails substantial information loss and can be particularly detrimental in low-resource settings. This paper introduces g-vamp (Generate Variants and Merge Predictions), a general framework that enables mwe identification systems to handle token sharing without discarding annotated expressions or relying on complex encodings. g-vamp operates by generating multiple variants of the training corpus in which token-sharing conflicts are eliminated in a controlled manner. A Transformer-based model is trained independently on each variant, and the resulting predictions are subsequently merged to recover mwe s that involve shared tokens at inference time. We evaluate g-vamp on version 1.3 of the parseme multilingual corpus, covering 26 languages with diverse structural and typological characteristics. The results show that g-vamp consistently improves over a strong baseline in 24 languages and surpasses state-of-the-art systems in 16 languages, despite relying exclusively on a single multilingual Transformer model and no language-specific linguistic resources. These findings demonstrate that controlled corpus diversification constitutes an effective and language-agnostic strategy for addressing token sharing in mwe identification.