Dynamic Multi-Window Differential Attention for Named Entity Recognition

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP) that focuses on identifying and classifying named entities in text.Despite significant progress in deep learning models, challenges remain in handling long sequences, capturing long-range dependencies, and effectively integrating local and global context.In this paper, we introduce a novel Multi-Window Differential Attention Mechanism (MW-DAM) for NER.Our method enhances the traditional multi-window attention mechanism by incorporating a differential attention approach that suppresses noise, thereby enhancing focus on relevant features.Specifically, we compute the difference between two distinct attention distributions, which induces sparsity in the attention map and reduces interference from irrelevant context.We also propose a dynamic window mechanism that adapts the context window size based on the input sequence length, to optimally capture of both local and global context.Experimental results on several benchmark NER datasets demonstrate that our model outperforms existing methods, achieving superior performance in terms of F1-score and robustness, particularly in the presence of noisy or incomplete data.Our model also significantly improves performance on long sequences and long-range dependency modeling.