Automated Selection-based Mixture-of-Experts with Dual-stage Input–Target Pattern Learning for Sequential Recommendation

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Sequential recommendation (SR) aims to predict the next items for users by learning the users’ representations from their historical sequences. In this process, most existing methods rely on a single globally shared encoder to model the relationship between input sequences and target items, implicitly learning input–target patterns. However, such implicit learning treats patterns uniformly and largely overlooks their intrinsic characteristics and differences among training samples. Through the empirical studies in this paper, we find that input–target patterns exhibit both conflicting and generalizable characteristics, which impose distinct modeling requirements. Neglecting these properties leads to suboptimal user representations and limited generalization. Motivated by these findings, we propose a novel MoE architecture, Automated Selection-based Mixture-of-Experts (ASMoE), with a dual-stage training scheme to address these issues. In our ASMoE, we introduce an automated expert selection mechanism to adaptively allocate selectable experts and accommodate diverse modeling requirements of input–target patterns. Furthermore, we develop a dual-stage training scheme to enhance our ASMoE for input–target pattern learning. The first stage performs initial learning over diverse patterns. In the second stage, we explicitly construct the potentially generalizable input–target patterns via a category-aware mask generator and a similarity-aware penalty, thereby facilitating the fine-tuning of our ASMoE towards generalizable knowledge. Extensive experiments on four public datasets demonstrate the effectiveness of our ASMoE. The source code of our ASMoE is provided at https://github.com/xiaolLIN/ASMoE.