AI-based semantic search improves e-commerce product search more consistently when augmenting keyword matching than when replacing it

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Retailers increasingly assume that AI-based semantic search improves e-commerce product discovery, but its value may depend on both query characteristics and the evaluation dimension considered. This study evaluates five retrieval approaches—BM25, TF-IDF, dense MiniLM, a lexical–semantic hybrid, and a cross-encoder reranker—on WANDS and a held-out United States ESCI test subset using controlled candidate-set reranking and offline algorithmic UX proxies. Hybrid retrieval and cross-encoder reranking significantly improve nDCG@10 over BM25 on both benchmarks, whereas pure dense retrieval has no significant overall nDCG@10 gain on either benchmark after Holm correction and underperforms BM25 for long and attribute-heavy query-type proxies on WANDS. On WANDS, neural models also narrow category and embedding diversity. A robustness check with a bi-encoder five times larger reproduces the same WANDS pattern, so these results are not artefacts of encoder size. The aggregate prediction that semantic models increase drift or irrelevant exposure is disproved: the evaluated neural models reduce the study’s dataset-specific risk proxies. Thus, AI search is conditionally beneficial, but its relevance gains should be assessed alongside diversity and domain-transfer limitations.