Harvesting trade partners online with AI: A joint learning method for predicting the business role of massive web pages

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

In small and medium-sized enterprises (SMEs), users often perform tedious and complex human–computer interactions between search engine data ( S D ) and webpage data ( W D ) when seeking potential business partners online via search engines (SEs). While artificial intelligence (AI) has been shown to improve search efficiency, conventional machine learning methods still face challenges in retrieval accuracy and efficiency, particularly in complex big-data environments. To address these limitations, this study proposes an AI module termed the Multi-source Data Joint Learning (MDJL) model, designed to assist SME users in efficiently retrieving online business partners. The MDJL model employs a multimodal information fusion approach to separately represent unstructured data elements in S D and W D as vectors. An encoder-decoder architecture then jointly learns the associations between S D and W D . Based on these learned joint representations, MDJL incorporates a multi-label attention mechanism to train a classifier that accurately predicts the business roles of companies associated with webpages. The model’s effectiveness is validated using real business datasets annotated by domain experts. Experimental results show that MDJL can effectively learn joint information from any two cross-correlated datasets. The resulting classifier exhibits high efficiency and robustness, especially when data from certain sources are partially missing or unknown. From a managerial standpoint, a trained MDJL model can further help SMEs digitize and consolidate the business knowledge of highly skilled employees, supporting more informed and efficient decision-making in partner search activities.