UI2Case: A Multi-Agent Framework for Test Case Generation from UI Designs via Multimodal Large Language Models

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

High-fidelity user interface (UI) designs are widely used in many UI-centric development workflows to document layout structures, visual appearances, and intended interactions before implementation, thereby providing an important basis for early validation and downstream quality assurance when such artifacts are available. However, automatically deriving comprehensive, accurate, and structured UI test cases from static UI design artifacts remains challenging, as it demands robust multimodal understanding and reasoning over implicit interactions and multi-screen navigation without access to implementation-level signals. To address these challenges, we present UI2Case, a multi-agent framework that leverages multimodal large language models (MLLMs) to automatically generate test cases directly from UI designs. The framework decomposes the generation process into four agents: a Navigation Agent for inferring page transitions, a Layout Agent for parsing component hierarchies, an Interaction Agent for identifying user actions, and a Synthesis Agent for deterministically assembling structured test cases. Evaluation on six real-world UI designs from an industrial product line shows that UI2Case achieves an average F1-score of 97.0% while substantially reducing test case generation time compared with manual authoring. Additional experiments on the RICO-36 benchmark show that UI2Case outperforms single-model prompting and structured GUI testing baselines, remains portable across five multimodal backbones, and benefits from each task-specific agent. The framework has also been integrated into production testing, where manual test case authoring effort was reduced by over 70% in aggregate. These results demonstrate the effectiveness and practical potential of combining MLLMs with agent-based collaboration for UI test case generation.