Explainable fault diagnosis with large vision language models

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Fault diagnosis in complex manufacturing systems presents a critical challenge that requires not just feature extraction but also symbolic knowledge and expert reasoning for reliable, explainable diagnostics. This paper introduces FD-LVLM (Fault Diagnosis with Large Vision Language Model), a framework for intelligent mechanical fault diagnosis. We first propose a method for creating a Fault Diagnosis Semantic Dataset (FDSD), which embeds expert-level diagnostic knowledge into structured textual descriptions paired with visual images of sensor signals generated by the Continuous Wavelet Transform. This multi-source dataset is then used to fine-tune a Large Vision Language Model (LVLM) using our proposed Asymmetric QLoRA (A-QLoRA), a novel parameter-efficient strategy that selectively adapts the model's visual pathways while preserving its core linguistic reasoning. To use the model's fine-tuned knowledge and enhance its deductive power, we introduce a reasoning chain, a structured prompting framework applied during inference that guides the LVLM through a step-by-step, expert-aligned diagnostic process. This complete framework allows the model produce accurate fault classifications, fine-grained explanations, and actionable maintenance suggestions. Crucially, we validate the model's explainability by demonstrating that its diagnostic process is a context-aware, progressive sharpening of visual attention, where the model's focus dynamically shifts in response to the evolving textual diagnosis. This process is shown to be directly linked to the visual evidence for multiple fault types. Validated on two distinct datasets, FD-LVLM demonstrates significantly enhanced performance over deep learning benchmarks. This paper presents a robust, end-to-end methodology for developing diagnostic systems that are not only highly accurate but also inherently explainable.