An End-To-End Multimodal VLM Framework for Automated PPE Annotation and Site-Aware SOP Compliance Monitoring

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Traditional computer vision systems for Health, Safety, and Environment (HSE) monitoring are constrained by labour-intensive manual labelling and limited contextual awareness. Conventional models identify isolated objects but lack an understanding of spatial layout, equipment states, and procedural logic. As a result, organizations that rely on these systems often find themselves managing a narrow slice of their actual safety exposure: a helmet is either present or absent, a glove is either detected or not, but the surrounding context, whether that equipment is being used correctly, in the right sequence, under the right conditions, remains invisible to the model. This paper presents a centrally hosted, end-to-end AI framework that automates the lifecycle of safety monitoring using a Multimodal Vision-Language Model (VLM). The framework utilizes a foundation VLM to autonomously generate structured training datasets, eliminating manual annotation bottlenecks that have historically slowed the deployment of computer-vision-based safety tools. These datasets facilitate both the training of lightweight convolutional networks and the site-specific fine-tuning of targeted VLMs, allowing organizations to choose the deployment profile that best matches their hardware and latency constraints. By embedding contextual intelligence and procedural reasoning derived from Standard Operating Procedures (SOPs) into the model's visual grounding, the framework establishes an adaptive paradigm for AI-driven safety monitoring, shifting the industry closer to its "Goal Zero" safety ambitions, the aspiration that no incident, however minor, should be considered an acceptable cost of operations.