Shape-invariant prompt learning with prompt relation structuring for open-vocabulary detection

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Open-vocabulary detection (OVD) aims to detect objects beyond training categories by leveraging the semantic knowledge of pretrained vision-language models. Recent studies indicate that modeling shape variation benefits OVD by making region features more adaptable to diverse object geometries. Although geometry-dependent adaptation has demonstrated the importance of shape-aware modeling, independently learning shape-specific adapters on different feature subsets may result in fragmented optimization and limited gradient sharing across shape groups, making it difficult to maintain consistent semantic alignment. Motivated by prompt-based adaptation and vision-language alignment, we propose a unified OVD framework named SIP-OVD. SIP-OVD introduces Shape-Invariant Prompt (SIP) learning, where shape-invariant prompts modulate CLIP region features according to object geometry, and a shared semantic adapter aligns the prompted embeddings with text embeddings in a shared semantic space. Furthermore, we introduce Prompt Relation Structuring (PRS), which regularizes prompt similarity to preserve smooth geometric transitions and discriminative shape boundaries. In addition, we incorporate Query-conditioned Iterative self-Training (QIT) as a complementary training module that mines reliable unseen-class regions to provide additional supervision for model refinement. Extensive experiments on COCO-OVD and LVIS-OVD validate the advantage of SIP-OVD for novel and rare category detection. Specifically, SIP-OVD achieves 47.5 AP 50 on novel categories and 40.1 mask AP on rare categories. Code will be available at https://github.com/messeyAmumu/SIP-OVD .