AKDF: Adaptive knowledge distillation for secure, efficient, copyright-protected model publishing

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

The rise of large language models (LLMs) has transformed natural language processing, powering applications from creative writing to code generation. However, their vast size and proprietary nature present two major challenges, including efficient deployment on limited hardware and secure protection of intellectual property. This work introduces the Adaptive Knowledge Distillation Framework (AKDF), a unified training approach that simultaneously com-presses LLMs and embeds ownership signals for copyright assurance. AKDF employs parameter-efficient Low-Rank Adaptation (LoRA) to distill a 1-billion-parameter student model from an 8-billion-parameter teacher while integrating an output-level watermarking module directly into the distillation process. This design reduces trainable parameters and encodes verifiable ownership signatures without altering the frozen base weights. Experiments on ARC-Easy, PIQA, and WMT16 show that the student retains approximately 76% of the teacher’s performance while maintaining competitive reasoning and translation quality under substantial compression. AKDF also enables secure and communication-efficient model publishing, providing a practical path to-ward bandwidth-efficient and ownership-aware deployment of large-scale language models.