Jihun Kim · ACM SIGSAC Conference on Computer and Communications Security (CCS) 2026 · 2026
DOI: 10.1145/3830454.3846405
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Cloud LLM services require users to transmit prompts in plaintext to the provider—an unacceptable structural risk for R&D, legal, medical, financial, and defense industries—while running an open-weight model on-premises removes the risk, running a frontier model (700B+ parameters) on-premises is prohibitively expensive. We demonstrate a deployed system that delivers on-premises-level confidentiality while running in the cloud, without trusted hardware on the inference path. Each transformer layer is split by linearity: attention and all non-linear operations run on a client-owned edge node, while the feed-forward (FFN) linear matrix multiplications run on the provider's server. Every activation crossing the boundary is masked by additive noise drawn from a secret auxiliary-vector pool held only by the edge; because the FFN is linear, the edge recovers the exact result. The edge reuses this pool across tokens—masking each activation with a fresh random subset-sum—and continuously refreshes it at near-zero cost, avoiding the exhaustion that one-time masking faces as its supply of precomputed (s, Ws) pairs is consumed. The server is fully stateless—no KV cache—so one server serves many edges, its memory independent of the user count. We have run the system end-to-end across two cloud GPU nodes on a 119B open-weight MoE model at batch sizes 1 to 120; the live demonstration serves it at interactive rates and shows, on the wire, that only masked activations cross the boundary.
No comments yet — start the discussion below.