Privacy-Preserving Transformers for Time-Series Forecasting: A Survey and Two-Axis Taxonomy of Cryptographic, Differential-Privacy, and Feature-Path Methods

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Transformer architectures, originally developed for language modeling, have been increasingly applied to time-series forecasting, but deploying them on sensitive data such as patient vital signs, financial records, or energy use raises privacy risks at two points: inference (the server sees raw inputs and outputs) and training (the model can memorize individual records). This survey organizes privacy-preserving methods for Transformer forecasting along a two-axis taxonomy: protection stage (inference, training, and data-level) and protection locus/mechanism (cryptographic wrapper, gradient, input, feature, output, or data path). For inference, we review secure multi-party computation (CrypTen through MPCFormer, SecFormer, Iron, BOLT, PUMA, BumbleBee, and CipherPrune) and homomorphic encryption (CKKS, THE-X, hybrid HE-MPC, and automated FHE compilation via Orion), focusing on how each handles the Transformer non-linearities (softmax, GELU, and LayerNorm). For training, we organize differential-privacy mechanisms by where noise enters the gradient path (DP-SGD, DP-FedAvg, DP-FTRL, PATE, and adaptive clipping), the input path, and the more recent feature path and analyze the privacy, utility, and dimensionality trade-offs. We further survey privacy in large language and foundation models, federated Transformers, time-series privacy attacks (membership inference, gradient inversion, and attribute inference), differentially private synthetic data, and applications in energy, health care, and finance, closing with open problems and future directions. Throughout, we show that the surveyed mechanisms provide non-comparable privacy guarantees and should be chosen by threat model instead of headline utility.