Abstract
System prompts play a critical role in guiding the behavior of large language models (LLMs), yet they remain highly vulnerable to prompt leakage attacks that attempt to extract hidden instructions. Existing defense methods, such as defensive prompting and output filtering, typically rely on direct access to system prompts, which is impractical in many real-world scenarios, particularly in third-party LLM services and emerging prompt marketplaces where prompts are proprietary and inaccessible. To address this limitation, we propose Surrogate-based Filtering with Leak Guards (SurF-Guard), a defense framework that protects system prompts without requiring prompt visibility. The framework leverages a pool of surrogate prompts to simulate interactions with user queries and detect potential prompt leakage, while two lightweight leak guards—canary tokens and instruction identifiers—provide additional signals for identifying explicit prompt disclosure. Unlike existing defenses that assume prompt access, SurF-Guard enables effective protection in settings where system prompts remain hidden. Extensive experiments on both open-source LLMs (e.g., Vicuna, LLaMA, and GPT-series models) and real-world platforms such as Poe show that SurF-Guard substantially reduces prompt injection success rates. We further analyze the trade-off between defense strength and response quality using a response-following metric, showing that SurF-Guard achieves strong attack resistance while maintaining high-quality responses for benign inputs.