Security failure
Data Leakage
When an AI system exposes sensitive, confidential, regulated, or unauthorized information through outputs, retrieval, memory, or tool use.
What failed
Data leakage occurs when an AI system exposes sensitive, confidential, private, regulated, or unauthorized information. Leakage can happen through model output, retrieved context, tool responses, logs, memory, prompt injection, or cross-user state contamination.
Architecture context
Customer support assistants, internal knowledge copilots, email agents, HR assistants, sales tools, legal workflows, RAG systems, memory-enabled assistants, and agents with broad tool permissions.
Impact
Enterprise AI systems often interact with customer data, employee data, internal documents, credentials, business strategy, legal material, and regulated records. Data leakage can create legal, compliance, security, reputational, and customer-trust risk.
Symptoms
- The system reveals data the user should not access.
- It includes confidential source text in an output.
- It mixes information across users, tenants, or projects.
- It exposes hidden prompts, credentials, or system metadata.
- It sends sensitive information to an external tool or URL.
Detection signals
- Sensitive data patterns in outputs.
- Access-control mismatches between user and retrieved content.
- Prompt-injection attempts to extract data.
- Cross-tenant or cross-user context references.
- Tool calls containing sensitive payloads.
Mitigations
- Enforce permissions before retrieval and tool use.
- Redact sensitive data where appropriate.
- Limit tool access by role and workflow.
- Prevent cross-tenant memory contamination.
- Add prompt-injection defenses.
- Log and audit sensitive-data access.
- Escalate high-risk outputs for review.