Sections in this lesson

Sensitive Data & Exfiltration

Identify secrets, personal records and restricted documents before a model call.

Reviewed: · Content owner: Interview Preparator

What you will learn

  • Minimize what enters context
  • Enforce destinations outside the model
  • Test observable data paths

Before you start

The flow

  1. 1.Identify scope
  2. 2.Inspect evidence
  3. 3.Apply host control
  4. 4.Run negative cases
  5. 5.Compare benign behavior
  6. 6.Review remaining risks

Minimize what enters context

Identify secrets, personal records and restricted documents before a model call. A system prompt is not a secret vault: material included in context may be reproduced or transmitted through another capability. Redaction after generation cannot undo disclosure to a provider.

Choose the minimum authorized fields and document the provider data path and retention contract. Logs, traces, URLs and tool arguments are all possible disclosure destinations, not only the final answer.

Enforce destinations outside the model

allowedUrl accepts exact approved HTTPS hosts and rejects credentials, alternate schemes and non-default ports. A domain suffix match would mistakenly accept an attacker-controlled lookalike. URL parsing alone is not complete SSRF protection.

A production adapter must validate redirects, resolved addresses, proxy behavior and network egress. The example performs no fetch at all; rejected URL fixtures do not demonstrate an OS-level network sandbox.

Test observable data paths

Use synthetic canaries, not real credentials, when testing outputs and audit records. Prefer fixed event codes and necessary identifiers to logging full prompts. Redaction rules need tests for transformed and structured secrets.

If exposure occurs, stop affected capabilities, revoke credentials/grants and record a sanitized incident. Merely removing a phrase from the UI or changing the system prompt is not containment.

Try it

policy.mjs
javascript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import { createHash, timingSafeEqual } from 'node:crypto';
export function payloadDigest(call) {
  // Canonical ordered fields; extra or ambiguous fields cannot bypass payload binding.
  if (!call || Object.keys(call).sort().join(',') !== 'action,resource' || !['readPlan','completePlan'].includes(call.action)
    || !/^PLAN-[0-9]+$/.test(call.resource)) throw new Error('Invalid call');
  return createHash('sha256').update(JSON.stringify([call.action,call.resource])).digest('hex');
}
export function authorize(scope, call, resource, grant, now, epoch = 1) {
  const digest = payloadDigest(call);
  if (!Number.isFinite(now) || !Number.isSafeInteger(epoch)) return 'DENY';
  if (!scope || !resource || resource.id !== call.resource || resource.tenant !== scope.tenant || !resource.readers.includes(scope.user)) return 'DENY';
  if (call.action === 'readPlan') return 'ALLOW';
  // Grant comes from a trusted server-side approval store, never from model output.
  if (!grant || !Number.isFinite(grant.expiresAt) || grant.revoked || grant.epoch !== epoch || grant.expiresAt <= now || grant.tenant !== scope.tenant
    || grant.user !== scope.user || grant.runId !== scope.runId || grant.digest !== digest) return 'APPROVAL_REQUIRED';
  return 'ALLOW';
}
export function escapedText(value) {
  if (typeof value !== 'string' || value.length > 4000) throw new Error('Invalid output text');
  return value.replaceAll('&','&amp;').replaceAll('<','&lt;').replaceAll('>','&gt;').replaceAll('"','&quot;').replaceAll("'",'&#39;');
}
export function allowedUrl(value, hosts) {
  try { const url = new URL(value); return url.protocol === 'https:' && !url.username && !url.password && url.port === '' && hosts.includes(url.hostname); }
  catch { return false; }
}
export function trustedToken(claims, policy, now) {
  // Claims must already be cryptographically verified by an auth adapter.
  return claims?.issuer === policy.issuer && claims?.audience === policy.audience && claims?.expiresAt > now
    && claims?.tenant === policy.tenant && Array.isArray(claims?.scopes) && claims.scopes.includes(policy.scope);
}
export function approvedArtifact(bytes, expectedSha256) {
  const actual = createHash('sha256').update(bytes).digest();
  if (!/^[a-f0-9]{64}$/.test(expectedSha256 ?? '')) return false;
  return timingSafeEqual(actual,Buffer.from(expectedSha256,'hex'));
}

Run npm run examples:security and npm run eval:security. This source is a deterministic teaching fixture; it does not prove live model safety, NLI, production auth or OS/network isolation.

Interview questions

Is a system prompt a secret vault?

Choose the minimum authorized fields and document the provider data path and retention contract. Logs, traces, URLs and tool arguments are all possible disclosure destinations, not only the final answer.

Does a hostname allowlist fully prevent SSRF?

A production adapter must validate redirects, resolved addresses, proxy behavior and network egress. The example performs no fetch at all; rejected URL fixtures do not demonstrate an OS-level network sandbox.

What is containment after a leak?

If exposure occurs, stop affected capabilities, revoke credentials/grants and record a sanitized incident. Merely removing a phrase from the UI or changing the system prompt is not containment.

Practice

Extend one negative scenario and its authorized positive counterexample for Sensitive Data & Exfiltration.

Check your solution

  • State the trusted input and effect boundary.
  • Assert the expected status and observed effect.
  • Report coverage and limitations without claiming production readiness.

Official references

Continue learning

Category sequence

AI Security

Lesson 3 / 14

Was this lesson useful?

Change notes: 2026-10-31: security/reliability curriculum and source-bound fixture.