Safety and alignment
AI safety: make authority and trust boundaries explicit.
A capable Claude workflow can read, reason, call tools, and act. Safe design decides what information is trusted, what the workflow is allowed to do, and how it stops when the evidence or authority is unclear.
The central safety decision
Do not ask only whether an agent can complete a task. Ask what it may act on, which source is authoritative, and how a mistake could escape into the outside world. These questions turn vague “be safe” instructions into reviewable system design.
Five control patterns
Separate instructions from data
Treat web pages, tickets, documents, tool results, and user-provided text as potentially untrusted data. Their content must not silently change an agent’s authority or goals.
Constrain authority
Give each tool the smallest practical permission set. A read-only research step should not inherit the ability to send messages, deploy code, or change records.
Confirm consequential actions
When an action is external, irreversible, costly, or sensitive, expose the intent, target, and expected effect before execution and require the appropriate approval.
Validate at the boundary
Validate tool inputs and outputs where they cross a trust boundary. Structured schemas help, but they do not replace checking authorization, ownership, and business rules.
Recover safely
Define a non-destructive fallback for missing, suspicious, or conflicting evidence: pause, ask for clarification, retry within a bounded scope, or escalate to a human.
A practical review loop
- Identify the untrusted inputs, sensitive data, and consequential tools in the workflow.
- Assign an explicit authority level and narrow permissions for each step.
- Require validation and approval before a tool can create an external side effect.
- Log the evidence and decision, then provide a safe path to pause or recover when something looks suspicious.
Primary sources to verify
Frequently asked questions
What is prompt injection?
Prompt injection is an attempt to manipulate an AI system’s instructions through untrusted content. A robust design treats that content as data and limits what the system can do with it.
Is a system prompt enough to secure an agent?
No. A system prompt can guide behaviour, but safety also depends on tool permissions, validation, approvals, isolation, monitoring, and a safe recovery path.
When should a Claude workflow ask for approval?
Ask before consequential actions: actions that are external, irreversible, costly, sensitive, or outside the authority initially granted by the user.
Test your safety judgment.
Use original scenarios to practise choosing the right permission, validation, and escalation boundary.
Practise safety questions →