Engineering leadership ·

AI Agents, Engineering Autonomy, and Risk Management

At one of my clients, we were discussing how much autonomy to give AI coding agents. When should an agent be able to make a change on its own, and when should someone step in? The more we talked about it, the more it sounded like a question engineering teams have been answering for years: how do we delegate authority while staying accountable for the outcome?

Teams already make that decision for employees, contractors, and automated systems. AI agents bring different strengths and failure modes, but the same starting point applies. What could go wrong, how far could the effects spread, and how quickly could we detect and recover from a mistake?

Capability, authority, and accountability

A capable contributor can have narrow authority. An agent can work independently within those limits while a person or team remains accountable for the outcome.

Match autonomy to the risk

Set permissions according to what a change can affect. These examples are starting points. A test change that weakens a release gate, for instance, deserves more scrutiny than its file type suggests.

RiskExampleSafeguards
LowDocumentation, tests, isolated UI changesAutomated checks, with auto-merge where appropriate
ModerateFeatures, API changesAutomated checks and review appropriate to the change
HighAuthentication, payments, customer dataExplicit approval and a controlled rollout
CriticalSecrets, destructive operations, security boundariesRestricted access and independent authorization

Review those boundaries as the team learns which safeguards hold up in practice. Treat merging, deploying, and activating functionality as separate decisions. A team might allow a change to merge and deploy behind a feature flag while requiring explicit approval before customers see it.

Build safeguards into delivery

AI agents introduce specific risks. They can act on instructions embedded in untrusted content, behave unpredictably, and produce changes faster than people can review them. Scope their tool permissions, treat external content as untrusted, and require independent authorization for consequential actions.

Measure delivery and recovery together

Count the cost of failures and recovery when judging how fast a team can deliver. Small teams feel the cost of blanket approvals especially keenly. Limiting the reach of a mistake, detecting it quickly, and making recovery straightforward gives teams room to delegate more. Reserve human judgment for decisions where it changes the outcome.

Further reading