MITIGATION · m-human-dual-control
Human dual-control — four-eyes rule for irreversible high-impact approvals
An AI agent operating with broad authority can propose actions that are irreversible: deleting records, modifying IAM policies, moving funds. A single human reviewer at the approval gate is a single point of failure, one compromised account, one fatigued reviewer, or one successful social-engineering attempt is enough to commit the action. Human dual-control addresses that by requiring two distinct, independent humans to approve before the action commits.
At a glance
TL;DR
- Before an irreversible high-impact action commits, two distinct humans, neither of whom is the requester, must independently approve it.
- Identity disjointness is enforced structurally: the system rejects an approval from the proposer or from the first approver acting again.
- Each reviewer sees the same decision-summary card but reaches a verdict before seeing the other's decision, preventing the second reviewer from deferring to the first.
- Both approvals are cryptographically signed and written to the WORM-locked audit trail; the record cannot be suppressed by the actor whose action is under review.
How it behaves
What it is
Separation of duties is the principle that no single person should hold enough authority to unilaterally complete a high-consequence operation. In traditional financial controls this is the four-eyes rule; in security engineering it is the two-person rule. Human dual-control applies that principle to the HITL approval gate in an agentic AI system.
The mechanism is straightforward. Before an irreversible, high-impact action commits, production-config writes, IAM policy edits, money movement above threshold, account deletions, code-execution tool invocations at elevated privilege, the system requires independent approvals from two distinct humans, neither of whom is the original requester. Each reviewer sees the action summary (decision summary provides it), the evidence chain, the agent's stated confidence, and the gates the action has already passed. The system enforces identity disjointness: an approval from the proposer, or a second approval from the first approver, is rejected. Both approvals are cryptographically signed (Sigstore signing) and written to the audit trail. Critically, the second reviewer does not see the first reviewer's decision before submitting their own, which prevents sequential disclosure from pressuring agreement.
A single reviewer at this gate is vulnerable in three distinct ways: credential theft (one compromised account approves catastrophe), social engineering (one reviewer persuaded by contextually plausible content), and decision fatigue (one reviewer approving under volume pressure). Requiring a second independent reviewer addresses each failure mode separately. An attacker must compromise two accounts. Social engineering must succeed twice, against independently-informed reviewers who have not seen each other's reasoning. Fatigue must coincide across two distinct roles.
This is the human-in-the-loop analogue of multi-agent consensus, which requires a quorum of independent agents on the machine side, and the process complement to out-of-band verification, which uses a single human across two independent channels. Pair with m-risk-prioritized-queue for the routing tier that determines which action classes reach the dual-control gate, m-decision-summary so both reviewers see the same structured card, m-sigstore for signed approval records, and m-insider-threat-program for the personnel-side control on reviewer eligibility.
Detection signals
- Dual-approval latency. Consistently near-zero time between first and second approval indicates the second reviewer is rubber-stamping rather than exercising independent judgment.
- Approval-disagreement rate. A persistent rate of zero disagreements across many cases indicates the two reviewers are seeing identical framing or one is deferring to the other, not that every action was genuinely unambiguous.
- Dual-control bypass events. Any action that committed with fewer than two independent approvals is an audit-trail violation; halt and investigate each occurrence.
Threats it covers
-
WHY IT HELPS Compromised-operator escalation is the threat, an attacker who has taken over an admin account, or an insider acting unilaterally, attempts to approve a privilege-escalating action through the HITL gate. Requiring a second, independent human means a single compromised identity cannot unilaterally commit the action; the attacker must also control or deceive the second approver.
-
WHY IT HELPS Malicious code injection targets the code-review seam. An AI-generated or attacker-modified change of RCE-class scope (IAM policy edits, production secrets rotation, code-execution tool invocations) reaches a single reviewer whose fatigue or volume load causes it to pass unchallenged. A two-person approval requirement means review fatigue in one reviewer is not sufficient; the change must independently pass a second reviewer's scrutiny.
-
WHY IT HELPS Manipulation through AI-generated content is the threat. An attacker uses fluent, contextually plausible content to persuade a reviewer to approve a harmful action. A second independent reviewer, who reaches their verdict without seeing the first reviewer's decision, must be persuaded separately, social engineering that succeeds once must succeed twice against independently-informed reviewers.
Principle coverage
Defence-in-Depth stage: Prevent — and it advances:
- Separation of Duties Separation of duties requires that no single identity hold enough authority to complete a high-consequence operation alone. Human dual-control applies that principle at the HITL approval gate: the proposer cannot approve their own action, and the first approver cannot also be the second, so unilateral commitment of an irreversible action is structurally impossible.
- Human Oversight (HITL / HOTL) Human oversight is only as strong as the approval path protecting it. Dual-control ensures that a single human reviewer, who may be fatigued, compromised, or manipulated, cannot be the sole point of failure at the gate before an irreversible action commits.
- Reversibility / Dry-run / Hold periods Dual-control is a pre-commitment check that guards the irreversibility boundary: it introduces a structural pause before an action that cannot be undone, making the point of no return require two independent human decisions rather than one.
- The Lethal Trifecta The lethal trifecta arises when an agent has broad permissions, exfiltration capability, and access to sensitive data simultaneously. Dual-control at the high-privilege action gate limits the blast radius by requiring two independent approvals before the most dangerous operations in that trifecta can execute.
- Safety / Harm-limitation Safety requires that consequential agent actions not proceed on a single point of human judgment. Dual-control enforces a minimum of two independent human decisions before an irreversible high-impact action commits, reducing the probability that a single error, compromise, or manipulation results in harm.
Design & governance principles (open design, economy of mechanism, accountability, …) are architectural, not advanced by a single placed control.
Implementation options
Five verified patterns spanning cloud IAM, blockchain multisig, secrets infrastructure, platform code review, and self-build approval workflow. Most production deployments combine one infrastructure-layer option with GitHub branch protection as the code-path floor.
AWS IAM An IAM role trust policy includes the aws:MultiFactorAuthPresent condition; only principals who called sts:AssumeRole or sts:GetSessionToken with a valid MFA TOTP receive the temporary credentials that permit the sensitive action. A second distinct IAM role, held by a second distinct identity, is required to countersign the downstream resource change.
Why choose it: Best when sensitive operations live on the AWS API surface (IAM policy writes, S3 bucket policy changes, RDS instance deletion). Two separate IAM roles with non-overlapping principals enforce identity disjointness at the IAM layer, not in application code. The aws:MultiFactorAuthPresent condition key is the AWS-native mechanism: it is true only when the session was established via STS with a TOTP challenge, and it is absent on long-term access keys, which closes the long-lived-credential bypass. Pair with AWS CloudTrail org-trail logging (m-actor-recorder-split) so both role-assumption events are captured in an account the acting agents cannot modify.
More details:
Gnosis Safe A Gnosis Safe multisig wallet enforces an N-of-M signing threshold: a transaction is only executed after N distinct owners have submitted valid signatures. A Safe Guard contract (ITransactionGuard, available since Safe contracts v1.3.0) adds pre-execution and post-execution policy checks via checkTransaction, rejecting transactions that violate additional constraints even after the signing threshold is reached.
Why choose it: Best for blockchain-native deployments where the action is an on-chain transaction and the dual-control requirement must survive a compromised off-chain agent process. The signing threshold is enforced inside the EVM, a compromised agent that controls one signing key cannot unilaterally broadcast. A 2-of-3 or 2-of-5 threshold is the common institutional configuration. The Safe Guard layer adds programmable policy on top: checkTransaction can enforce counterparty allowlists, value bounds, and caller-identity checks beyond what the signature count alone expresses. Adds approximately 5–15k gas per transaction for the guard invocation.
More details:
HashiCorp Vault Vault operator init splits the root key into N shares using Shamir's Secret Sharing (default: -key-shares 5, -key-threshold 3). Each share is distributed to a distinct operator. Unsealing Vault requires threshold-many operators to present their shares sequentially; no single operator can unseal alone.
Why choose it: Best when Vault holds secrets that back privileged operations (PKI roots, database credentials, cloud access keys) and the dual-control requirement is at the secrets-infrastructure layer rather than the application layer. The Shamir split is enforced by the Vault server itself, there is no API to bypass it. For a strict 2-of-2 dual-operator requirement, configure -key-shares 2 -key-threshold 2 at init time. Note: automated unseal (auto-unseal via KMS) bypasses the human dual-control requirement entirely and must not be used when the dual-operator property is the security goal.
More details:
GitHub branch protection Enable "Require pull request reviews before merging" and set the required approver count to 2 or more. Enable "Require approval of the most recent reviewable push" so a new commit to an already-approved PR invalidates prior approvals and requires fresh review. Add a CODEOWNERS file to require that at least one approval comes from a designated owner of the affected code path.
Why choose it: Best as the structural gate for any agent-generated code change or infrastructure-as-code diff that merges to a shared branch. The protection is enforced at the repository host, not the coding tool, it holds regardless of how the code was generated or what local flags were set. CODEOWNERS integration means a security-relevant path (IAM policy files, Terraform modules) can require a specific reviewer group, not just any two users. Apply via branch ruleset (preferred over legacy branch protection) for organisation-wide policy that covers all branches and all contributor identities including bot accounts.
More details:
Self-build A purpose-built HITL workflow that creates a pending gate on action proposal, routes the gate to two distinct approval queues backed by disjoint signer pools, captures each signed approval (m-sigstore), enforces proposer ≠ approver1 ≠ approver2 at the identity layer, defers first-approval visibility from the second reviewer until after their submission, and only releases the action to execution when both approvals are recorded.
Why choose it: The only option that implements the full dual-control requirement as specified when the action surface is not AWS IAM, blockchain, or Vault, arbitrary agentic tool calls, internal API mutations, data-deletion pipelines. The signer pool must be genuinely disjoint: two users from the same team who are always paired by default are a disjoint pool in name only. Rotate pairings via queue-routing policy. Pair with m-risk-prioritized-queue to determine which action classes route to the dual-control tier. The main operational risk is TTL expiry: a gate that expires without a second approval must fail-closed, not auto-approve.
More details:
Trade-offs
- Latency is the dominant adoption cost and it is intentional: two-person approval adds reviewer-coordination time measured in hours during business hours and longer off-hours. Reserve the dual-control tier for the action classes that warrant it; over-application produces rubber-stamping within days as reviewers adapt to volume.
- AWS IAM and GitHub branch protection add no material latency to the action path itself, the friction is in the human coordination, not the infrastructure. Vault Shamir unseal is the exception: unsealing is a synchronous manual process that can take 30–60 minutes when key-holders are geographically distributed.
- Development effort is low for AWS IAM (policy condition key) and GitHub (branch ruleset configuration), medium for the self-build signer-pool workflow (queue routing, identity-disjointness enforcement, TTL expiry logic), and near-zero for Gnosis Safe (threshold configuration at wallet setup).
- The self-build pattern carries the highest operational surface: the queue routing logic, the identity-disjointness check, and the first-approval visibility deferral must each be correct for the control to hold. Test all three independently before enabling dual-control in production.
When NOT to use
- Do not route routine or high-frequency actions to the dual-control tier. A queue processing dozens of items per hour will produce rubber-stamping within days regardless of the technical enforcement.
- Do not apply to reversible operations where an undo path exists and the rollback cost is lower than the reviewer-coordination cost of dual-control.
- Do not use when speed is a safety requirement, incident response, time-critical fraud intervention, emergency circuit-breaker trips. The reviewer-coordination latency that makes dual-control effective makes it dangerous in time-critical contexts. Use a single on-call authority with mandatory post-action review instead.
Limitations
- Dual-control assumes the two approvers are genuinely independent. Two reviewers who report to the same manager, share the same shift, or are consistently paired by queue defaults are not independent in the failure-mode sense. Rotate pairings, mix seniority levels, and measure pairing-distribution entropy as a control-health signal.
- This control does not prevent coordinated insiders: two corrupt approvers can collude. Pair with m-insider-threat-program for vetting and monitoring, and with m-actor-recorder-split so both signed approvals are written to a tamper-evident audit trail that neither approver can modify.
- The Vault Shamir option degrades to a different failure mode under auto-unseal: if KMS-backed auto-unseal is enabled, no human dual-control occurs at unseal time. The manual Shamir process is the intended mechanism; auto-unseal trades that control for operational convenience.
- GitHub branch protection enforces dual-control on the merge event, not on the content of the change. A reviewer who approves without reading the diff satisfies the technical gate while defeating its purpose. Spot-audit approved AI-generated pull requests and track the disagree-with-summary rate as a calibration signal.
Maturity tier reasoning
- Tier 2 fits because separation of duties is a mature compliance discipline in financial services, government, and healthcare, and every technical primitive here, IAM condition keys, Gnosis Safe threshold signing, Vault Shamir seals, GitHub branch protection, is production-available and documented by its maintainer.
- What keeps this out of Tier 1 is the absence of an agentic-specific action-class taxonomy that specifies which tool calls, API mutations, and data-deletion operations warrant dual-control in a deployed AI agent system. Every deployment defines its own thresholds.
- The self-build signer-pool workflow has no off-the-shelf implementation and no industry-standard design specification for the identity-disjointness check or the first-approval visibility deferral; operational norms are still deployment-specific.
Last verified against upstream docs: 2026-05-30.