Model Inconsistency Leading to Variable Approvals

The decision that won’t sit still

A language model making an approval decision is not a calculator. Ask a calculator “is 49.50 under the 50 limit?” and it answers the same way every time. Ask a language model to judge whether an expense claim is “within policy and adequately justified,” and, unless you have explicitly pinned it down, you can get approved one minute and rejected the next, with nothing about the claim having changed. For a chatbot, that variability is the feature that makes it sound human. For an agent that approves expenses, grants access, or releases money, the same variability is a security flaw: the system enforces policy differently each time it looks, and nobody notices it contradicting itself.

That is T48. The cause is not an attacker tampering with the agent’s memory (that is Memory Poisoning) and not a cleverly worded prompt. It is the model’s own non-determinism, the randomness built into how it generates text. To exploit it, an attacker changes nothing. They just ask again.

It helps to understand why identical inputs produce different outputs. At each step, a language model produces a probability distribution over possible next tokens and then samples from it. A setting called temperature controls how much randomness is in that sampling: above zero, the model will sometimes choose a less-likely continuation. That is what gives it fluency, and what makes its judgements unrepeatable. A claim sitting right on a policy boundary is exactly where the model is most uncertain, so its internal distribution is split: perhaps 60% toward “approve,” 40% toward “reject.” Re-running that decision is not re-reading the facts; it is re-rolling a weighted die.

How the failure plays out

The OWASP MAS Guide’s worked example is an RPA agent that processes expense reimbursements. Two identical claims (same receipts, same line items, same submitter) pass through it at different moments. Because inference runs at a non-zero temperature, the first is approved and the second is flagged for manual review. Crucially, no human ever sees the contradiction: each claim is handled independently, each decision is treated as final, and there is no step whose job is to notice that the pipeline just judged the same facts two different ways. A single wrong decision is forgivable. Humans make those too. The real defect is that the system has no mechanism to catch itself being inconsistent.

Why a fairness bug becomes an exploit

Once an approval is effectively a weighted coin-flip, anyone who can resubmit gets free re-rolls. That is what turns an internal quality problem into an attackable control bypass. Suppose a borderline claim is approved roughly 40% of the time. An attacker who simply submits it again and again is approved within a couple of attempts on average, and every rejected attempt costs them nothing. They never forge a receipt or alter a figure; they resample the model until it says yes. The control isn’t being broken so much as worn down by repetition, which is why ordinary fraud checks, all of which look at the content of a single claim, never fire.

Why autonomy makes it worse

In a human process, inconsistency is partly self-correcting: a reviewer who approved a claim on Monday and meets the same claim on Tuesday raises an eyebrow. Automation removes that safety net. The agent has no memory that it judged these facts before, and no instinct that “I already saw this.” Worse, in a multi-agent pipeline the randomness compounds. If a subordinate extraction agent variably reads the claim and a policy agent then variably judges it, the two sources of noise multiply rather than cancel, widening the band of claims whose outcome is effectively random. T26 is the same root cause in a higher-stakes setting, where the unstable decision drives an irreversible on-chain transaction. There is no “flag for review” once the funds have moved.

How you would notice

Nothing about any individual claim here is fraudulent, so the signal lives in repetition and variance, not in content. The practical approach is to fingerprint each claim at ingestion (a SHA-256 hash of its line items and merchant identifiers) and then watch the decisions attached to that fingerprint:

• The same claim hash recurring above a threshold within a rolling 24-hour window. This is the footprint of someone re-rolling the dice.
• A single claim hash that has been both approved and rejected inside one 7-day window. This decision-variance alert directly catches the model contradicting itself.
• Low model-confidence approvals: if the pipeline exposes a confidence score, an approval issued below a confidence floor should be routed to a human rather than actioned.
• A per-user approval rate climbing above its 30-day baseline without a matching rise in claims submitted. This is the statistical shape of a resubmission strategy.

The unifying idea: log a canonical input hash and the decision for every call, so “the same input produced two different answers” becomes a query you can alert on rather than an invisible event.

How to actually fix it

The fixes fall straight out of the cause, and they layer. This is Defence-in-Depth applied to a probabilistic component:

1. Make the decision deterministic. Set temperature to zero (or near-zero) on any model call that drives an approval, and reserve stochastic settings for tasks where variety is actually wanted. This removes the re-roll at the source: the same input now yields the same output, so resubmission gains the attacker nothing. It is the highest-value change and it costs nothing.
2. Never let the model be its own final judge. Even pinned to temperature zero a model can be wrong, and a probabilistic one should certainly not be the last gate. Put a rule-based pre-filter in front of it (reject clearly out-of-policy claims before the model ever sees them) and a validation step behind it (re-check the model’s “approve” against hard policy limits before any money moves). The deterministic layers, not the model, hold the line.
3. Remove the attacker’s free re-rolls. Rate-limit resubmissions and act on the detection signals above: a user who submits the same claim hash N times in a window triggers a fraud review rather than another inference. Even if some non-determinism survives, the attacker can no longer cheaply sample it.
4. Make it observable. Log every decision with its input hash and confidence score. Without this, none of the detection above is even possible.

Read together: temperature-zero removes the vulnerability, the deterministic guardrails catch whatever residue remains, rate-limiting strips the attacker’s leverage, and logging makes the whole loop visible.

Where it sits in the catalogue

T48 is the single-agent, decision-surface case of model instability. It extends T5 (Cascading Hallucination Attacks): T5 is about unstable outputs propagating between agents, whereas T48 is about one agent’s unstable output enforcing policy inconsistently. T26 is the same instability where the decision is an irreversible blockchain action.