Risk-based audit sampling under PCAOB AS 2315 directs the engagement team to allocate substantive procedures toward higher-risk accounts. The continuous version of that decision — how heavily, in what sequence, with what update rule as new evidence arrives — is a sequential decision problem that the standard sampling-guide formulations leave to engagement judgment. Markov Decision Process formalism makes the structure explicit: the engagement state at any point is the current evidence position on each in-scope account or account-group; the available actions per period are test_now, defer, and rely_on_controls; the transition dynamics describe how each action updates the evidence position; the reward function is asymmetric, with the cost of a Type II error (failing to detect a material misstatement) materially exceeding the cost of a Type I error (over-testing a clean account).
The MDP framing produces an optimal policy as a transparent function of the engagement risk profile and the cost-asymmetry the partner is willing to accept. Unlike intuitive sampling allocations, every input — transition probabilities, residual misstatement probabilities, the Type-II-to-Type-I cost ratio — is documented and auditable. The policy is therefore defensible against PCAOB inspector second-guessing under AS 1215 (audit documentation) in a way that “the partner used judgment” is not.
This framing aligns with PCAOB AS 2315 (Audit Sampling), AS 2810 (Evaluating Audit Results), AS 2110 (Identifying and Assessing Risks of Material Misstatement), and AS 1215 (Audit Documentation). The international counterpart for sampling guidance is ISA 530.
The MDP formulation
For risk-based sampling decisions that recur across multiple engagements (continuous audit programs, internal-audit testing rotations, recurring portfolio reviews), the stationary infinite-horizon discounted MDP is the natural formulation. The tuple $(\mathcal{S}, \mathcal{A}, P, R, \gamma)$ has:
- $\mathcal{S}$ — finite set of states (engagement-evidence states for one in-scope account or factored account-group)
- $\mathcal{A}$ — finite set of actions per state (
test_now,defer,rely_on_controls) - $P(s’ \mid s, a)$ — transition probability from state $s$ to state $s’$ under action $a$
- $R(s, a)$ — expected immediate reward (negative for cost) of taking action $a$ in state $s$
- $\gamma \in [0, 1)$ — discount factor that determines the relative weight on future vs. immediate cost
Two terms anchor what follows; each is defined in plain English before the symbol appears.
- Bellman optimality equation — the rule that the value of being in state $s$ today equals the best one-step reward plus the discounted value of wherever you might end up tomorrow. In audit terms: the optimal long-run cost of starting at an in-scope account in evidence state $s$ equals the best choice across
test_now,defer, andrely_on_controlsof (immediate cost of that action + discounted expected cost of the resulting next state). Bellman (1957) is the original; every modern MDP textbook works from this recursion. - Contraction mapping — a function that, when applied repeatedly to two starting points, brings them closer together with each application. The Bellman operator (the right-hand side of the equation below, viewed as a function of $V$) is a contraction because the discount factor $\gamma < 1$ shrinks the difference between successive $V$ estimates by a factor of $\gamma$ at every iteration. Practitioner interpretation: no matter where you start your initial guess for $V^*$, the value-iteration loop is guaranteed to converge to the same answer, and it does so quickly — typical engagement-scale MDPs solve in tens of iterations.
The Bellman optimality equation under this formulation:
$$V^{*}(s) = \max_{a \in \mathcal{A}} \left[ R(s, a) + \gamma \sum_{s’ \in \mathcal{S}} P(s’ \mid s, a) V^{*}(s’) \right]$$
Banach fixed-point is the formal theorem from real analysis that any contraction on a complete space has a unique fixed point, and iterating the contraction from any starting point converges to that fixed point. Applied here (Puterman, 1994, §6.2): the Bellman operator has a unique fixed point $V^{*}$ for any $\gamma < 1$, and value iteration converges to $V^{*}$ at geometric rate $\gamma$ regardless of initialization. The optimal stationary policy $\pi^*(s)$ at state $s$ is the action achieving the maximum. The audit-context translation: the math guarantees the engagement team can rerun the optimization from arbitrary starting estimates and arrive at the same recommended policy — no hidden sensitivity to initial guesses, which matters for PCAOB inspection defensibility.
A genuine finite-horizon, fixed-budget engagement formulation requires augmenting the state space with remaining-budget and time-to-go dimensions and solving via backward induction; the implementation cost is meaningfully higher and the resulting policy is non-stationary. Most production audit-sampling applications are well-approximated by the stationary discounted formulation above, with $\gamma$ calibrated to reflect the engagement’s effective time-horizon. A common practitioner heuristic relates the discount factor to the effective horizon $h$ via:
$$\gamma = 1 – \frac{1}{h}$$
This relationship follows from setting the present-value weight of the terminal period to approximately $1/e$: $(1 – 1/h)^h \to e^{-1}$ as $h \to \infty$, so that an engagement with effective horizon $h = 20$ periods (e.g., 20 audit weeks, 20 months of internal-audit rotation) implies $\gamma \approx 0.95$. This is a calibration convenience; for engagements where the precise horizon is known and the budget constraint binds tightly, the finite-horizon backward-induction formulation (Puterman, 1994, Ch. 4) is the more appropriate framework. That formulation solves the recursion
$$V_t(s, b) = \max_{a \in \mathcal{A}} \left[ R(s, a) + \sum_{s’} P(s’ \mid s, a) V_{t+1}(s’, b – \text{cost}(a)) \right]$$
backward from the terminal period $T$, explicitly tracking remaining budget $b$ and time-to-go $t$. The article below uses the stationary discounted formulation throughout.
The asymmetric loss function
The cost-asymmetry between Type I and Type II errors is the partner-level judgment that anchors the entire framework. PCAOB AS 2810 §2-3 require the auditor to evaluate whether the audit evidence supports the opinion; failing to detect a material misstatement (Type II) carries professional, regulatory, and reputational cost orders of magnitude greater than the cost of additional substantive procedures on a clean account (Type I).
Quantitatively, denote:
- $c_{I}$ = cost of an additional unit of substantive procedure (analyst hours)
- $c_{II}$ = expected cost of failing to detect a material misstatement (regulatory inspection adverse finding, restatement penalty, reputational damage)
The asymmetry $\lambda = c_{II} / c_{I}$ is typically $10^2$ to $10^4$ in financial-statement audit contexts. The MDP reward structure incorporates $\lambda$ directly:
$$R(s, \text{test\_now}) = -c_I, \qquad R(s, \text{defer}) = -\eta \cdot p_{II}(s) \cdot c_I, \qquad R(s, \text{rely\_on\_controls}) = -\lambda \cdot p_{II}(s) \cdot c_I$$
where $p_{II}(s)$ is the conditional probability of an undetected material misstatement given current evidence state $s$ and $\eta$ is a one-period exposure carry parameter for postponing work while residual risk remains live. In the worked example below, $\eta = 2$ is intentionally much smaller than $\lambda = 1{,}000$: deferral is penalized for carrying exposure forward, but much less severely than an outright rely-on-controls decision. Two modeling choices are encoded here and should be made explicit.
First, the rely_on_controls reward charges the expected Type II cost as an immediate scalar — equivalent to assuming the cost is realized at the time of the decision rather than accumulated over future periods. This is the conservative formulation: it forces the present-value calculation to weigh the full expected cost against the immediate cost of testing, rather than discounting the misstatement-detection failure into the future. A continuing-risk formulation is also possible, but it requires additional modeling choices about how residual misstatement exposure decays through time and how that exposure enters the state transition kernel. That is a different MDP specification, not a notational restatement of the immediate-charge model. The immediate-charge formulation used in this article is mathematically simpler and aligns with partner intuition: the decision to rely on controls without substantive testing carries an upfront expected penalty proportional to the residual misstatement probability.
Second, the test_now action is modeled as incurring only the known cost of substantive procedures; partial detection (i.e., the possibility that testing reduces but does not eliminate Type II risk) is captured by the transition matrix’s probability of moving to a substantively-tested absorbing state, not by a residual reward term. In other words, the test_now action incurs a known cost; the rely_on_controls action incurs a large expected cost weighted by the asymmetry parameter and the residual misstatement probability; the defer action incurs a smaller one-period exposure carry cost that keeps postponement from dominating simply because it avoids immediate testing hours.
Value iteration and worked example: per-account stationary MDP under cost-asymmetric rewards
The example below applies the stationary infinite-horizon discounted MDP to a single in-scope account classified into three engagement-evidence states (high_risk_untested, moderate_risk_partially_tested, low_risk_substantively_tested). The optimal policy answers: given the current evidence state on this account, which action minimizes total discounted expected cost over the engagement program’s effective horizon?
For multi-account engagements, the same per-account MDP is solved independently for each account-group (factored state-space approximation; the failure-modes section below addresses the cross-account-dependency assumption). The cost-asymmetry parameter is set to $\lambda = 1{,}000$ (Type II cost is 1,000× Type I cost).
The code block below is fully self-contained and executable top-to-bottom. It includes all imports, defines the value-iteration function, constructs transition and reward matrices with seeded randomness (transition probabilities are sampled from Dirichlet distributions centered on stylized prior-period audit outcomes on a synthetic restatement-like panel, ensuring np.random.seed(42) materially affects reproducibility), runs value iteration to convergence, recomputes the Q matrix from the converged V* to derive the optimal policy pedantically, and prints the concrete optimal policy array and state-value function. A consolidated runnable version for both snippets appears in the companion artifact companion_artifacts/006_markov_decision_audit_sampling.py.
from __future__ import annotations # for tuple[np.ndarray, np.ndarray] in Python <3.10
import warnings
import numpy as np
np.random.seed(42)
def value_iteration(
states: list,
actions: list,
transitions: np.ndarray,
rewards: np.ndarray,
gamma: float = 0.95,
theta: float = 1e-6,
max_iter: int = 1000
) -> tuple[np.ndarray, np.ndarray]:
"""Solve a finite-state MDP via value iteration.
Parameters:
states — list of state labels (length n_states)
actions — list of action labels (length n_actions)
transitions — array of shape (n_states, n_actions, n_states); P(s' | s, a)
rewards — array of shape (n_states, n_actions); R(s, a)
gamma — discount factor
theta — convergence threshold on max state-value change
max_iter — maximum iterations
Returns:
V — optimal state-value function, shape (n_states,)
policy — optimal action index per state, shape (n_states,)
"""
n_states, n_actions = len(states), len(actions)
assert transitions.shape == (n_states, n_actions, n_states)
assert rewards.shape == (n_states, n_actions)
assert np.allclose(transitions.sum(axis=2), 1.0), "Transitions not row-stochastic"
V = np.zeros(n_states)
converged_at = None
for iteration in range(max_iter):
V_prev = V.copy()
# Bellman update for all (s, a) pairs simultaneously
# Q[s, a] = R(s, a) + gamma * sum_{s'} P(s' | s, a) * V_prev[s']
Q = rewards + gamma * np.tensordot(transitions, V_prev, axes=([2], [0]))
V = Q.max(axis=1)
if np.abs(V - V_prev).max() < theta:
converged_at = iteration + 1
break
if converged_at is None:
warnings.warn(
f"value_iteration: max_iter={max_iter} reached without converging to "
f"theta={theta}; max state-value change at termination = "
f"{np.abs(V - V_prev).max():.6e}. Returned policy may be sub-optimal.",
RuntimeWarning,
)
# Recompute Q from final V* before extracting policy (pedantically correct)
Q_final = rewards + gamma * np.tensordot(transitions, V, axes=([2], [0]))
policy = Q_final.argmax(axis=1)
return V, policy
# ─────────────────────────────────────────────────────────────────────────────
# State and action space definition
# ─────────────────────────────────────────────────────────────────────────────
states = [
"high_risk_untested",
"moderate_risk_partially_tested",
"low_risk_substantively_tested"
]
actions = ["test_now", "defer", "rely_on_controls"]
n_states, n_actions = len(states), len(actions)
# ─────────────────────────────────────────────────────────────────────────────
# Transition matrix with seeded Dirichlet sampling around deterministic centers
# ─────────────────────────────────────────────────────────────────────────────
# Base deterministic transition probabilities (centers for Dirichlet)
base_transitions = np.zeros((n_states, n_actions, n_states))
# State 0: high_risk_untested
base_transitions[0, 0] = [0.10, 0.40, 0.50] # test_now
base_transitions[0, 1] = [0.95, 0.05, 0.00] # defer
base_transitions[0, 2] = [1.00, 0.00, 0.00] # rely_on_controls
# State 1: moderate_risk_partially_tested
base_transitions[1, 0] = [0.05, 0.20, 0.75] # test_now
base_transitions[1, 1] = [0.00, 0.95, 0.05] # defer
base_transitions[1, 2] = [0.00, 0.50, 0.50] # rely_on_controls
# State 2: low_risk_substantively_tested (absorbing)
base_transitions[2, 0] = [0.00, 0.00, 1.00]
base_transitions[2, 1] = [0.00, 0.00, 1.00]
base_transitions[2, 2] = [0.00, 0.00, 1.00]
# Sample from Dirichlet with concentration α = 100 * base probabilities
# (high concentration keeps draws close to deterministic centers)
transitions = np.zeros_like(base_transitions)
for s in range(n_states):
for a in range(n_actions):
alpha = 100.0 * base_transitions[s, a] + 0.01 # +0.01 to avoid zero α
transitions[s, a] = np.random.dirichlet(alpha)
assert np.allclose(transitions.sum(axis=2), 1.0), "Transitions not row-stochastic"
# ─────────────────────────────────────────────────────────────────────────────
# Reward function with asymmetric Type I / Type II cost
# ─────────────────────────────────────────────────────────────────────────────
LAMBDA = 1000.0 # Type II / Type I cost ratio
ETA = 2.0 # one-period exposure carry penalty for deferral
c_I = 1.0 # cost of one substantive-procedure unit (analyst-hour units)
# p_II(s): residual undetected-misstatement probability per state
p_II = {0: 0.30, 1: 0.10, 2: 0.01}
rewards = np.zeros((n_states, n_actions))
for s in range(n_states):
rewards[s, 0] = -c_I # test_now: known cost
rewards[s, 1] = -ETA * p_II[s] * c_I # defer: one-period exposure carry cost
rewards[s, 2] = -LAMBDA * p_II[s] * c_I # rely_on_controls: expected Type II cost
# ─────────────────────────────────────────────────────────────────────────────
# Solve MDP via value iteration
# ─────────────────────────────────────────────────────────────────────────────
V_star, policy = value_iteration(states, actions, transitions, rewards, gamma=0.95)
print("════════════════════════════════════════════════════════════════════════")
print(" MDP-Optimal Policy for Risk-Based Audit Sampling")
print("════════════════════════════════════════════════════════════════════════")
print("\nOptimal state-value function V*:")
for s, label in enumerate(states):
print(f" {label:40s}: V* = {V_star[s]:8.2f}")
print("\nOptimal policy π*(s):")
for s, label in enumerate(states):
print(f" {label:40s}: action = {actions[policy[s]]}")
print(f"\nOptimal policy array (action indices): {policy}")
print(f" [test_now=0, defer=1, rely_on_controls=2]\n")
# ─────────────────────────────────────────────────────────────────────────────
# Compare against naive always-test-now baseline
# ─────────────────────────────────────────────────────────────────────────────
def policy_value(policy_idx: np.ndarray, transitions: np.ndarray,
rewards: np.ndarray, gamma: float, n_iter: int = 1000,
theta: float = 1e-6) -> np.ndarray:
"""Evaluate a fixed policy via iterative policy evaluation.
Mirrors the convergence-warning discipline of value_iteration: emits a
RuntimeWarning if n_iter is reached without max state-value change falling
below theta.
"""
n = len(policy_idx)
V = np.zeros(n)
converged_at = None
for it in range(n_iter):
V_new = np.zeros(n)
for s in range(n):
a = policy_idx[s]
V_new[s] = rewards[s, a] + gamma * np.dot(transitions[s, a], V)
if np.abs(V_new - V).max() < theta:
V = V_new
converged_at = it + 1
break
V = V_new
if converged_at is None:
warnings.warn(
f"policy_value: n_iter={n_iter} reached without converging to "
f"theta={theta}. Returned V may be biased.",
RuntimeWarning,
)
return V
naive_policy = np.array([0, 0, 0]) # always test_now
naive_V = policy_value(naive_policy, transitions, rewards, gamma=0.95)
print("Value gap: MDP-optimal vs. naive 'always test_now' baseline:")
for s, label in enumerate(states):
gap = V_star[s] - naive_V[s]
print(f" {label:40s}: optimal={V_star[s]:8.2f}, naive={naive_V[s]:8.2f}, gap={gap:+8.2f}")
print("════════════════════════════════════════════════════════════════════════\n")Output interpretation. With seed=42 and the Dirichlet-sampled transitions above, the optimal policy array is [0, 0, 1], meaning:
high_risk_untested→test_nowmoderate_risk_partially_tested→test_nowlow_risk_substantively_tested→defer
The MDP-derived policy diverges from the naive “always test_now” baseline on the low_risk_substantively_tested state because additional testing incurs the known cost $c_I$ but yields negligible risk reduction (the residual misstatement probability $p_{II} = 0.01$ is already near-zero). The value gap on that state is positive (the optimal policy saves cost), confirming that defer is the rational choice. The total value gap scales materially in real engagements with dozens of accounts and longer horizons.
Bridge to AS 2201: control-reliance consequences of the optimal policy
PCAOB AS 2201 §B7-B9 governs the auditor’s decision to rely on operating-effectiveness conclusions for an in-scope ICFR control. Reliance directly limits the substantive testing required at the assertion level; absence of reliance expands it. The MDP optimal policy formalizes this dependency: a defer or rely_on_controls action at the assertion-evidence MDP corresponds to the auditor concluding the relevant control is operating effectively and reducing substantive scope accordingly; a test_now action corresponds to expanded substantive procedures because the available control-operating-effectiveness evidence is insufficient or unavailable.
The value function $V^{*}(s)$ therefore quantifies, in dollar terms, the testing-vs-reliance trade-off the engagement partner makes under AS 2201. A conservative cost-asymmetry parameter $\lambda$ shifts the optimal policy toward more substantive testing relative to control reliance; an aggressive $\lambda$ shifts it toward more control reliance. The MDP framework makes that choice explicit and auditable in the workpapers rather than leaving it as a tacit partner judgment.
Sensitivity to the cost-asymmetry parameter
The optimal policy depends jointly on $\lambda$, the transition kernel, and the one-period deferral penalty $\eta$. In some seeded toy configurations, the policy is stable across wide ranges of $\lambda$; in others, the moderate-risk state flips as reliance becomes more or less expensive. For partner-level engagement planning, the recommended practice is to compute the optimal policy at three $\lambda$ values (conservative, standard, aggressive) and present the comparison. The code block below intentionally reuses the helper functions and seeded transition synthesis already defined above so the sensitivity logic is easier to audit; the companion artifact contains a single-file standalone implementation.
# Assumes states, actions, transitions, and value_iteration from the prior block
c_I = 1.0
ETA = 2.0
p_II = {0: 0.30, 1: 0.10, 2: 0.01}
# Sensitivity sweep over λ using the same seeded transition realization
lambda_grid = [100.0, 1000.0, 10000.0]
policies_by_lambda = {}
for lam in lambda_grid:
rewards_lam = np.zeros((len(states), len(actions)))
for s in range(len(states)):
rewards_lam[s, 0] = -c_I
rewards_lam[s, 1] = -ETA * p_II[s] * c_I
rewards_lam[s, 2] = -lam * p_II[s] * c_I
_, pol = value_iteration(states, actions, transitions, rewards_lam, gamma=0.95)
policies_by_lambda[lam] = pol
print("════════════════════════════════════════════════════════════════════════")
print(" Policy Sensitivity to Cost-Asymmetry Parameter λ")
print("════════════════════════════════════════════════════════════════════════\n")
for s, label in enumerate(states):
print(f"{label}:")
for lam in lambda_grid:
print(f" λ = {lam:>7.0f} → {actions[policies_by_lambda[lam][s]]}")
print()
print("════════════════════════════════════════════════════════════════════════\n")If the policy is stable across the three $\lambda$ values, the engagement team can report it with high confidence — the conclusion is robust to reasonable cost-asymmetry assumptions. If the policy is highly sensitive to $\lambda$, the engagement team must escalate the asymmetry calibration to the partner explicitly; the policy choice becomes a partner-level judgment rather than an analytical output.
Operational integration
The MDP framework integrates into engagement planning in three places.
Initial planning. Before substantive testing begins, the engagement team estimates initial state distribution across in-scope accounts, parameters $p_{II}(s)$ from prior-period evidence and AS 2110 §28-49 risk-assessment procedures, and the partner-elicited $\lambda$. The initial optimal policy provides the starting allocation of test hours.
Mid-engagement reallocation. As substantive procedures yield evidence, the engagement-evidence state of each account updates. The MDP solution can be re-run with the updated state distribution, producing an updated optimal policy. This formalizes what experienced partners do intuitively — reallocate hours toward accounts where unexpected findings have shifted the risk picture.
Post-engagement documentation. PCAOB AS 2315 §31 requires documentation of the sampling approach. The MDP framework produces a documentable audit trail: state space, action set, transition assumptions, reward parameters (especially $\lambda$), and the optimal policy at each engagement decision point. This is materially more defensible than an undocumented intuitive sampling allocation.
Reference points in the published prosecution record
The asymmetric cost structure this article formalizes — Type II vastly more expensive than Type I — is grounded in published prosecutions of audit failures where risk-based testing went wrong. Three reference points illustrate the pattern:
- Madoff auditor — David Friehling. United States v. Friehling, U.S. District Court for the Southern District of New York (filed March 2009; guilty plea November 2009 to securities fraud, investment-adviser fraud, falsifying audit reports, and obstructing the IRS; sentenced May 2015 to time served plus probation). Friehling signed annual audit opinions on Bernard L. Madoff Investment Securities for 17 years without performing meaningful substantive testing on the firm’s purported $50+ billion in trading activity. The MDP framework in this article — applied with credible $p_{II}(s)$ estimates and a partner-level $\lambda$ — would have flagged the impossibility of issuing an unmodified opinion under the implied substantive-procedure budget. The Friehling matter is the canonical published example of what happens when risk-based sampling collapses to no sampling at all.
- Arthur Andersen — Enron audit. Arthur Andersen LLP v. United States, 544 U.S. 696 (2005) (Supreme Court unanimously vacated the firm’s obstruction-of-justice conviction on jury-instruction grounds in 2005; the firm had already surrendered its CPA licenses by then, ending the franchise). The underlying audit failures — repeated risk-rated “high” accounts that received no expansion of substantive procedures despite years of warning signs — are well-documented in subsequent PCAOB enforcement materials and SEC restatement studies. The Enron-Andersen episode is the canonical case study in why partner-level cost-asymmetry calibration matters: a too-low $\lambda$ produces policies that systematically under-test high-risk accounts.
- KPMG — Xerox audit. SEC v. KPMG LLP, et al., Civil Action No. 03-CV-0671 (S.D.N.Y., January 2003; settlement April 2005 for $22 million in fines, profit disgorgement, and admissions). KPMG’s audits of Xerox accelerated $3 billion in equipment-lease revenue without challenging the timing classifications, an audit-sampling failure where the firm’s substantive procedures did not concentrate on the highest-revenue-recognition-risk accounts the optimal policy under this article’s framework would have selected.
Forensic accountants and FBI special agents who investigated these matters — many of them through the FBI’s Public Corruption / Securities Fraud / Forensic Accountant career tracks — confronted the after-the-fact evidence of policies the auditors had implicitly chosen. The MDP formalism turns those implicit policies into documentable, partner-elicited choices that survive PCAOB inspection.
Failure modes and defenses
Four patterns recur in MDP deployment to audit settings.
Stationarity vs finite-horizon mismatch. The stationary discounted formulation assumes the engagement program runs indefinitely with constant transition dynamics; the discount factor $\gamma$ approximates the effective horizon. For genuinely fixed-budget single engagements where the partner needs the budget-binding constraint reflected explicitly, the finite-horizon backward-induction formulation is more appropriate (state-augment with remaining-budget; solve $V_t(s, b)$ recursively from terminal period; non-stationary policy). The stationary formulation in this article is the right tool for continuous audit programs and recurring portfolio reviews; it is an approximation for single-engagement budgeted decisions.
State-space explosion. Real engagements have hundreds of in-scope accounts. Modeling each as a separate dimension produces a state space of size $|\mathcal{S}|^{n_{\text{accounts}}}$, which is intractable. Defense: factor the state space by independent account groups (typically by audit cycle — order-to-cash, procure-to-pay, record-to-report) and solve each group independently as in the worked example above. This factoring loses the cross-account dependencies but keeps the framework computationally tractable; the dependencies that matter most (cross-cycle accruals, intercompany transactions) can be captured via a small number of inter-group state coordinates.
Transition-probability misspecification. The framework assumes the engagement team can specify $P(s’ \mid s, a)$ accurately. In practice, these probabilities are themselves estimated from prior-period data and carry estimation error. Defense: report sensitivity of the optimal policy to perturbed transition probabilities; if the policy is robust, proceed; if not, escalate the model uncertainty to the partner. Robust MDP formulations (Iyengar, 2005) handle uncertainty in $P$ explicitly.
Cost-asymmetry parameter elicitation. The $\lambda$ parameter is genuinely subjective. Defense: rather than asking the partner to specify $\lambda$ directly, present the policy under three benchmark $\lambda$ values and ask the partner to identify which policy aligns with their judgment. The implied $\lambda$ becomes the engagement parameter.
Bridge to Stochastic Volatility Models for Restatement-Timing Anomalies
The MDP formulation above assumes the transition kernel $P(s’ \mid s, a)$ is known with certainty. When the underlying engagement-risk dynamics are themselves stochastic (e.g., the restatement-timing volatility documented across public-company panels), the framework extends to robust MDP formulations or partially-observable MDPs. Stochastic Volatility Models for Restatement-Timing Anomalies takes the stochastic-volatility piece head-on via GARCH-style models on restatement-timing series.
Authority:
MDP and dynamic-programming theory:
- Bellman, R. (1957). Dynamic Programming. Princeton University Press.
- Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, §6.2 (contraction mapping for stationary discounted MDP), Ch. 4 (finite-horizon backward induction).
- Sutton, R.S., & Barto, A.G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press, Ch. 4 (dynamic programming).
- Iyengar, G.N. (2005). “Robust Dynamic Programming.” Mathematics of Operations Research, 30(2), 257-280. (Robust MDP formulation under transition-probability uncertainty.)
Audit standards:
- PCAOB AS 2315 — Audit Sampling, §31 (documentation requirement).
- PCAOB AS 2810 — Evaluating Audit Results, §2-3.
- PCAOB AS 2110 — Identifying and Assessing Risks of Material Misstatement, §28-49 (risk-assessment procedures and fraud-risk linkage used to parameterize initial account states).