Skip to content
ICM Holdings International
Search⌘ K
Index
Decisions under uncertainty

9 · Mechanisms and worked studies

The decision boundary matters more than average confidence

Locate conditional action intersections across the prior and use selected payoff or likelihood contrasts to identify further evidence questions.

137 pages · Edition 2026-10-10

Mechanisms and worked studies

A sensitivity and falsifier map

Find the action switch that evidence can affect

A decision boundary is a point at which a change in the information available to the institution changes its preferred feasible action. The boundary depends on the payoff matrix and objective. It is not a universal probability at which a project becomes acceptable. In the binary expected-value model, each action's expected payoff changes linearly with the prior. Comparing those lines shows where the optimiser can move from one action to another. This gives a concrete reason to ask for better information: the current evidence may leave the decision near a consequential switch.

The app reports a sampled prior policy and the intersections of each pair of action value lines. These are related but distinct outputs. A pairwise intersection tells the reader where those two actions have equal expected payoff. It does not guarantee that either is best among the full menu at that point. Another action can have greater value. The report therefore retains the app's best-of-all-actions reading at each intersection. Treating every pairwise crossing as an actual policy switch would overstate the information a simple threshold table supplies.

In the useful-signal matrix, the Commit and Hold lines cross at a supplied computed prior where Stage is better than both. This is an especially useful check. A diagram showing only Commit and Hold would suggest a commitment boundary that is not the boundary of the complete decision. The robust middle action changes the policy architecture. The committee should review the complete menu before using an attractive threshold to justify a full commitment, and the analyst should show which comparisons are inactive in the global optimum.

The sampled prior sweep is a visual guide, not an exact list of every transition. Its points are supplied by the canonical engine at fixed prior increments. A policy switch can occur between them. The pairwise intersections offer more precise candidate checks under the linear payoff structure, while the best-action field distinguishes an actual upper-envelope comparison from an inactive crossing. For a richer nonlinear or constrained model, a sampled sweep would need additional analysis. The paper does not claim that its app solves every possible decision boundary.

Information matters when it can move a posterior into a different action region. A signal that narrows uncertainty but leaves both branches in the same region has no algebraic action benefit in this fixed model. A less precise signal can be useful if it moves a meaningful branch across a boundary and supports a different response. This does not imply that instrument quality is irrelevant. It means quality should be judged against the decision architecture and the consequence of each branch, not against a generic target for forecast accuracy divorced from the available actions.

The useful and no-policy-change studies share the same payoff matrix and likelihoods but use different priors. In one, the conditional posteriors select different actions; in the other, both remain in the Commit region. The contrast demonstrates that an instrument does not possess a fixed commercial information value independent of the client's current evidence. The same instrument can support a valuable commission at one point and an economically redundant commission at another. The commissioning record should therefore be dated and should preserve the current prior rather than cite an earlier project's value.

Illustrative calculation. Assumptions and sensitivity are published with the model.

The response menu changes the current policy account

The plotted values are the canonical sampled best expected payoffs at alternative priors. They compare the original, restricted and reduced-upside menus before buying a signal. Connecting segments aid reading between sampled points; they do not publish an EVSI sweep or a probability distribution over priors.

Original deterministic illustration from frozen current-policy output. Every point binds to its named fixture and actual output pointer. The acquisition cost and signal likelihoods do not enter this current-policy sweep.

Original deterministic illustration from frozen current-policy output. Every point binds to its named fixture and actual output pointer. The acquisition cost and signal likelihoods do not enter this current-policy sweep.

The false-positive-burden case also shows why the favourable branch should be evaluated against the actual boundary. A positive signal can materially increase the success probability and still leave Hold preferable. Describing it as a strong improvement in belief can be correct while describing it as a reason to commit can be wrong. The committee should ask what action becomes feasible and preferable at the posterior, not only whether the posterior is larger than the prior. This avoids confusing a direction of update with a sufficient decision condition.

Boundary proximity does not automatically mean more research should be purchased. The signal's cost, attainable performance, timing and available response all still matter. A decision near a crossing may involve small monetary differences or an unresolved nonmonetary constraint. A costly study can be disproportionate even if it changes the selected action. Conversely, a meaningful downside can justify improving evidence when a small probability change has large consequences. The signal-value calculation integrates the supplied payoffs and branch masses; the boundary analysis explains where its action improvements originate.

The committee should distinguish uncertain parameters from sensitivity ranges. A sweep across priors shows what the decision would do if different values were supplied. It does not establish that every prior in the sweep is equally plausible or that the current prior has a uniform distribution. A separate evidence account should describe the plausible range and its basis. If the input owner cannot support that range, the sweep remains a mechanism test. Its value is to identify where additional evidence could matter, not to manufacture an empirical probability distribution from a convenient horizontal axis.

The same applies to a payoff sensitivity. Changing a failure exposure can move the boundary without changing the instrument. A legal allocation, an operating safeguard or a smaller commitment may therefore reduce the need for research by making the action more robust. This is an institutional choice rather than an evidence forecast. A team should compare feasible ways to improve the decision: buy better information, alter the action, reduce a consequence or accept an explicitly bounded exposure. The model can compare supplied revised matrices, but the feasibility and cost of those changes require separate evidence.

An intersection should not be treated as a sharp empirical fact when the input cells are rough. The algebra can produce an exact point from approximate values. Reporting that point with many decimals may obscure the uncertainty in the boundary itself. The paper displays the canonical outputs at a declared precision for reproducibility, and the tables identify their illustrative status. An actual decision dossier should preserve the computational point while explaining the evidence range and practical conditions around it. Precision of calculation is different from precision of the institution's knowledge.

The response rule should also handle a result close to the boundary. The app provides a deterministic tie-break for reproducible output. A committee may instead request an admissibility check, choose a more reversible action or require a defined margin before a consequential commitment. Those conditions should be agreed before receiving the signal where feasible. Otherwise a sponsor can selectively invoke the boundary when it favours commitment and dismiss it when it favours holding. A predeclared rule makes that inconsistency reviewable.

Policy charts should show what is plotted. The engine's prior sweep displays the best expected payoff, which is the upper envelope of the supplied actions at each sampled prior. It does not display signal value at each prior or the chance that a threshold will be crossed. Those would require additional computations. A report that titles the existing sweep information value versus prior would misstate the output. The graphical exhibit should retain its axes, units, action scope and selected canonical pointers so that a reader can distinguish a policy picture from an acquisition-value picture.

A boundary enquiry should test the action envelope, not merely a pairwise intersection. If the crossing actions are both below Stage, moving a posterior across their intersection does not change the selected policy. The useful enquiry is whether the evidence can reach a region where a different admissible action becomes best. An uncertain payoff can move that region as well as an uncertain prior, so retain the reason for each sensitivity. A hypothetical team targeting the inactive Commit and Hold crossing would be buying precision at a point that does not govern the current menu. The existing envelope and branch account expose that error without assigning a probability to parameter uncertainty or pretending that the sampled sweep computes acquisition value.

Evidence and scope

The Magenta Book connects evaluation planning with decision needs and the use of findings.

Evidence: Magenta Book: Central Government guidance on evaluation

Illustrative calculation. Assumptions and sensitivity are published with the model.

Illustrative pairwise intersections and the complete-menu winner

The table preserves the canonical distinction between pairwise equality and the globally preferred action.

Scroll for all columns. With the table focused, use the left and right arrow keys.

Illustrative pairwise intersections and the complete-menu winner
Action AAction BPrior at equality, percentBest among all actions
CommitStage43.59Commit
CommitHold35.71Stage
StageHold17.65Hold

Research around an active boundary

Check the full action menu before interpreting a crossing. Compare possible posteriors with the complete-menu action regions, retain the evidence basis for a plausible prior range, and explain whether the proposed study can move a branch into a materially different response. Separate a sampled policy plot from a signal-value calculation. Agree treatment of near-boundary findings before the result arrives.

  1. Identify active and inactive pairwise comparisons.
  2. Map the possible posteriors to feasible action regions.
  3. Challenge uncertainty in the payoff boundary.
  4. Record any required decision margin.

A robust choice may still deserve monitoring

A current action that remains best across the supported range can reduce the case for another immediate study. It does not remove the need to watch for a changed opportunity, new constraint or evidence outside that range. Define a monitoring trigger tied to a material change in the decision perimeter. This preserves robustness without treating the current policy as permanently immune to new information.

The policy sweep is sampled and does not assign plausibility

The engine reports a finite prior sweep and pairwise linear intersections. It does not compute an empirical distribution of priors, uncertainty intervals for thresholds or a signal-value curve across the sweep. Its best-action reading must be interpreted with the complete supplied menu. Broader boundary claims require a model that actually calculates them.

Evidence at the point of use

Open a citation in the chapter to inspect its source, stated support and limits.

Find the next connection

Open the full search desk