Skip to content
ICM Holdings International
Search⌘ K
Index
Decisions under uncertainty

7 · Mechanisms and worked studies

A test changes beliefs through likelihoods

Follow sensitivity, specificity, prevalence and Bayes updating

137 pages · Edition 2026-10-10

Mechanisms and worked studies

An illustrative test evidence tree

Update from likelihoods, not signal labels

A signal is useful when its possible outcomes have different relationships to the state the institution needs to understand. The binary fixture supplies sensitivity as the probability of a positive signal given success and specificity as the probability of a negative signal given failure. These are conditional likelihoods, not the probability that success follows a positive signal. The latter depends on the prior as well. Confusing these directions can make an instrument look more persuasive than the supplied assumptions justify.

The engine first calculates the probability mass of each signal branch. A positive signal can arise from success with a positive response or from failure with a false positive. A negative signal can arise from success with a false negative or from failure with a negative response. The branch mass adds the relevant prior-weighted possibilities. It then calculates the posterior success probability within the branch by dividing the success contribution by the total branch mass. This is a transparent conditional update under the supplied binary state and likelihood assumptions.

The useful-signal case makes the distinction visible. The positive branch raises the success probability enough to select Commit, while the negative branch lowers it enough to select Hold. The update does not produce certainty in either branch because the instrument is imperfect. The report should retain both posterior probabilities and branch masses. A highly favourable posterior in a rare branch may contribute less to expected decision value than a more moderate posterior in a common branch. Conditional appeal and ex ante value are different readings.

The false-positive-burden case changes the prior and weakens the negative-state filter. A positive signal can still be more likely when success occurs than when failure occurs, but failure is sufficiently common in the supplied state mixture that positive signals do not necessarily justify commitment. In this fixture, both signal branches select Hold. The instrument updates beliefs without creating a preferred action change. This is an example of why sensitivity alone cannot determine a commercial research purchase. The initial incidence and the response thresholds matter.

Signal names should not supply an interpretation that the likelihoods contradict. Positive is an instrument label, not a promise of favourable evidence. In the inverse-signal study, a positive result reduces the success probability while a negative result increases it. The supplied likelihoods make the signal negatively associated with success. The engine uses that association correctly: the decision response is attached to the posterior, not the emotional connotation of the word positive. An institution must train users to read the instrument's actual conditional meaning.

This inverse case is a useful adversarial test because it exposes a hidden assumption in many presentations. A decision tree can look convincing while its branches have been labelled in a way that presupposes the desired conclusion. If the analyst follows the word positive rather than the likelihoods, the same instrument can lead to the wrong action. The paper deliberately preserves the counterintuitive fixture and its raw outputs. Its purpose is to test method interpretation, not to suggest that a poorly designed or negatively associated instrument should be purchased without validation.

The likelihood inputs require their own evidence provenance. An instrument supplier may report accuracy over a population whose state mix differs from the intended use, or may estimate performance against an imperfect reference. The app does not inspect that study design. Its supplied sensitivity and specificity are assumptions unless an actual validation record supports them for the relevant state and use conditions. A commissioning dossier should separate the instrument's development evidence from evidence of transfer to the current opportunity and from the probability estimate used before the signal.

The signal should also be independent of the action in the manner the model assumes. If committing changes the measurement conditions or the state, a single likelihood table may not describe every action. If observing the signal requires an operating trial that itself changes the opportunity, information acquisition and intervention are intertwined. The binary engine treats the supplied signal as a source of evidence preceding the selected action. It does not identify a causal effect of testing or model a measurement process that changes its own state.

An impossible branch needs explicit treatment. If its probability mass is absent under the supplied assumptions, there is no posterior distribution conditional on that event in this model. The engine reports the posterior as unavailable and gives the branch no expected-value contribution. A reviewer should not replace that unavailable value with an arbitrary success probability to make a table complete. If the allegedly impossible outcome actually occurs, the event is evidence that the assumed state or likelihood model needs review. It is not a routine branch to which the same model can assign a confident response.

The branch probabilities should sum to the full represented signal universe. In the binary fixture, positive and negative are exhaustive outcomes. A real instrument may produce missing, ambiguous, delayed or invalid results. If those outcomes have different action consequences, collapsing them into positive or negative can distort value. The institution should either justify the collapse or use a broader signal model. The simple app does not estimate the probability of an indeterminate result. Its illustrative likelihoods should not be used to hide an unresolved test-operating condition.

A posterior should remain associated with the specific fixture that produced it. Copying a favourable branch probability into another payoff matrix or opportunity without preserving the prior and likelihood context can create a misleading decision narrative. The study's downloadable inputs, outputs and bindings make that association concrete. A reader can reproduce the branch and see the supplied assumptions. If the context changes, the analyst should create a new fixture or revise the actual decision record with an explicit history, rather than reuse a posterior as a general property of the instrument.

The update should be checked in both directions. A team may inspect the favourable branch carefully because it supports the desired action while treating the adverse branch as a formality. Yet information value depends on the full conditional policy, including the action that avoids unnecessary exposure after an adverse result. A study that can only confirm a commitment and has no permissible adverse response may create less value than its presentation implies. The action charter and branch table should be reviewed together so that the institution's response is genuinely conditional.

A signal-label audit can distinguish a reporting error from an instrument error. In the inverse fictional case, the label positive is less associated with success than negative. Swapping display names while preserving the joint masses should swap the branch labels and leave the underlying conditional policy value unchanged. Swapping the likelihood values without the labels changes the instrument and requires a new comparison. If a provider's report uses favourable where the specification uses positive, reconcile the mapping before dispatching a response. The audit follows the conditional evidence object, not the emotional meaning of its name, and does not validate the likelihoods or their use population.

A result outside the binary specification needs a different disposition from an impossible model branch. A hypothetical inconclusive report can occur in practice even though the current instrument object has no inconclusive outcome. Do not force it into negative, renormalise the remaining results or fabricate a posterior. Retain the current fallback action while obtaining the interpretation or a model that represents the additional outcome. By contrast, a branch with zero supplied mass has an undefined posterior within the declared model; it contributes no weighted value but remains a diagnostic of an incompatible observed result if it actually occurs. These distinctions protect the response rule at the handoff where a neat tree meets an imperfect report.

Evidence and scope

FDA diagnostic guidance defines sensitivity and specificity conditionally and requires interpretation in the intended population and study context.

Evidence: Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests

A conditional positive branch

Positive branch mass = p × sensitivity + (one minus p) × (one minus specificity). Posterior success after a positive signal = p × sensitivity / positive branch mass, when that mass is positive. The negative branch uses the complementary conditional likelihoods. An absent branch has no defined posterior in this model.
Illustrative calculation. Assumptions and sensitivity are published with the model.

Illustrative useful-signal branch account

Probabilities are displayed as percentages; the frozen output retains fractions.

Scroll for all columns. With the table focused, use the left and right arrow keys.

Illustrative useful-signal branch account
SignalBranch massPosterior successChosen actionConditional payoff, USDkWeighted payoff, USDk
Positive43.00%79.07%Commit121.4052.20
Negative57.00%10.53%Hold0.000.00
Illustrative calculation. Assumptions and sensitivity are published with the model.

Illustrative inverse signal: names do not determine meaning

The supplied likelihoods reverse the intuitive favourable reading of a positive signal.

Scroll for all columns. With the table focused, use the left and right arrow keys.

Illustrative inverse signal: names do not determine meaning
SignalBranch massPosterior successChosen action
Positive50.00%16.00%Hold
Negative50.00%64.00%Commit

Validate the conditional interpretation before using the branch

Require the intended state, likelihood direction and validation context to be explicit. Inspect both possible branches and their feasible responses. Treat an unavailable posterior as a model boundary, and review unrepresented outcomes rather than inventing their probabilities. Read a signal from its supplied association with the state, not from the connotation of its label.

  1. Separate likelihood from posterior.
  2. Inspect prior and reference-population compatibility.
  3. Publish branch masses and conditional actions.
  4. Escalate unrepresented or impossible outcomes.

A simple signal can communicate more clearly than a detailed instrument model

A binary abstraction can be appropriate when its decision consequences and validation bridge are defensible. Simplicity becomes misleading when indeterminate outcomes, context dependence or action effects are omitted despite changing the response. Preserve the simple model only after recording why the omitted mechanisms do not control the decision, or use it as a labelled screen pending a richer instrument account.

The instrument likelihoods are supplied

The app does not estimate sensitivity or specificity from study data, correct reference-standard bias, model indeterminate results or detect dependence with prior evidence. FDA material is used only for narrow diagnostic reporting definitions and context; it does not validate this commercial illustration or establish medical advice. The branch computations remain conditional on the declared likelihood assumptions.

Evidence at the point of use

Open a citation in the chapter to inspect its source, stated support and limits.

Find the next connection

Open the full search desk