Skip to content
ICM Holdings International
Search⌘ K
Index
Decisions under uncertainty

3 · Decision architecture

A probability has a provenance

Distinguish frequency, elicited belief, forecast and illustrative assumption

137 pages · Edition 2026-10-10

Decision architecture

A probability evidence worksheet

A percentage needs an event and an evidence history

A probability input is a statement about an event, a population, a time horizon and an information set. Without those elements, a number can travel between decisions while changing its meaning. The prior probability in the illustrative engine refers to success under the stated binary opportunity definition before the proposed signal is observed. It is not a forecast accuracy score, a preference weight, an instrument sensitivity or the confidence an expert feels about a report. The input register should preserve this distinction so that a familiar percentage does not substitute for a specified event.

Success must be defined independently of the action labels. If Commit and Stage create different conditions for success, a common state may be inappropriate. The simple decision table assumes the state is shared across actions and only the payoffs differ. An actual intervention that changes the likelihood of success requires action-dependent state probabilities or a causal model. Supplying a larger success payoff for an active action does not make the engine account for its effect on the event probability. Reviewers should ask whether the proposed state is a fact about the opportunity or a result of the chosen intervention.

The probability's horizon should align with the payoffs. A chance of reaching an early technical milestone is not automatically the chance of delivering a lifetime commercial return. A long-duration programme may pass commissioning and fail later through operating, demand or regulatory mechanisms. A binary aggregate can be useful for a bounded illustration, but it compresses those mechanisms into a deliberately simplified state. If a committee needs to distinguish them, the event model must expand. The source of a prior for an early milestone cannot silently certify a broader success state.

The register should state how the number was produced. It may be a frequency from a defined sample, a model output, a structured expert judgement, a transferred estimate or an explicit scenario assumption. These origins have different review obligations. A frequency needs its inclusion rule and denominator. A model output needs its inputs, calibration and limitations. An expert judgement needs its question, elicitation conditions and range. A scenario assumption needs its purpose and sensitivity. Calling every origin data prevents the committee from understanding which uncertainty has been measured and which has been supplied for analysis.

The frozen useful-signal study supplies its prior as an illustrative assumption. It is chosen to make the conditional decision mechanism visible. The paper does not infer that prior from a collection of client projects, claim that it is representative of an industry or estimate an actual commercial success rate. The purpose is methodological: a reader can change the input and inspect the resulting actions. The distinction matters because a perfectly reproducible calculation can only be as empirically applicable as the event definition and evidence supporting its inputs.

An estimate transferred from another context requires a bridge. A study of a mature operating system might not inform a first deployment; a likelihood measured with skilled operators might not apply to a different implementation team. A comparable label is insufficient. The bridge should explain the relevant differences in population, conditions, incentives and measurement. When those differences cannot be resolved, the author should report a range or retain the input as an assumption. The model can show whether the resulting uncertainty changes the decision; it cannot validate the transfer by arithmetic.

Probability provenance should not be confused with precision. Reporting more decimal places does not add evidence. A broad but honestly elicited range may be more decision-useful than a narrow point estimate with no traceable origin. The institution should retain enough precision to reproduce the calculation while displaying only the precision justified for interpretation. In an illustrative calculation, exact fixture values serve reproducibility, whereas the number of displayed decimals is a presentation convention. An actual evidence register would need a reason for the reported precision and an account of uncertainty in the estimate itself.

The prior also depends on what is already known. If the team has observed a precursor that is correlated with the proposed test, using an unconditional historical frequency as the current prior may count evidence incorrectly. A current prior should represent the institution's information before the new signal. The app does not reconstruct that information set or detect dependence with an earlier test. The author must specify it. This is particularly important when a signal supplier reports an impressive historical accuracy that was evaluated after filtering opportunities using information already known to the client.

An expert's confidence in a probability is a different question from the probability itself. A panel can agree that an opportunity has a moderately favourable chance while having little evidence about that chance. It can also disagree despite substantial evidence because members use different event definitions. The register should separate the event estimate from the evidence assessment and disagreement record. This enables a committee to ask whether a point estimate is stable enough for the decision without treating a confidence adjective as a numerical likelihood. The paper uses no invented conversion between those concepts.

Measurement uncertainty introduces another distinction. An instrument may estimate a physical quantity with a stated uncertainty, while the decision requires the probability that a performance condition is satisfied. Turning the former into the latter requires a model of the quantity and the threshold, plus relevant uncertainty about measurement and context. The simple binary signal fixture supplies its conditional likelihoods directly. It does not derive them from a metrological uncertainty budget. A report that uses physical measurement precision to justify a commercial likelihood must make that bridge explicit and reviewable.

The register should identify the evidence that would change the prior before commissioning the proposed signal. This guards against a number becoming a fixed institutional belief. If new information contradicts its basis, the team should update the current decision and reassess the research purchase. A previously useful test may become redundant after another credible result arrives. Conversely, a new concern may increase the value of resolving the state. The charter is therefore a dated record, not a permanent certificate of how much a study is worth.

Sensitivity should concentrate on the decision boundary rather than only the input's plausible range. A committee may tolerate substantial uncertainty when the preferred action remains stable across that range. It may need better evidence when a small change selects a different action with consequential payoffs. The app's prior sweep gives a deterministic view of the best available action at sampled probabilities, and its pairwise intersections provide additional checks. These outputs support a decision sensitivity discussion. They do not supply the probability distribution of the prior or the evidential plausibility of each point.

Illustrative calculation. Assumptions and sensitivity are published with the model.

Completed illustrative probability provenance register

The record deliberately exposes the absence of empirical calibration.

Scroll for all columns. With the table focused, use the left and right arrow keys.

Completed illustrative probability provenance register
FieldUseful-signal fixture entryReview obligation
EventDefined opportunity successChallenge whether binary states are sufficient
Prior40.00 percentScenario assumption only
PopulationNo empirical universe declaredDo not infer an industry frequency
Information setBefore the supplied binary signalDo not double count existing evidence
HorizonSame aggregate horizon as payoff matrixNo early milestone to lifetime transfer
OwnerCommissioning analyst and action committeeApprove interpretation and sensitivity

The input owner should sign the probability's interpretation, not merely the numerical field. A record stating that the prior refers to the defined success state, the named opportunity and the pre-signal information set is more useful than a signature beside a percentage. The reviewer should then distinguish disputed interpretation from disputed magnitude. If participants disagree about the event, averaging their numbers can produce a precisely formatted estimate of no shared proposition. Resolving the definition should precede numerical aggregation.

There is also a legitimate unresolved state. An institution may lack a defensible probability model for a novel threat, an unprecedented interface or an ill-defined outcome. Entering a convenient middle value does not solve this problem. The committee can use scenario comparisons, robust constraints or a bounded exploratory commission while documenting the inability to support a probability. If it uses an illustrative prior to understand a mechanism, it should preserve that label. A decision-support app should help expose the missing evidence rather than pressure the user to disguise it as a forecast.

Evidence and scope

IPCC guidance separates confidence based on evidence and agreement from quantified likelihood in climate assessments.

Evidence: Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties

Evidence and scope

NIST distinguishes statistical and other evaluation methods within a measurement uncertainty account.

Evidence: Essentials of expressing measurement uncertainty: basic definitions

Accept a probability with its meaning

Require an event definition, horizon, information set, provenance and named owner before accepting a probability as an empirical input. Keep a transferred estimate provisional until its bridge is reviewed. When only a scenario assumption is defensible, label it and evaluate how the decision changes across alternatives. Reject invented precision and conversions from confidence adjectives into unsupported likelihoods.

  1. State the event and horizon.
  2. Record the pre-signal information set.
  3. Identify frequency, model, judgement or assumption provenance.
  4. Challenge transfer and boundary sensitivity.

A point assumption is useful early in a project

A team can learn from a provisional point input before collecting sufficient evidence. The important condition is that the provisional status travels with every interpretation. Use the point to expose the action mechanism, pair it with explicit alternative fixtures and record the evidence needed to replace it. A provisional number should not acquire empirical authority simply because later reports reproduce it accurately.

The engine does not infer input uncertainty

The app treats the supplied prior and likelihoods as fixed numbers. It does not estimate their confidence intervals, elicit expert distributions, infer action-dependent probabilities or validate a transferred measurement model. The probability register and qualitative uncertainty controls identify the analysis that would be required; they are not additional computations hidden behind the binary engine.

Evidence at the point of use

Open a citation in the chapter to inspect its source, stated support and limits.

Find the next connection

Open the full search desk