Mechanisms and worked studies
A value-of-sample-information comparison
Separate usefulness from the purchase decision
The expected value of a proposed signal is the improvement in the expected payoff of the conditional policy it enables. The engine chooses a best action after each possible signal, weights that action's conditional payoff by the branch probability and adds the contributions. It then subtracts the best current expected payoff. This gross value describes how much the signal improves this decision before acquisition cost. The net information value subtracts the supplied study cost. Both quantities should be reported because they answer different commissioning questions.
The useful-signal fixture has a conditional policy that differs from the current Stage choice. Commit is selected after the positive signal and Hold after the negative signal. The gross improvement arises from making a more favourable response in each branch under the stated posterior. This is not simply a reward for a more accurate forecast. The forecast matters through the action consequences. A signal with substantial discriminatory power can have little decision value if both resulting posteriors remain within the same action region.
The no-policy-change fixture demonstrates that mechanism. Its prior is already favourable enough that Commit remains best after both possible signals. The instrument changes the success probability, but the action does not change. The weighted conditional payoffs recover the current policy's expected payoff exactly in the current rational evaluation. Its information value is 0.00 USDk, with exact zero retained in the exported fraction record. This result follows from the unchanged policy rather than display rounding or a cancellation tolerance. The study cost remains positive in the illustrative comparison, so the purchase has negative net model value. A decision owner should still review whether omitted states or an unrepresented action could change this conclusion.
The false-positive-burden fixture reaches the same policy conclusion from a different direction. Both branches select Hold because neither posterior justifies a paid exposure under the supplied payoff matrix. This is not the same uncertainty configuration as the highly favourable no-change case. One starts from an action that already exploits the upside; the other starts from a robust refusal of the exposure. Both show why forecast improvement and decision improvement are distinct. Research value depends on where the posteriors fall relative to the action boundaries.
The expensive-signal fixture preserves the useful instrument and conditional actions while increasing the supplied acquisition price. Its gross information value remains the same, but its net value becomes negative. This clean comparison separates a methodological finding from a procurement recommendation. The instrument can be useful, and its reported performance can be credible, while its price is too high for this particular decision. A sponsor should not defend the purchase solely by pointing to the signal's gross benefit or a ceiling for perfect information.
The restricted-action fixture offers a third route to a poor purchase. The instrument remains unchanged, but the institution cannot take Commit. The positive signal can still improve the response within the remaining menu, yet the improvement is too small to justify the supplied price. This case is a practical warning against attributing all value to the evidence provider. The client action architecture determines how much useful evidence can become an improved outcome. A successful study delivery does not guarantee an executable response with enough value to cover the commission.
The purchase rule is conditional on including the relevant study cost. The app accepts a single aggregate amount. If the report is delayed, internal analysis is extensive or an operating trial disrupts service, those consequences need their own evidence and valuation. They may be included in a documented aggregate monetary cost if appropriate, or imposed as separate constraints. The engine does not calculate them automatically. A positive net information value based only on a vendor invoice is not a complete commissioning appraisal when other material costs remain unresolved.
The gross value provides a maximum justified purchase price within this fixture's assumptions, but that statement needs care. It is the price at which net expected monetary value reaches the comparison boundary, not a recommended bid or an objective market valuation of the study. Different institutions can have different payoffs and response menus, so the same instrument can support different willingness to pay. A procurement team should retain commercial negotiation, deliverable quality and contractual scope as distinct issues. The decision value is an internal ceiling for a specified use, not evidence of the provider's cost.
The gross signal value should not exceed the perfect-information value when the state, menu and payoff perimeter are fixed. This relationship is a useful reproducibility check and a semantic check on the study design. A claimed signal benefit above the ceiling can indicate a changed menu, inconsistent baseline, overlapping benefit or an implementation mechanism that has been added without disclosure. The right response is to inspect the perimeter rather than force the result into an expected relationship. The paper's frozen fixtures retain each component for that inspection.
There is a subtle distinction between an information purchase and an action commitment. Approving the study means accepting an expected-value rationale for learning under the declared decision. It does not authorise whichever action an eventual result recommends. The committee may need a separate review of feasibility, rights, liquidity and contractual conditions after the evidence arrives. Those conditions should be anticipated in the action charter so that the study does not depend on an implausible automatic commitment. A conditional policy can be a planning rule without being an unconditional execution mandate.
The study specification should preserve the expected action changes. If the final instrument addresses a different state, has different conditional performance or omits a branch, the original information-value calculation may no longer justify the commission. Delivery review should therefore compare the actual evidence process with the commissioned likelihood model and response requirements. The paper does not invent field performance for any instrument. It supplies a structure for checking whether a real deliverable answers the decision that was used to justify its cost.
An uncertain instrument can require a richer information-value model. The app treats sensitivity and specificity as fixed inputs, so it does not integrate uncertainty about those rates or evaluate an adaptive study design. A team may run explicitly varied fixtures to see how the purchase result changes, but that is a scenario comparison rather than a posterior distribution over study effectiveness. If performance uncertainty is consequential, the commissioning package should state the missing integration and use a suitable method. More plotted scenarios do not automatically become an empirical confidence interval.
The case slate supplies a purchase-failure classifier. No-policy-change fails through action invariance; expensive-signal fails through acquisition burden; restricted-action fails through the missing response; false-positive-burden exposes a signal that does not move the policy under its supplied account. A hypothetical procurement team should therefore identify the failure route before requesting an improved proposal. A discount can address price, but cannot make an absent response feasible. Better signal discrimination can affect a policy boundary, but cannot create authority. A redesigned action menu can change attainable value while leaving the instrument unchanged. Classifying the route prevents a generic request for more accurate evidence from missing the decision defect.
The purchase record should also distinguish evidence acceptance from action selection. A hypothetical instrument can deliver exactly the contracted result and still lead to Hold under the predeclared policy. That is a fulfilled commission, not a failed study. An instrument can deliver a persuasive result outside its accepted scope and leave the conditional policy unusable; that is an evidence problem even if the eventual action succeeds. An accepted result followed by an unavailable response creates an implementation problem. Recording these dispositions keeps the gross-value calculation tied to the assumed policy and provides a concrete basis for later contract and learning reviews, without claiming that a single realised outcome measures the expected value of the study.
Evidence and scope
DOE links measurement plans to defined baselines, methods and reporting requirements for performance verification.
Evidence: Measurement and Verification Activities Required in the Energy Savings Performance Contract Process
Gross and net signal value
Illustrative purchase decisions across distinct mechanisms
All amounts are in USDk. The unchanged-policy information value is exactly zero under the current rational evaluation. Complete JSON retains the exact fractions and their numeric display values.
Scroll for all columns. With the table focused, use the left and right arrow keys.
| Illustrative purchase decisions across distinct mechanisms | ||||
|---|---|---|---|---|
| Fixture | Gross signal value | Study cost | Net value | Mechanism |
| Useful signal | 33.20 | 5.00 | 28.20 | Conditional actions improve the policy |
| No policy change | 0.00 | 5.00 | -5.00 | Both branches still choose Commit |
| Expensive signal | 33.20 | 80.00 | -46.80 | Same useful signal, greater price |
| False positive burden | 0.00 | 4.00 | -4.00 | Both branches still choose Hold |
| Restricted action | 3.45 | 5.00 | -1.55 | A valuable response is unavailable |
Buy the conditional action improvement
Approve a decision-value commission only after comparing its expected conditional policy with the best current action and including the relevant cost. Keep gross usefulness separate from affordability. Reassess if instrument performance, action availability or the opportunity changes. Record any separate mandatory or exploratory purpose without claiming that the binary purchase calculation measures it.
- Compare against the best current feasible action.
- Inspect all branches of the proposed policy.
- Place the complete relevant cost in the perimeter.
- Preserve the ceiling and limitations beside the net result.
A negative immediate result can coexist with broader benefits
A study may support several future decisions, staff capability or a public evidence obligation. Those benefits can justify a broader commission when they are specified and reviewed. They should not be attached as vague rescue terms to a negative current-decision result. Define the additional use, avoid overlapping benefits and use a method appropriate to its horizon and action architecture.
The instrument-specific value assumes fixed performance
The engine does not infer performance from data or integrate uncertainty in its conditional likelihoods. Its net value uses a supplied aggregate cost and the fixed binary decision. Scenario comparisons can challenge these inputs but do not establish empirical instrument effectiveness, a sequential stopping rule or a research programme valuation.