Skip to content
ICM Holdings International
Search⌘ K
Index
Risk as a decision system

7 · Mechanisms and worked studies

Control effectiveness is a claim to test

Separate design intent, operation, test evidence and residual exposure

152 pages · Edition 2026-10-10

Mechanisms and worked studies

A control assurance worksheet

Control effectiveness is a claim to test

Control effectiveness is a proposition about a mechanism under relevant conditions. It should not be treated as an adjective attached permanently to a policy, installation or assurance report. The proposition has several parts: the control addresses the stated event path, is available when needed, performs its intended conditional function and is sustained by adequate resources. A failure in any part can change the response. The institution should identify which part a test establishes, because a successful functional demonstration does not settle availability over the exposure period.

The recurring case separates conditional efficacy from shared availability. The supplied prevention and mitigation effects apply when the support state is available. Increasing availability makes those effects applicable more often; it does not improve their conditional performance. A maintenance intervention may chiefly change availability, while a redesign may change conditional prevention. A response that changes both should disclose both changes and the evidence for each. This separation enables a useful question: what is the institution paying to improve, and what can be tested before it relies on that improvement?

A test should contain an adverse condition that could defeat the claim. Demonstrating a safeguard in a fully staffed, powered and rehearsed setting may establish capability under those conditions. If the risky event includes absent staff or lost power, the demonstration may not cover the decisive state. The challenge is not to make testing arbitrarily severe. It is to match the test to the mechanism being used in the decision. A relevant failure should have a recorded consequence for the model or the permitted act; otherwise the test becomes a ritual of documentation.

Evidence and scope

GAO connects cost assessment with technical baselines, assumptions and sensitivity.

Evidence: Cost Estimating and Assessment Guide

Operating availability requires a defined exposure window and a treatment of latent failure. A safeguard can appear installed and ready while a defect remains undiscovered until demand. An inspection interval may affect how long that state persists. The simple availability input does not distinguish visible downtime from hidden unavailability. A real assurance case should identify which is covered by the evidence. If only scheduled availability is measured, the analyst should explain whether the claimed protection also depends on unobserved on-demand performance. Combining these concepts under a single label can overstate what the record establishes.

Control cost should include the recurring resources required for the claimed operating state. A low-cost proposal that omits necessary support can be less economical than it appears, because its assumed availability may not be deliverable at that cost. Conversely, a high-cost proposal may include capabilities outside the loss mechanism being measured. The model cannot decide whether those capabilities should be valued separately. The analyst must identify the cost boundary and avoid comparing an incomplete cheap package with a complete expensive package as though their scope were identical.

The availability-repair and cost-heavy-repair fixtures retain the same protective effect and differ in annual resource cost. This is a controlled comparison of the optional economic condition. It demonstrates that improved residual loss does not by itself justify a response. The expensive branch can deliver the same stipulated protection while failing the stated net-value floor. A report that includes only the loss reduction would hide that failure. A report that rejects all expensive protection on this basis would also overreach, because a mandatory constraint or another consequence may justify resources beyond the optional expectation measure.

Break-even availability is another conditional screening output. It identifies the shared availability at which avoided expected loss matches the declared annual cost under the fixed gross event and conditional effects. It is not a standard of protective adequacy. A control can pass that economic threshold while leaving a consequence the institution cannot accept. A proposed availability below the threshold can fail the optional condition even if the control has a genuine protective effect. The output should be read as a relationship between the supplied cost and mechanism, with a separate record for evidence and appetite.

Illustrative calculation. Assumptions and sensitivity are published with the model.

Same protection, different optional economic answer

The cases share the same gross event and conditional effects. Money is USD thousand per year; break-even availability is a cost threshold, not a protective acceptance standard.

Scroll for all columns. With the table focused, use the left and right arrow keys.

Same protection, different optional economic answer
StudyResidual expectationAnnual costNet annual valueBreak-even availabilityOptional disposition
Availability repair38.0060.00102.0033.33%Eligible; held
Cost-heavy repair38.00180.00-18.00100.00%REJECT optional value
Free ineffective control200.000.000.000.00%HOLD protection claim
Illustrative calculation. Assumptions and sensitivity are published with the model.

The availability repair produces net annual value of USD 102.00 thousand, while the cost-heavy repair produces USD -18.00 thousand under the same stipulated protection. The latter is rejected under the fictional optional floor of USD 0.00 thousand per year. A mandatory objective can reopen the question under a different stated mandate; it cannot make the frozen optional result positive. This preserves the adverse answer without using an annual screening measure to overrule obligations outside the model.

Illustrative. The free ineffective fixture reports avoided expected loss of USD 0.00 thousand. Its zero break-even convention therefore supplies no protective evidence. The completed disposition holds any claim that the activity mitigates the specified event, while permitting a separate assessment of administrative or mandatory purposes. The result shows why a control dashboard should report the causal effect and the economic threshold as different fields. A favourable value in one field can coexist with an absent protective effect.

The free-ineffective-control case tests a subtle interpretation error. With no annual cost, the canonical break-even convention can return a zero availability threshold even when the conditional effects are absent. This is arithmetically coherent for the supplied cost comparison: no recurring cost needs to be recovered. It does not mean the activity protects the objective. The same case has no avoided expected loss. A committee that treats a low break-even threshold as an assurance score would therefore be reassured by a control that does nothing under the stated mechanism. The adverse branch is to reject that interpretation, not to dispute the arithmetic convention.

An ineffective positive-cost control tests another boundary. Improving availability cannot create a protective effect when the supplied conditional prevention and mitigation are absent. The control may still perform another valuable function, but that function needs a separate mechanism and basis. The disciplined response is to stop crediting the activity against the dispatch-loss claim while investigating any other mandate. Renaming the activity or increasing its availability percentage does not repair the missing causal intervention. An institution should be able to record that conclusion without implying that every aspect of the activity is useless.

Illustrative calculation. Assumptions and sensitivity are published with the model.

Equal hypothetical protection does not remove a resource difference

Each series applies the same hypothetical support availability to its retained package, with identical gross event and conditional effects but different annual costs. The actual fixture comparison separately retains each declared proposal availability.

Original canonical total-cost sweep selection. These are discrete hypothetical availability tests, not delivered package performance or an optimal supplier choice. Compare the actual fixture values in the cost table before assigning a disposition.

Original canonical total-cost sweep selection. These are discrete hypothetical availability tests, not delivered package performance or an optimal supplier choice. Compare the actual fixture values in the cost table before assigning a disposition.

Assurance should preserve the difference between test completion and claim acceptance. A test report can be complete, reproducible and relevant while revealing that the intended performance has not been achieved. The owner should address the failed claim, and the acceptance authority should decide the permitted exposure. The assurance reviewer should not revise the success criterion after seeing the result merely to avoid a negative report. Nor should management treat every negative observation as proof that the broader project must stop. The response follows the consequence of the failed proposition within the decision record.

Completed assurance-test design

The record specifies what would challenge the availability-repair claim before an actual operating report is available.

Scroll for all columns. With the table focused, use the left and right arrow keys.

Completed assurance-test design
Test elementCompleted designAuthority consequence
ObjectShared support enabling prevention and mitigationNo credit for unrelated installed activities
Relevant adverse stateLoss of the operating support relied on by both functionsReconstruct mechanism if functions respond differently
Evidence soughtDemand-relevant test basis and operating exposure recordNo substitution of installation certificate
Cost boundaryRecurring resources needed for proposed availabilityReappraise if support omitted
Change ruleSupport, staffing, maintenance or conditional efficacy changesReassessment required
Current resultNo actual report providedImplementation HOLD

A control test also needs a change rule. The reviewed operating configuration, support arrangement and conditional performance basis should be named. A later supplier substitution, staffing change or maintenance reduction can invalidate the earlier conclusion. An assurance statement with no expiry condition can outlive the system it describes. The record should specify what changes require reassessment and who must notify the authority. This is particularly important for an availability claim sustained by recurring resources, because a budget reduction can alter the mechanism without changing the installed hardware.

Test the claim that matters

Evaluate the specific conditional effect, operating support and cost boundary. Keep economic recovery thresholds separate from protective adequacy.

  1. Define a relevant adverse state before testing.
  2. Reject a protective interpretation when avoided loss is absent, even if break-even availability is low.
  3. Preserve the optional rejection when a more expensive package has the same protection.

In the fictional dispatch record, the availability repair remains an evidence purchase rather than implementation. The custodian is instructed to obtain the support-performance basis, the cost boundary and the relevant test conditions within the bounded mandate. The Operating Owner must address applicability. The cost-heavy branch is not carried forward merely because the same availability certificate might support it. Its optional economic failure is already visible under the favourable stipulated inputs. The committee can therefore request targeted evidence for the eligible branch while retaining a completed rejection of the expensive optional proposition.

Direct tests cannot cover every operating condition

Testing can itself be costly or unsafe, and some rare adverse states cannot be recreated. The response may combine design analysis, bounded judgment and monitoring, provided the untested claim remains visible and the authority accepts that evidential limit.

No empirical control effectiveness is inferred

All conditional effects, availability and resource costs are illustrative. The model calculates their implications; it does not establish test coverage, latent failure rates or delivered performance.

Evidence at the point of use

Open a citation in the chapter to inspect its source, stated support and limits.

Find the next connection

Open the full search desk