Evidence quality belongs beside performance

A performance value and the evidence supporting it answer different questions. The value states how an alternative is represented in the comparison. The evidence record explains why that representation should be considered credible for the decision being made. A highly attractive value supported only by a sponsor's assertion should not become equivalent to a less attractive value established through a relevant operating record merely because both occupy numeric cells. Common formatting makes them look comparable before their evidential basis has been examined.

The record should begin with the proposition the value is intended to support. An acquisition quote may support the price of equipment within its stated scope. It may not support installed capital cost, integration expenditure or ongoing operating burden. A successful technical demonstration may support the ability of a component to perform a task under controlled conditions. It may not establish the ability of the organisation to operate the full system. The committee needs the proposition and perimeter before deciding how much confidence to place in a supplied criterion value.

Evidence can be direct, inferred or expressly assumed. Direct evidence still has a scope: a measured result belongs to a particular operating condition, period and measurement process. An inference uses that evidence to represent another condition or a wider system. An assumption may be necessary when neither is available, particularly in early option development. Those categories do not create a universal hierarchy of usefulness. A directly measured but irrelevant observation can be less useful than a carefully explained inference. Their value depends on the decision question and the consequences of error.

A source citation is not enough to establish transfer. A supplier may present a successful deployment in a different operating environment. That record can demonstrate that the approach has been used, while leaving local integration, staffing, data quality and service requirements unresolved. The transfer argument should identify which conditions are similar, which differ and why the differences do not defeat the proposed mechanism. Without that bridge, an external example becomes a persuasive image rather than evidence about the contemplated commitment. The dossier should preserve the limitation instead of letting the citation imply local validation.

Comparable evidence needs a common perimeter. If a reliability proposal is evaluated over a mature operating process while a commercial initiative is evaluated over an ambitious future scope, their impact judgements may be answering different questions. If one cost estimate includes integration and another excludes it, normalisation cannot make the values comparable. The analyst should reconcile scope, time horizon and included obligations before the score is calculated. A precise comparison of inconsistent estimates is still an inconsistent comparison. Its numerical neatness can make the problem harder to notice.

Confidence should initially be represented beside performance, rather than compressed into it. A committee can then distinguish a proposal with strong performance and weak support from a proposal with modest performance and strong support. Multiplying a performance score by a general confidence judgement obscures that distinction. It can also imply an expected-value interpretation that the confidence judgement does not justify. If a team uses such an adjustment, it needs a clear model of what the multiplier means. The additive Portfolio compass does not supply that model.

The alternative proposed here is an evidence acceptance rule. For each criterion, state the minimum support required at the relevant decision stage, identify weaker support explicitly and define what the recommendation can claim while the gap remains. A preliminary comparison may allow an assumed readiness value when it authorises investigation. A final commitment may require an executable delivery plan and evidence of operating ownership. The rule should be known before the committee discovers which proposal benefits from lenient treatment. Otherwise evidence standards can be adjusted opportunistically alongside weights.

An evidence gap should lead to a decision-directed question. Asking for more information in general often produces another large document without resolving the choice. The analyst should identify which uncertain value could change the recommendation and what observation would distinguish the relevant alternatives. If the leader remains preferred under all declared plausible values for a weakly supported attribute, investigation may still be useful for implementation planning, but its immediate selection value is limited within that challenge set. If the choice changes, the gap becomes a potential condition of approval or a reason to stage commitment.

The worked implementation-evidence studies use expressly authored bounds, rather than pretending to estimate uncertainty from data. They change the readiness and execution-risk representation of a fictional digital-twin proposal while keeping the other supplied assumptions fixed. The comparison asks whether those changes alter the preferred option. It does not assign a probability to either bound or claim that the bounds contain every possible outcome. Their usefulness comes from exposing what the model would do under stated alternatives. A real committee would need evidence to justify why those alternatives are appropriate challenges.

The evidence ledger should preserve disagreement about interpretation. An engineer and an operator may read the same demonstration differently because they are concerned with different failure mechanisms. An economist and a sponsor may disagree about whether a claimed impact is incremental to the reference action. Averaging their judgements can conceal the issue. The analyst should record the proposition each reviewer accepts, the proposition each rejects and the evidence that could reconcile the difference. A disagreement can be a useful specification of the next investigation rather than noise to be removed.

The ledger also needs provenance that survives a meeting. A value should link to the record used, its date, the scope it covers and the person responsible for interpreting it. If a committee revises a value during discussion, the reason should be attached to the revision. A spreadsheet cell that changes without a record is difficult to challenge later. The analyst should be able to reconstruct whether the change reflected new evidence, a new interpretation, a corrected scope or a sponsor's preference. Those are different causes with different implications for review.

Estimates should not acquire false certainty through successive copying. A preliminary assumption can pass from a concept note to a business case and then to a presentation, gaining the appearance of institutional acceptance without gaining evidence. The provenance record should retain the original epistemic status until a documented review changes it. Approval of a document is not necessarily validation of every estimate it contains. The committee should know which values were checked, which were accepted provisionally and which were outside the review's scope.

A missing value should remain missing until a responsible treatment is chosen. Substituting a neutral score can favour an option by hiding a weakly supported attribute. Substituting the worst score can eliminate an option without evidence that its performance is actually poor. The choice between exclusion, bounded scenarios and information collection belongs to the decision policy. The canonical app rejects incomplete numeric input and does not infer a missing value. That behavior makes the unresolved policy visible instead of performing an unexplained imputation behind the interface.

Assurance should be proportionate to consequence. A reversible trial can proceed with less complete evidence than a commitment that creates a long-lived dependency, provided the trial is designed to resolve a specific question. A minor uncertainty about a non-decisive attribute may not justify the same work as an unsupported assumption governing the preferred option. Proportionality does not mean lowering standards until a proposal passes. It means directing scarce verification effort toward the propositions that could change the decision or leave the organisation unable to respond after commitment.

The evidence discussion should end with a controlled use statement. The analyst can say that a value is supplied for illustration, accepted for screening, bounded for a challenge or established for a defined operational scope. The committee can then attach a condition to its recommendation. This is more informative than labelling an entire model high confidence while leaving individual assumptions unexplained. The record should make clear what the evidence permits the organisation to do now and what remains to be established before the next commitment.