Objectives must be distinguishable

A criterion should tell the committee what difference between alternatives matters and why that difference belongs in the decision. A list of attractive words does not yet do this. Resilience, innovation, efficiency and sustainability can each refer to outcomes, implementation methods or organisational aspirations. If their meanings remain open, sponsors can describe the same feature favourably under several headings and reviewers can interpret the same score differently. The criteria architecture needs to establish a stable question before anyone supplies a performance value.

An outcome describes a state the organisation values, such as more reliable service or better information for an operating decision. A means describes how a proposal may produce that state, such as a sensor network, an additional maintenance crew or a changed operating procedure. Confusing the categories can reward an expensive means even when another intervention produces the valued outcome more effectively. An organisation may have a legitimate architectural requirement, but it should identify that requirement directly. Preference for a familiar implementation should not acquire the status of an outcome merely through the name of a criterion.

A proxy stands between an outcome and what can be measured. The number of functioning sensors may be easier to verify than the quality of the decisions they improve. The proxy can be useful when the relationship is specified and the limitation is understood. It becomes misleading when the committee treats verified sensor installation as verified operating benefit. The evidence record should identify the link from measured attribute to valued outcome, the conditions supporting that link and the circumstances in which it could fail. A proxy without a mechanism invites metric substitution.

Constraints serve another role. A system may need to connect to an existing interface, meet a safety requirement or operate within an approved site envelope. These conditions define what the committee can consider. Their evidence may resemble a performance measurement, but their interpretation is different. A preference score can distinguish better and worse performance within the admissible envelope. It should not permit a benefit elsewhere to compensate for a prohibited departure. Keeping a constraint outside the additive score can make the comparison simpler and the authority behind it clearer.

Implementation attributes describe the route to an outcome. Readiness, available skills and integration complexity can matter because they change the timing, resource demand or likelihood of delivering benefit. They should not automatically be scored as benefits in themselves. A ready proposal with little useful outcome may be easy to deliver and still deserve little discretionary capital. Conversely, an immature enabling proposal may be important to a strategy, although the immediate commitment should remain small and conditional. The architecture should identify why an implementation attribute is relevant to this decision stage.

Double counting occurs when the aggregation rewards the same valued difference through overlapping criteria. Suppose a proposal receives a strategic-impact judgement partly because it reduces service interruptions, then receives a resilience judgement based on those same reductions. If both scores enter the sum, the committee may give interruption reduction more influence than it intended. The problem is not that the criteria are correlated. The problem is that their definitions allocate the same consequence twice without an explicit preference decision. A causal map can make that allocation visible before weights disguise it.

Correlation alone is an insufficient deletion rule. Service reliability and customer trust may move together while representing distinct outcomes that a committee legitimately values. Initial cost and implementation complexity may be associated without meaning the same thing. Removing one criterion because its values resemble another can erase a legitimate concern. The overlap test should ask whether improving a proposal on the underlying mechanism changes several criterion judgements for the same reason. If so, the committee should decide whether it intends that cumulative influence. Statistical resemblance is a prompt for interpretation, not a substitute for it.

The opposite error is to assume independence merely because the criteria have different names. A readiness assessment may already include procurement risk, operating skills and technical maturity. An execution-risk score may include those same attributes. Giving both substantial weight can impose an unintended penalty on an unfamiliar proposal. An analyst can preserve both criteria when their roles differ, for example by defining readiness as evidence of an executable plan and risk as consequences of residual failure after that plan is established. The distinction requires actual scoring definitions. A heading alone cannot establish it.

Strategic impact is especially vulnerable to becoming an omnibus criterion. If it includes revenue, cost savings, risk reduction, stakeholder benefit and delivery confidence, adding separate cost, risk and readiness dimensions can recreate those concerns within the total. The analyst should unpack the impact narrative and decide which components belong elsewhere. In the Portfolio compass, impact is a supplied judgement rather than a measured common outcome. The user therefore needs an external definition that excludes whatever is intentionally scored under the other dimensions. The app cannot inspect the prose reasoning behind a supplied number.

A criteria tree can help, provided the hierarchy is substantive. At the top sit the outcomes that justify intervention. Beneath them sit attributes used to judge differences in those outcomes. An implementation branch can identify how delivery conditions affect the usefulness of a commitment, while an eligibility register holds requirements that cannot be traded. Every leaf should have a description, direction of preference and evidence rule. The tree need not become an elaborate diagram. Its purpose is to prevent a broad label and its detailed components from entering the aggregation as separate unacknowledged rewards.

A useful practical test is a thought experiment rather than a new formula. Ask a scorer to imagine a proposal whose interruption exposure improves while every other relevant feature remains unchanged. Which criterion scores would move, and what consequence justifies each movement? If the scorer changes several values for the same benefit, the tree needs review. If the scorer changes none, an important outcome may have no representation. The exercise reveals how the definitions work before actual proposals create defensive positions around them.

The test should be repeated with cost and implementation maturity. If a cheaper proposal gains under cost and strategic impact because impact includes financial efficiency, the cost preference may be represented twice. If improved readiness gains under readiness and reduces execution risk, the committee should explain whether it values earlier implementation separately from a reduced likelihood of failed delivery. These can be distinct concerns, but their separation must be deliberate. The architecture should make it possible to describe a plausible change that moves a criterion without automatically moving every neighbouring judgement.

Evidence definitions should be stable across alternatives. A proposal should not receive an impact judgement from a detailed operating study while a competitor receives a judgement from an untested sponsor narrative without that difference being visible. The architecture should identify the acceptable evidence and the treatment of weaker support. It can allow different information at an exploratory stage, but then the recommendation should carry the resulting limitation. A common performance scale does not create common evidential quality. The latter belongs beside the supplied value in the record.

The architecture also needs a retirement rule for criteria. A concern may be important during option development and cease to distinguish the admissible shortlist. If every eligible proposal meets the same interface requirement, retaining it as a scored criterion adds presentation without distinguishing performance. If a requirement changes into a genuine preference within the admissible envelope, the change should be documented. Criteria should follow the decision problem, but they should not be edited opportunistically after a favoured proposal performs poorly. Change control protects both relevance and legitimacy.