Model validation for a decision engine: a checklist

What does model validation mean for a decision engine?

Model validation asks whether a decision engine fits the world. The question concerns the match between the equation and reality: do the dimensions capture what matters, and does a low score come before a bad outcome? A validated model earns trust for a defined purpose and a defined range of cases, and the validation report states both.

Verification asks a different question: does the software compute the equation correctly? A model can pass verification and still describe the world poorly, and a well-fitting model can carry an implementation bug. Reviewers who run both checks, in that order, learn where a problem sits. The figure lists the 6 validation checks in the order a reviewer runs them.

How do you define the purpose of a decision engine before validating it?

Write one sentence that names the decision, the user, and the output. For example: the engine scores a regional hub choice on 9 dimensions, for an executive committee, and returns a number between 0 and 1. The sentence sets the standard for every later check, because a model is validated against its purpose and always in context.

Add the range of cases the engine is meant to serve. EQOS Asia describes decisions that span several Asian countries, currencies, and regulatory settings (EQOS Asia method page, as of October 2026). A reviewer then knows which cases belong in the test set and which ones fall outside the claim, and the report can say so plainly.

How do you check the inputs of a decision engine model?

Check 3 things about every input: its definition, its source, and its date. The 9 EQOS Asia dimensions are coherence, integrity, resonance, care, trust, transparency, clarity, stability, and energy (EQOS Asia method page, as of October 2026). A reviewer asks for a written definition of each one, so two raters mean the same thing when they say 0.6.

Then check the scale. The EQOS Asia score runs between 0 and 1, while the simulator sliders run from 0 to 100 (EQOS Asia simulator page, as of October 2026), so a conversion rule belongs in the file. Structured interviews of 2 hours or 3 days and vectorized documents feed the Audit computation (EQOS Asia method page, as of October 2026), and a reviewer samples them for completeness.

Does the structure of a decision engine fit the decision it scores?

Structure means how the inputs combine. EQOS Asia states that its score is a weighted geometric product, so the lowest rating drags the whole system down (EQOS Asia 2008 back-test page, as of October 2026). A reviewer asks whether that behaviour matches the decision: a collapsed dimension should hurt a hub choice, and the product form does that by design.

Hand cases test the structure in minutes. In a simplified equal-weight model, 9 ratings of 0.5 return 0.5, 9 ratings of 0.8 return 0.8, and 8 ratings of 0.8 with one of 0.2 return about 0.69. Each result can be computed on paper. If the engine disagrees, the structure or its implementation needs repair before any back-test is worth reading.

How does a back-test validate a decision engine on past cases?

A back-test applies the engine to a past case with data that was public at the time, then compares the score with what happened. EQOS Asia applied its Codex to public 2006-2007 data for the 2008 crisis and reports φ(S) = 0.234, in the critical zone below 0.3 (EQOS Asia 2008 back-test page, as of October 2026). The page describes itself as a retrospective educational exercise.

A reviewer asks 3 questions of any back-test. Were the cases chosen before the outcome was known? Does the set include successes as well as failures? Was the data available at the decision date? EQOS Asia also publishes a WeWork back-test on public 2017 data with φ(S) = 0.189 (EQOS Asia WeWork back-test page, as of October 2026). Two published cases start the discussion, and a larger, pre-registered set strengthens it.

Which stress tests belong in a decision engine validation?

Stress tests move inputs and read the response. A one-at-a-time sensitivity test ranks the inputs by swing, and a what-if scenario moves several at once. A validated model responds in the direction an expert predicts, and by a size the expert finds plausible. A response that points the wrong way is a finding, and it goes into the report.

Add a boundary test: set one dimension close to 0 and confirm that the score collapses, and set all dimensions to the same value and confirm, in the simplified model, that the score equals that value. The EQOS Asia scale places the STOP zone below 0.618 (EQOS Asia method page, as of October 2026), so a reviewer also checks scores just under and just over that line for stable verdicts.

How do you document limits and keep a decision engine validated over time?

The validation report lists the purpose, the test cases, the results, and the known limits. The EQOS Asia method names a systematic ethical safeguard as the validation step of its Audit computation (EQOS Asia method page, as of October 2026), and a report adds the human review that surrounds the written material. Each report carries the engine version and the date, so a later reader knows which model was tested.

Validation continues after launch. The EQOS Asia model can be recomputed monthly through Sentinel (EQOS Asia method page, as of October 2026), which gives a reviewer a stream of fresh scores to compare with outcomes. A free 30-minute pre-diagnostic by video, held under NDA, is the place to ask for the validation material behind a score (EQOS Asia contact page, as of October 2026).

Frequently asked questions

About the author

Stéphane Jambu

President, EQOS Forecast Asia

Stéphane Jambu leads EQOS Forecast Asia and works as a consultant in e-commerce, search visibility, and AI. LinkedIn profile