The most dangerous marketing AI agent may be the one that gives the clearest explanation for the wrong decision.

Many evaluations stop at response quality: Is it factual? Coherent? Well written?

For marketing operations, that is incomplete. A recommendation can sound convincing while using the wrong comparison window, ignoring a stock-out, misreading blended ROAS or suggesting an unnecessarily aggressive budget move.

I would evaluate a marketing agent as a decision chain:

1. Evidence integrity

Are the sources, definitions, time windows and calculations correct?

2. Diagnostic validity

Does the conclusion follow from the evidence? Were plausible alternative causes tested?

3. Action quality

Is the recommendation specific, proportional, reversible and bounded by guardrails?

4. Restraint

Does the agent recognise missing evidence, ask for it and escalate when appropriate?

Hypothetical test: campaign CPA rises 25%. A weak evaluation rewards “reduce bids”. A stronger one supplies different contexts—a tracking outage, an ended promotion, conversion lag, a spend spike or stable contribution margin. The agent passes only if it distinguishes them and changes its response accordingly.

OpenAI’s evaluation guidance recommends explicit criteria and representative test data. NIST warns that laboratory benchmarks may not reflect real-world deployment and recommends evaluation in practical scenarios.

For CMOs, the implication is simple: model accuracy is not decision reliability.

Historical incidents, near-misses and expert overrides—properly anonymised and permissioned—can become a high-value evaluation set.

A persuasive answer is presentation. A defensible decision is performance.