Why AI Strategy Decks Get an Easy Review: The Governance Fix

Reviewers scrutinise AI-generated strategy decks less rigorously than hand-built work, and the effect has a name: automation bias. Evidence from aviation, medicine and consulting shows it strikes experts too. Five structural governance fixes counteract it without asking anyone to try harder – because exhortations to be more sceptical have no reliable effect on a cognitive bias.

The fixes are: blind re-examination, in which the deck is stripped of formatting and citations before the second review pass; a named devil's advocate; a structured dissent log; source-first review; and a minimum review period. Each one converts a general expectation of scepticism into a specific, owned task with a specific output.

In the mid-1990s, Kathleen Mosier of San Jose State University and Linda Skitka of NASA Ames Research Center gave a name to a pattern aviation psychologists had been observing: automation bias – the tendency of decision makers to rely on automated cues as a heuristic replacement for vigilant information seeking and processing.^[1]

The bias expresses itself in two distinct error types, identified in Goddard, Roudsari and Wyatt's 2012 systematic review in the Journal of the American Medical Informatics Association:^[3]

• Errors of commission: the reviewer follows incorrect advice from the system, overriding a correct prior judgement because the automated output looked authoritative.
• Errors of omission: the reviewer fails to act, or fails to notice a problem, because the system did not flag it – heuristic replacement for independent information seeking.

Two features deserve board attention: (1) Automation bias was documented in experienced commercial pilots, not only in students – expertise does not exempt anyone. (2) The bias is cognitive rather than moral. Exhortations to be more sceptical fail; fixes must be structural.

Dell'Acqua et al. at Harvard Business School with BCG in 2023, 758 BCG consultants: inside the AI frontier, consultants using AI completed 12.2% more tasks, 25.1% faster, at significantly higher quality. On a complex managerial task deliberately placed just outside that frontier, consultants using AI were on average 19 percentage points less likely to produce correct solutions than the control group.^[4]

The study's most important finding for review governance: independent graders rated AI-assisted responses higher on coherence, clarity and persuasiveness regardless of whether the underlying answer was correct – including among the responses that were wrong. Polish was scored as quality.

The Goddard et al. review found automation bias occurs more often with task-inexperienced users but also occurs with experienced users. Physicians with greater experience may be somewhat less reliant on decision support without being immune to incorrect advice. The mechanism is a property of human cognition under load.

Generative AI intensifies a bias first measured in cockpits and clinics because it manufactures, at scale, every surface signal that heuristically indicates quality:

• Clean formatting and consistent structure – the visual grammar of a finished product, which reviewers associate with careful work.
• Citations and source-flavoured references – which signal verification whether or not verification occurred.
• Grammatical correctness – which readers unconsciously read as analytical care.
• A sound executive narrative arc – which signals rigour independent of the rigour actually applied.

Goddard et al. identify workload, task complexity and time constraint as environmental mediators that push users toward heuristic acceptance.^[3] A board calendar supplies all three at once: complex decisions, time pressure and cognitive load from an agenda that runs to dozens of items.

A 2024 study from the Center for Security and Emerging Technology at Georgetown University examined automation bias across three case studies – Tesla's autopilot incidents, aviation incidents at Boeing and Airbus, and Army Patriot and Navy Aegis air-defence incidents – and concluded that human-in-the-loop oversight cannot prevent all errors if the organisational process is not designed to counteract the bias.^[6] The humans were present in every case. What differed was whether the process was designed to make their oversight effective.

Two fixes attack the surface signals and the diffusion of responsibility that automation bias exploits:

Fix 1: Blind re-examination

Before the second review pass, strip the deck of formatting, citations and polish, and circulate a plain rendering of its claims, assumptions and figures. Reviewers interrogate the analytical skeleton without the persuasive dressing. The surface signals that trigger heuristic acceptance are removed; what remains is the claim, the assumption behind it, and the figure the recommendation rests on.

Fix 2: A named devil's advocate

Assign one reviewer, by name, whose sole job is to find the flaw in the recommendation. This converts a general expectation of scepticism into a specific, owned task with a specific output: the strongest available case against the recommendation, written down before sign-off.

Both fixes have empirical grounding in the accountability findings of Mosier and Skitka: participants who perceived themselves accountable for their strategies of interaction with the automation were significantly more likely to verify its correct functioning and committed significantly fewer automation-related errors.^[2]

Fix 3: A structured dissent log

Each reviewer must record at least one disagreement, question or unresolved concern per section before signing off. This forces engagement with the material rather than skim-reading, and it produces a written record of where the recommendation is fragile. The dissent log is not a grievance record – it is the evidence that the review was genuine.

Fix 4: Source-first review

Reviewers read the underlying source documents before seeing the AI synthesis, reversing the usual order. They form an independent view of the evidence first, so the generated deck is compared against the sources rather than merely admired in isolation. This breaks the path-dependence that makes the first impression the last.

Fix 5: A minimum review period

Set a protected floor for how long review takes. Treat a same-day turnaround on a complex AI-assisted recommendation as a process failure in itself. Goddard et al.'s finding that time constraint is a mediator of automation bias makes the minimum review period a governance control, not a scheduling preference.

The CSET study's central recommendation: organisations can embed or counteract automation bias through their protocols, processes and training.^[6] Process design, not exhortation, is the effective mitigator.

Boards should require a signed statement from the review chair confirming each of the five fixes was applied to any AI-assisted recommendation put to the board:

• A blind re-examination pass was conducted.
• A devil's advocate was named and filed a report.
• A dissent log was completed per section.
• Reviewers saw source documents before the synthesis.
• The minimum review period was respected.

The statement should also record exceptions explicitly – if a fix was not applied, the statement says so and why. Exceptions are not failures; undisclosed omissions are.

The reason a process attestation is the right ask: scepticism is invisible after the fact. Nobody can inspect a state of mind. A signed record of process can be checked, compared and challenged. It creates precisely the perceived accountability that the Mosier and Skitka research found measurably reduces automation-related errors.

This example is an illustrative composite, not empirical data from a named company. A strategy committee at a mid-sized industrial group received an AI-assisted market-entry recommendation – exemplary on its face: consistent structure, a sourced-looking evidence base, a clean options comparison. Approved with one meeting of discussion.

Eighteen months later the entry had materially underperformed. The post-mortem found no fraud and no obvious negligence. One load-bearing assumption – about how incumbent competitors would respond on pricing – rested on a market figure that was two years stale. No reviewer had examined it. The deck's polish had substituted for scrutiny.

Changes made after the post-mortem:

• Blind re-examination rule adopted: any AI-assisted recommendation circulates to the second reviewer as a plain claim sheet, stripped of formatting and citations.
• Named devil's advocate added to every significant AI-assisted recommendation.
• Dissent log requiring at least one recorded concern per section.

What the new process would have caught: the stale pricing-behaviour assumption was visible as a dated figure once the polish was stripped, and the claim sheet identified it as unsupported during the blind re-examination pass. It was invisible in the formatted original; it was obvious in the skeleton.

Platforms such as Decisity make the structural fixes cheaper to run, because every claim in a generated deck can be traced to its source and re-examined without the dressing. The blind re-examination pass costs a fraction of the time when the plain claim sheet is a feature of the platform rather than a manual extraction exercise.

Related reading:

• AI Governance Framework for Boards
• Presenting AI-Assisted Strategy Analysis to a Skeptical Board
• Who Is Accountable When an AI-Drafted Strategy Goes Wrong?
• Do AI Strategy Platforms Make Every Firm's Analysis the Same?

• Mosier, K. and Skitka, L. (1996): Human decision makers and automated decision aids – made for each other? NASA technical report
• Mosier, K. and Skitka, L. (1996): Automation bias and automation-complacency in high-stakes decision making. Human Factors and Ergonomics Society annual meeting proceedings
• Goddard, K., Roudsari, A. and Wyatt, J. (2012): Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association
• Dell'Acqua et al. (2023): Navigating the Jagged Technological Frontier – Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Harvard Business School
• Ethical Psychology: Automation bias systematic review (2026)
• Center for Security and Emerging Technology (CSET), Georgetown University: AI Safety and Automation Bias (2024)