How to check an AI-generated program design before a client does
You have used a model to draft a leadership program design. It reads well. It names Kotter in the change module, Edmondson in the team module, Kirkpatrick in the measurement plan. Somewhere in the client's building is a person who has read two of those three, and they will be in the room when you present.
So: what do you check, and in what order?
Every piece of guidance I can find answers a different question than the one you just asked.
The nearest thing to L&D-specific advice on this is Articulate's checklist for evaluating AI-generated content. Its guidance on sources is a single line: "Double-check citations. Just because a citation looks real doesn't mean it is. Look them up to confirm the existence and legitimacy of any listed references."
That is good advice for a literature review. University library guides say the same thing at greater length, and so does the academic-publishing advice written for researchers. All of it is built to catch one failure: the fabricated reference. A citation that points at a paper which does not exist.
Fabrication is a real problem. It is not your problem.
Your problem is that the works you are citing are famous. No model is going to invent Leading Change, or make up an Edmondson paper, or hallucinate the four levels. The books are in the training data thousands of times over. What the model does instead is subtler and worse: it attaches a plausible claim to a real, checkable, correctly-titled source.
Which means the test "does this source exist?" returns yes on every error that can actually damage you.
I have shipped a design with a sentence in it I could not have defended if the CHRO had asked me where it came from. Nobody asked. That is not a reprieve — it is the reason the habit survives.
It is worth being clear that the profession has not solved this either.
In April, TechTrends published the first part of a two-part position paper by Atsusi Hirumi, Dina Kurzweil, Lisa Giacumo, Chris Olsen, Efren de la Mora Velasco and Henry Moon. The abstract names three barriers to evidence-informed practice in instructional design, and the middle one is the one to sit with:
"EIP in ID remains constrained by persistent challenges: distinguishing scientific from anecdotal evidence, limited guidance in ID models for acquiring and appraising research, and insufficient time or resources for rigorous inquiry."
Read that middle clause again. The design models most of us were trained on tell you how to analyse, design, develop and evaluate. None of them tells you how to find out whether the thing you are about to build has support behind it, or how to judge that support once you find it. The paper's own abstract says Part II will set out methods for acquiring and appraising evidence, which is a fair summary of the state of things: the method is being written now, not something you were taught and forgot.
So we have a professional vocabulary about evidence and no professional procedure for it. Add a drafting tool that produces confident attributions at speed, and you get the situation we are in.
For what it is worth, the discomfort shows up in the survey data too. TalentLMS's 2026 benchmark report finds that 22% of learning leaders name unreliable AI-generated content as a blocker. I will also tell you that figure comes from 101 US HR managers surveyed in September 2025, because a piece about checking claims should not ask you to take one on trust. It is a small sample. Treat it as a signal that the worry exists, not as a measurement of how widespread it is.
Here is the taxonomy, with a worked example of each. I checked every example against a publisher record or the source organisation's own page while writing this. Where a source is contested or partisan, I say so.
The pattern: a real body of work exists. The specific number in your document was never measured.
Worked example: 70-20-10. This is the most-quoted number in our field. It is usually credited to the Center for Creative Leadership's Lessons of Experience (1988). ATD — our own professional body — reviewed the evidence and reported that the phrase "never actually appears in the book", that the underlying work involved asking 191 executives about the career events that shaped them, and that the model is "a conceptual or theoretical model based on retrospective musings by executives about what made them successful." The same review concludes there is "a lack of empirical data supporting 70:20:10."
So a design document that says "research shows 70% of learning happens on the job" has a real source behind it and a claim the source does not make. I have quoted it in a proposal as though it were a finding.
The check: ask what was measured, on whom, and whether the number was measured or inferred. If the answer is "executives remembered it that way," your sentence has to say so — which, usefully, is still a sentence worth writing.
The pattern: the idea is sound and the name on it may not belong there.
Worked example: the four levels. Donald Kirkpatrick published four articles across 1959 and 1960 in the Journal of the American Society of Training Directors, one per level, and the name stuck. Will Thalheimer of Work-Learning Research has published a documented case that Raymond Katzell described the four-level framework first, and that Kirkpatrick credited Katzell in a 1956 publication before omitting him from the 1959–60 articles. Thalheimer proposes calling it the Kirkpatrick-Katzell model.
I want to be careful here, because being careful is the whole subject. That is a researcher's argument, set out with its evidence, not a settled consensus. I am not telling you Katzell originated it. I am telling you that the attribution most confidently repeated in our profession has a live, documented dispute attached to it — and that is the last place any of us should be casual.
The cleaner case in the same family is the fifth level. Plenty of AI-drafted measurement plans contain a "Kirkpatrick Level 5: ROI." There is no such thing. ROI is Jack Phillips's addition; his own ROI Institute paper states that "The ROI Methodology adds a fifth level to the four levels of evaluation". Two models, two authors, one label — and a client who knows the difference will notice.
If the measurement chapter is where your drafts tend to go soft, the Level 3 measurement plan worked through end to end shows what the evidence chain looks like when each link is named.
The check: find the first publication of the idea, not the most famous one. Where the two differ, say which you are using.
The pattern: you are citing the author accurately. You are citing the version they have since replaced.
Worked example: Kotter's eight steps. His own firm's page states that "he evolved the linear 8 Step from Leading Change to the 8 Accelerators outlined in his 2014 book, Accelerate." The eight as that page currently names them are: create a sense of urgency, build a guiding coalition, form a strategic vision, enlist a volunteer army, enable action by removing barriers, generate short-term wins, sustain acceleration, institute change.
Compare that to the eight most AI drafts will hand you, and to the eight in the deck you have been reusing since 2019. A design that presents "Kotter's eight sequential steps" in 2026 is presenting a version its own author has moved on from. That is not a fatal error. It is an avoidable one, and it is exactly the kind of thing a client who has bought Kotter's consulting will spot.
The check: go to the author's or their organisation's current statement of the model before you cite it. Authors revise. Citations do not revise themselves.
The pattern: two people wrote something together, then separated, and the joint label now covers two different things.
Worked example: situational leadership. The origin is real and precise: Paul Hersey and Kenneth Blanchard, "Life Cycle Theory of Leadership," Training and Development Journal, volume 23, issue 5, 1969, pages 26–34 (ERIC EJ004286).
Then it splits. The Center for Leadership Studies — Hersey's organisation, so read it as one party's account rather than a neutral one — publishes its points of distinction: Blanchard founded his own company in 1979 and introduced SLII in 1985 as a revised model; Situational Leadership uses readiness levels R1 to R4, while SLII describes enthusiastic beginners, disillusioned learners, capable but cautious performers and self-reliant achievers; and CLS argues the two differ fundamentally on whether development is linear or iterative. The two marks are held separately.
So a module that says "Hersey-Blanchard tells us a D1 learner needs a directing style" has merged two models whose owners keep them formally apart — and has used one's terminology under the other's name.
The check: are you citing the joint original, or one party's later revision? Name which. If your client's existing leadership material uses one of them, match it rather than blending.
The pattern: the study is real, the finding is real, and your sentence claims more than the study measured. This is the most common one, and the hardest to see, because nothing in the sentence is false — it is just wider than the evidence.
Worked example: psychological safety. The paper is Amy Edmondson, "Psychological Safety and Learning Behavior in Work Teams," Administrative Science Quarterly, 44(2), 1999, pages 350–383 (publisher record). It defines team psychological safety as "a shared belief held by members of a team that the team is safe for interpersonal risk taking." It studied 51 work teams in a single manufacturing company. And the finding has a mechanism in the middle of it: psychological safety is associated with learning behaviour, and learning behaviour mediates between psychological safety and team performance.
An AI draft will give you "research shows psychological safety drives team performance." Two things have quietly gone: the single-site sample, and the mediator. The mediator is the part a program designer actually needs, because it tells you what your program has to produce — learning behaviour — rather than leaving you to hope performance arrives on its own.
The check: state the population, the method and the mechanism, not just the conclusion. If your sentence would be false outside one manufacturing firm in the 1990s, either add the qualifier or lower the claim.
Five questions per claim. The first time takes an afternoon. After that it runs at roughly a minute and a half per claim, because most of your claims recur across projects and you reuse the answers.
1. Write the claim as a sentence that could be false. "Grounded in the science of learning" cannot be checked, because it does not assert anything. "Spaced practice produces better retention than massed practice" can. If you cannot write your grounding as a falsifiable sentence, you do not have a claim yet, you have a mood.
2. Name the specific work. Author, title, year, and where it was published. Not the surname as a badge. If the model gave you a surname and no work, that is the answer to question 5 already.
3. Is this the author's current version? Check the author's own page, their organisation's own page, or the most recent edition. This is one lookup and it catches failure mode 3 entirely.
4. What was actually measured, on whom, and does my sentence stay inside it? Population, sample, method, and whether the effect was direct or ran through something else. This catches failure modes 1 and 5, which together are most of the risk.
5. Label what is left, honestly, in one of three settings. Supported — there is a real body of evidence and here is the specific finding. Contested — the evidence points this way, there is a credible argument the other way, here is why I chose this direction for your context. My judgement — there is no body of evidence behind this one, here is my reasoning and here is what would change my mind.
Two or three modules in most of my designs sit in the third category. Those are the ones sponsors quote back at me.
Put the label next to the module it justifies, not in a bibliography at the back. A grounding line under module four, three lines long, lets a client check one module without reading the whole document. A reference list at the end is a place claims go to avoid being checked. If you want to see what that looks like at full length, the twelve-week first-time manager design is published with its grounding stated module by module, including the modules where the honest answer was practitioner judgement.
Three moves, in order of preference.
Lower the claim to what you can support. Usually available, usually costs you nothing. "Spaced retrieval practice is well supported for retention" instead of "research proves this design will improve retention by 40%."
Move it to judgement and say what would change your mind. This reads as someone who has looked, not someone who is hedging. Sponsors are markedly less unnerved by it than people expect.
Cut it. If the claim is load-bearing and unverifiable, it is a liability in a document you are about to put your name on.
The counter-intuitive part, and the reason this is worth the afternoon: admitting the third category is what makes the first two believable. A design in which every claim is apparently settled science is one a careful reader stops trusting at the first claim they know is shakier than that — and then they doubt the rest of the document, including the parts that were solid.
In the spirit of the above.
I read the publisher's record and abstract for Part I of the Hirumi paper. I did not read the full text — it sits behind a paywall I did not pay. So I can tell you what the abstract argues and nothing about how the argument is made.
I could not open the record for Part II at all today, so nothing about its proposed method appears here, even though I know it exists, because Part I's abstract says so.
The Katzell attribution in failure mode 2 rests on one researcher's published argument, not on a consensus or on Katzell's original. I have said so twice and I am saying it again.
The situational leadership divergence comes from one of the two parties' own page. It is the best available account of the distinction and it is not a disinterested one.
The 22% figure is 101 US HR managers surveyed in September 2025.
None of those limits stopped me writing the piece. All of them are in it, because a piece arguing for traceable claims that hid its own would not be worth reading.
Most of this work happens too late — after the design is drafted, when the claims are already load-bearing and cutting one means rebuilding a module. It goes much faster when the diagnosis at the start of the engagement is sharp enough that you know which frame you are actually committing to before you write anything.
If you want to test that end of it, the Diagnostic Conversation Coach is free, needs no signup and takes about ten minutes. It runs the discovery conversation with you and shows you where a client's stated problem diverges from the real one. If it does not sharpen the way you run a discovery call, nothing else here will interest you either.