Generative AI in training design: useful assistant, risky author

Training teams are under constant pressure to produce more content, faster, in more languages. Generative AI looks like the perfect tool: it can draft lesson outlines, rewrite text for different reading levels, generate quiz questions and translate in seconds. Used carefully, it can save real time. Used carelessly, it can put inaccurate safety information in front of workers. This article sets out a practical, risk-aware approach.

What generative AI is

Generative artificial intelligence (GenAI) is a subfield of artificial intelligence (AI) that uses generative models to generate text, images, videos, audio, computer code or other forms of digital data.¹ These models learn the underlying patterns and structures of their training data, and use them to generate new data in response to input, which often takes the form of natural language prompts.¹ Generative artificial intelligence has been applied across multiple industries for content creation and automation.¹

The accuracy problem

The key risk for training is accuracy. Generative AI models can produce incorrect answers or "hallucinations."¹ This is not a rare edge case: According to McKinsey’s State of AI 2024 survey, about 44% of organizations have experienced at least one negative consequence from GenAI, with inaccuracy and hallucination among the top cited risks.¹ In a safety course, a confident but wrong statement about an exposure limit or an isolation step can cause harm. Every AI-generated fact must be checked against an authoritative source.

There are also legal questions: The development of generative artificial intelligence has prompted litigation over the use of copyrighted works to train AI models and the content produced by these systems.¹ Check your organisation's policy and your tools' terms before using outputs commercially.

Good uses in learning design

• Drafting outlines from objectives you supply.
• Rewriting approved text into plain language or shorter forms.
• Generating question variants from verified content, for expert review.
• Scenario ideas for case studies and role plays.
• First-pass translation, followed by review by a fluent subject expert.
• Summarising long documents for designers, not for learners, without review.

Uses to avoid or tightly control

• Generating technical facts, limits or procedures without a verified source.
• Publishing AI content to learners without expert review.
• Uploading confidential documents to tools that are not approved by your organisation.
• Automated assessment decisions with consequences for people's jobs, without human oversight.

A risk framework

The NIST AI Risk Management Framework is a useful reference. The NIST AI Risk Management Framework (AI RMF) is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.² It is evolving: The AI RMF 1.0 is being revised as part of the White House AI Action Plan.² For industrial users, note that On April 7, 2026, NIST released a concept note for an AI RMF Profile on Trustworthy AI in Critical Infrastructure.² Use the framework's ideas — map the context, measure risks, manage them, and govern the process — scaled to the size of your programme.

A practical workflow

1. Brief with clear objectives, audience and approved source documents.
2. Generate drafts, keeping prompts and versions.
3. Verify every fact with a subject expert against sources.
4. Pilot with learners and correct issues.
5. Approve through your normal document control.
6. Monitor feedback and update content when sources change.

Equity and access

AI can help make content accessible in more languages and reading levels, supporting inclusion. International bodies are working on this: UNESCO partners with countries and institutions on ideas, research, and evidence-informed digital learning policy and practice, especially in disadvantaged settings.³ Keep a human reviewer fluent in each target language.

How these models work, briefly

Generative artificial intelligence (GenAI) is a subfield of artificial intelligence (AI) that uses generative models to generate text, images, videos, audio, computer code or other forms of digital data.¹ These models learn the underlying patterns and structures of their training data, and use them to generate new data in response to input, which often takes the form of natural language prompts.¹ Text tools are mostly large language models; generative pre-trained transformers (GPTs) are a type of LLM that is pre-trained to predict the next word.⁴ Predicting plausible text is not the same as checking facts, which is the root of the accuracy problem.

Hallucinations in training content

In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation, delusion, or mirage) is a response generated by AI that contains false or misleading information presented as fact.⁵ Chatbots powered by large language models (LLMs), like ChatGPT, may embed plausible-sounding random falsehoods within their generated content, such as fabricated citations.⁵ In safety and compliance training, a fabricated limit value, regulation number or procedure step can cause real harm. Data quality matters too: biased or inaccurate training data can make an LLM's output less reliable.⁴

A review protocol for AI-assisted content

1. Ground the model in approved source documents (procedures, standards, manuals) rather than its general knowledge.
2. Trace every fact: each technical claim must map to a source the reviewer can open.
3. Subject-matter expert sign-off for safety-critical content, recorded in document control.
4. Check language and inclusivity: reading level, culture, gender and accessibility.
5. Pilot with learners and correct confusing items before release.
6. Label AI involvement internally so future reviewers know which content to re-check.

Prompt patterns that help designers

• "From this procedure, list the steps where errors are most likely and explain why", as a starting point for analysis.
• "Write five scenario-based questions at the 'apply' level, with plausible distractors based on common mistakes listed here."
• "Rewrite this paragraph for a reading age of about 12 without changing technical terms."
• "Suggest three ways to assess this objective practically."

Each output is a draft for the designer to judge, not a finished product.

Data protection and intellectual property

Check your organisation's policy before pasting internal documents or learner data into external tools. Prefer enterprise agreements with clear data-retention terms, and avoid uploading personal data from assessments. Review the licence status of generated images and the rights of any source material you provide.

Using AI with learners

AI tutors and chat assistants can answer questions at any hour, but should be grounded in approved content, show their sources and tell learners when to ask a human. Monitor a sample of conversations for accuracy, and keep a simple way for learners to report wrong answers.

Key takeaways

• Use GenAI to draft and adapt, not to author unverified facts.
• Every technical statement needs expert verification against a source.
• Apply a risk framework, such as NIST AI RMF, and keep humans accountable.

Frequently asked questions

Can we let AI write a whole course?

We do not recommend it. AI can draft, but experts must verify content, and learning design still needs analysis of real tasks.

Is AI translation good enough for safety training?

It can provide a useful first draft, but a fluent subject expert should review it before use.

What is the NIST AI RMF?

A voluntary framework from NIST for managing AI risks, with a playbook and resources. It is being revised.

How do we protect confidential documents?

Use tools approved by your organisation and follow its data policies before uploading any content.

Can we let AI write our induction course?

It can draft parts, but a qualified reviewer must check every fact against approved sources before release.

Will AI replace instructional designers?

It changes the work: less first drafting, more analysis, review, assessment design and learner testing.

Sources

Every factual statement marked with a number links to the source below. Sources were retrieved on 23 October 2026; check the publisher for later revisions.

1. Generative artificial intelligence (Wikipedia) en.wikipedia.org
2. AI Risk Management Framework (NIST) nist.gov
3. Digital education (UNESCO) unesco.org
4. Large language model (Wikipedia) en.wikipedia.org
5. Hallucination (artificial intelligence) (Wikipedia) en.wikipedia.org