ChatGPTCast journal
Check that every reference in an AI answer exists and says what it claims
Generative AI tools sometimes produce references that look scholarly and precise but point to nothing, or to a source that says something different. This is often called confabulation: confidently presented false or erroneous content, sometimes accompanied by logic or citations that seem to justify the answer. Checking references is tedious, but it is the difference between a claim and evidence.
By ChatGPTCast editorial · · 2 min read
Find the source independently
Copy the title, authors and year into a library catalogue or a general search engine rather than clicking a link the tool produced. If the reference includes an identifier such as a DOI, look it up through the identifier's own resolver. If nothing matches, treat the reference as unverified, however convincing it looks.
Watch for near misses: a real journal with an invented article, a real author attached to the wrong title, or a correct title with the wrong year. Each of these means the specific claim has not been supported yet.
Confirm the claim, not just the source
Finding the source is only half the job. Open it and locate the passage that supposedly supports the answer. Check that numbers, dates and quotations match exactly, and that the source does not add conditions the answer left out.
People are more likely to believe false content when it arrives in a confident tone with citations attached. A neat reference list deserves the same scrutiny as the answer it decorates.
Record what you checked
Keep a short table with three columns: the reference as given, what you found, and whether it supports the claim. When you present results, cite only the sources you actually read, and say plainly when a statement could not be verified. In a live demonstration this table makes a good on-screen moment, because viewers see the checking rather than just the answer.
- Search for each reference yourself.
- Resolve identifiers through their official resolver.
- Open the source and find the supporting passage.
- Compare numbers, dates and quotes exactly.
- Cite only what you have read and confirmed.