Fact-Checking AI
AI tools can sound confident and still be wrong. This page looks at why that happens, why chatbots invent sources, and how to check AI output before you rely on it.
AI "Hallucination"
"Hallucination" in AI refers to when a model generates incorrect or misleading information that sounds plausible but isn't based on real data. This happens because AI predicts responses based on patterns rather than verifying facts. Since AI can confidently present false information, it's important to fact-check its outputs before using them.
ChatGPT and Fictional Sources
One common issue with ChatGPT and other LLMs is their tendency to generate fictional or nonexistent sources when asked to create a list of references. This happens because AI models do not retrieve data from a fixed database but instead predict text based on patterns in their training data.
Because of this, chatbots are not designed to function as search engines for finding reliable information. If you try to use them this way, you'll often receive responses that sound complete and authoritative but lack proper references. Without sources, it becomes difficult to verify the accuracy of the information provided.
Fortunately, progress is being made in improving the accuracy of GenAI by grounding responses in external sources. Some chatbots now link their responses to the sources they draw from, enhancing transparency. However, the reliability of these sources varies, and they may still contain misinformation or disinformation.
6 Things to Know About AI
AI-generated content is reshaping the information landscape, making it easier than ever to create and manipulate text, images, and videos. As AI tools evolve, it's important to stay aware of their strengths and risks.
Review the infographic below to better understand key takeaways about AI, misinformation, and digital literacy:
Open the infographic in a new tab (PDF)
Tips for Fact-Checking AI Content
While AI-powered search tools will likely improve in the future, they are not yet a reliable alternative to academic resources. For fact-based research, it's best to use library databases. You can use (YOUR LIBRARY)'s catalog, (YOUR LIBRARY) catalog, to find sources that are properly cited and vetted.
If you do use AI to assist in research, follow these steps to verify its claims:
- Cross-check with trusted sources.
- Do not rely on AI-generated content alone. Search for the same information in scholarly sources, books, and reputable news outlets to determine whether it aligns with expert consensus.
- Library databases and academic journals provide more credible and peer-reviewed information than AI-generated text.
- Evaluate the information critically. Ask yourself:
- Where did this information originate? Does it cite any sources?
- What data was the AI trained on? Does it include current, accurate, and credible research?
- Can I find similar information using trusted sources?
A Method for Checking: SIFT
Librarians teach a quick habit for checking information, called SIFT, created by Mike Caulfield, co-author of "Verified." It works for AI output too.
- Stop. Before you use or share a claim, pause and notice what you actually know about it and its source.
- Investigate the source. Find out who is behind the information and whether they have the expertise and track record to be trusted.
- Find better coverage. Look for what other reliable sources say about the same claim, and use the best of what you find.
- Trace claims, quotes, and media to the original context. Follow a claim back to where it started to see whether it has been changed or taken out of context.
Where AI Helps, and Where to Be Careful
Whether AI helps depends on the task. It can be genuinely useful for some kinds of work and unreliable for others, and the same tool can be both within a single assignment. The goal is not to avoid it or to trust it, but to know which job you are asking it to do.
Where it can help
Brainstorming and getting unstuck, summarizing or rephrasing material you already have, explaining a concept in plainer terms, tightening your own writing, and suggesting search terms or angles that point you toward real sources to check.
How to use it carefully
Treat every fact, number, quote and citation it gives you as unverified until you confirm it in a trusted source. Use it to find leads, not as the final word. Keep your own judgment in the loop, and disclose your use when your course requires it.
Where it falls short
It predicts plausible-sounding text, not verified truth. It can invent citations and sources. Its training data can be dated or biased, and unlike a real source, its output cannot be independently re-found the way a citation can.
Final Thoughts on AI and Information Integrity
While GenAI chatbots like ChatGPT should not be relied upon for factual research, they can still be useful for brainstorming, summarizing, and enhancing the writing process. However, the accuracy of AI-generated content remains inconsistent, and fact-checking is more important than ever in today's digital landscape.
Moving forward, AI literacy will be a critical skill for students and researchers alike. In Module 4, we'll explore how to use AI responsibly and effectively in academic settings.
Knowledge Check
What is an AI "hallucination"?
Why do AI chatbots like ChatGPT sometimes generate fictional sources?
Which of the following is the BEST way to fact-check AI-generated content?
What is a common reason AI hallucinates incorrect facts?