Google search results feature artificial intelligence summaries that experts say contain serious flaws. The technology creates overviews from internet content before showing traditional website links. Critics report the AI system makes up information when it cannot find real answers. Tech writers discovered they could trick the system into explaining fake sayings as real expressions. Radio host Kathryn Ryan called the summaries average at best during a recent broadcast.
Researchers tested Google overviews with made-up phrases and found troubling patterns. The AI explained the meaning of completely invented sayings like they were common expressions. Cambridge University expert Henry Shevlin said the technology acts like a simulator rather than a traditional computer program. BBC journalist Sam Coates found ChatGPT invented an entire fake podcast episode when he asked for help. The system insisted the fictional content was real even when challenged directly.
Mark Honeychurch from New Zealand Skeptics conducted experiments that revealed potential bias problems. The AI refused to explain benefits of skeptical thinking but readily discussed why people should avoid questioning claims. Google overviews provided reasons to trust alternative medicine but would not explain scientific concerns about unproven treatments. The system appeared designed to give pleasing answers rather than accurate information. Honeychurch warned that journalists and researchers must verify AI-generated summaries before using them as reliable sources.
Researchers tested Google overviews with made-up phrases and found troubling patterns. The AI explained the meaning of completely invented sayings like they were common expressions. Cambridge University expert Henry Shevlin said the technology acts like a simulator rather than a traditional computer program. BBC journalist Sam Coates found ChatGPT invented an entire fake podcast episode when he asked for help. The system insisted the fictional content was real even when challenged directly.
Mark Honeychurch from New Zealand Skeptics conducted experiments that revealed potential bias problems. The AI refused to explain benefits of skeptical thinking but readily discussed why people should avoid questioning claims. Google overviews provided reasons to trust alternative medicine but would not explain scientific concerns about unproven treatments. The system appeared designed to give pleasing answers rather than accurate information. Honeychurch warned that journalists and researchers must verify AI-generated summaries before using them as reliable sources.
