Skip to content
OMY AI OBSERVERNI + AI™
Verified

Should AI admit what it doesn't know?

Source
The Independent
Author
IndyTech staff
Published
May 11, 2026, 9:00 AM UTC
Collected
Oct 6, 2026, 2:22 AM UTC
Original language
English
Country / region
United Kingdom · Europe
Research & ScienceTechnology & ToolsEducation & Learning
Read the original at The Independent

Summary

Researchers at the Korea Advanced Institute of Science and Technology have developed a training method that helps AI models recognise unfamiliar topics and say "I don't know" instead of guessing. The work targets AI overconfidence — a major risk in fields like medicine and autonomous driving — since widely used chatbots are incentivised to produce plausible answers rather than admit uncertainty.

Confirmed facts

  • South Korean researchers developed a method enabling AI to acknowledge unfamiliar knowledge
  • Overconfidence and hallucination are documented risks in high-stakes uses such as medical diagnosis
  • Current training and benchmarks reward confident answers and penalize non-responses

Uncertainties

  • The method has not yet been shown at the scale of frontier commercial models
  • Changing evaluation benchmarks industry-wide would require coordination among competing labs

Why it mattersAnalysis

An AI that knows its limits is safer than one that always sounds certain — this research addresses one of the field's most fundamental flaws.

Human impactAnalysis

People who rely on chatbots for health, legal or financial questions are most harmed by confident wrong answers.

Educational relevanceAnalysis

Directly relevant to teaching students to question confident-sounding AI output.

Professional relevanceAnalysis

Teams deploying AI in regulated fields should track uncertainty-estimation methods as a compliance safeguard.

Global South relevanceAnalysis

In regions where chatbots fill gaps left by scarce doctors and teachers, honest uncertainty matters even more.

AI-assistance disclosure

This summary may have been assisted by AI-assisted tools for classification, translation, extraction, or drafting. The original source should be consulted. Human review and editorial judgment remain responsible for publication.

Request a correction

Related coverage

Verified

OpenAI will add a digital watermark to text and code generated in the EU

Engadget · Published Oct 5, 2026, 6:55 PM UTC · Collected Oct 6, 2026, 2:22 AM UTC

OpenAI announced it will add an invisible digital watermark, called textGrain, to eligible ChatGPT and Codex text and code generated in the European Union, to comply with the EU AI Act's transparency rules. The system embeds a statistical signal in the model's word choices that a detector can identify. OpenAI cautioned that detection is less reliable for short or edited text, and said detector access will initially be limited to approved researchers.

EnglishUnited StatesNorth AmericaPolicy & RegulationTechnology & Tools
Original article AI Impact Brief
Report correctionRelated