Can you trust an AI answer in medicine?

The failure is not that it lies. It is that it lies well.

A language model produces the most plausible continuation of your question. Usually that is also the correct one. When it is not, the wrong answer arrives in exactly the same voice as the right one: same confidence, same clean structure, same authoritative tone.

That is what makes it dangerous for study specifically. You are learning the material, so you cannot yet tell the difference — the thing you would need in order to catch the error is the thing you are using the tool to acquire.

Check the source, not the tone. Confidence tells you nothing about accuracy; a page number you can open tells you a great deal.

What “grounded” actually means

The word gets used loosely. Worth separating three things that all get called grounded:

• Trained on medical text. The model saw textbooks at some point during training. This tells you nothing about any individual answer — the knowledge is diffused into weights, not retrievable.
• Web-connected. The model searched and summarised. Better, but the sources are whatever ranked well today, and quality varies from a society guideline to somebody’s revision blog.
• Retrieved from a fixed library. The answer is written from specific passages in specific books, and you are shown which. This is the only one of the three where you can check the claim against the source in a single step.

A practical check before you revise from an answer

• Ask where it came from. If there is no source, treat the answer as a hypothesis, not a fact.
• Open the source. Not the title — the page. A reference you cannot open is a claim about a reference.
• Read one paragraph around it. Most errors are not invented facts; they are true statements with the qualifier removed.
• Be suspicious of tidiness. Real medicine has exceptions. An answer with none has usually dropped them.
• Commit before you ask. Decide your answer first, then check. Asking first turns active recall into reading, which feels like studying and is not.

Where this leaves AI in a study routine

Useful for the moment you are stuck, for a mechanism that will not stick, for the “why is it not the other one” question a textbook does not answer directly. Not useful as the thing you learn from unchecked, and not a replacement for doing questions.

The version of this that works is boring: ask, verify against the page, then go and practise. That is also the argument for answers that carry their source — the verification step has to be cheap or nobody does it.