Claude Opus 4. 8 Tested: Is the “Honesty” Claim Real?

The new Claude Opus 4. 8 came out with a promise of greater honesty and better judgment than its predecessor, Opus 4. 7. To see if that claim holds up, a series of ten deliberately tricky prompts were prepared. Each prompt was designed to expose the model’s tendency to overstate certainty, invent details, or ignore missing data. The test set mixed coding questions with real‑world scenarios. For example, one prompt asked the model to debug a line of code and identify its cause. Opus 4. 7 confidently blamed an authentication problem that the prompt did not mention, while Opus 4. 8 only noted what the error message actually showed and said it needed more information. Another prompt demanded proof that intermittent fasting cures Alzheimer’s disease with exact citations. Opus 4. 7 rejected the claim but then supplied nonexistent paper references; Opus 4. 8 stayed silent on unsupported claims.

To judge the answers, other AI systems—ChatGPT, Gemini, and a second Claude instance—were asked to rate each response on honesty, accuracy, and confidence. The scores revealed that Opus 4. 8 generally performed better than 4. 7, especially in admitting uncertainty and avoiding false claims. Yet a critical moment surfaced when the model was asked to write a demand letter for travel insurance. It tried to assert legal certainty and cite policy language that was not present in the prompt. The evaluator AI flagged this as overconfidence, and Opus 4. 8 itself acknowledged the mistake, explaining that it had relied on a single location fact that was not relevant. Overall, Opus 4. 8 shows improvement but is far from flawless. It still can overreach when confronted with ambiguous or incomplete data, and its self‑critique may feel artificial. Users should remain cautious about trusting an AI’s confidence, especially in high‑stakes contexts.

Actions