AI Accuracy Study Reveals Google's Progress and Pitfalls
Key Takeaways
- 1Google's Gemini 3 shows 91% accuracy in AI responses.
- 2Verifiability declined from 37% to 56% with new model.
- 3Increased reliance on questionable sources raises concerns.
A study commissioned by the New York Times and conducted by the AI startup Oumi analyzed 4,326 Google searches to evaluate the accuracy of Google's AI-based search responses using its Gemini models. The findings revealed that while the accuracy of AI-generated overviews improved significantly from 85% with Gemini 2 to 91% with Gemini 3, the capability to verify these responses deteriorated. Notably, the proportion of correct answers linked to unsubstantiated sources rose sharply, highlighting a critical challenge as Google enhances its AI accuracy at scale.
This shift underscores a vital strategic concern: as Google's AI becomes more accurate, reliance on sources such as Facebook and Reddit raises red flags about the overall integrity of information provided to users. The declining verifiability suggests that users may not receive the most accurate and reliable information despite improvements in response accuracy. Consequently, while Google's AI advancements strengthen its position, they also call attention to potential dependency on unreliable information sources, posing a risk to data sovereignty in AI systems.