Simple, yet powerful demonstration of the difference between artificial and human intelligence:
The model seems to primarily focus on the data’s summary statistics. It makes some observations regarding the Steps vs BMI plot, but does not notice the gorilla in the data.
[…] the model is unable to notice obvious patterns in its visualizations, and seems to focus its analysis on the data’s summary statistics.
On the implications of AI’s inability to see the pattern:
I have a few thoughts on potential implications:
First, it suggests that current LLMs might be particularly valuable in domains where avoiding confirmation bias is critical. They could serve as a useful check against our tendency to over-interpret data, especially in fields like genomics or drug discovery where false positives are costly. (But also it’s not like LLMs are immune to their own form of confirmation bias)
However, this same trait makes them potentially problematic for exploratory data analysis. The core value of EDA lies in its ability to generate novel hypotheses through pattern recognition. The fact that both Sonnet and 4o required explicit prompting to notice even dramatic visual patterns suggests they may miss crucial insights during open-ended exploration.
