A recent study found that various AI models have a hard time reliably reading an analog clock or figuring out the day on which a date will fall.
“Most people can tell the time and use calendars from an early age. Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people,” study lead author Rohit Saxena, a researcher at the University of Edinburgh, said in a statement. These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies."
That being said – and we have seen this many times now – the models being used in the study were GPT-4o, Claude 3.5, and Gemini 2. These are all models which are long out of date (GPT-4o, which a lot of these studies reference, was introduced on May 13, 2024 and has, for quite a while, not been available in the ChatGPT interface). Which brings up the bigger question of how insightful these studies are – and how much headlines such as “AI models can’t tell time or read a calendar, study reveals” really mean.
To investigate AI’s timekeeping abilities, the researchers fed a custom dataset of clock and calendar images into various multimodal large language models (MLLMs), which can process visual as well as textual information. The models used in the study include Meta’s Llama 3.2-Vision, Anthropic’s Claude-3.5 Sonnet, Google’s Gemini 2.0 and OpenAI’s GPT-4o.
