Apple researchers proved LLMs, like OpenAI's GPT-4o and Meta's Llama3, fail to correctly answer math questions on addition and subtraction, which kids aged 6-12 can answer. They claimed LLMs don't actually "reason" and instead simply focus on pattern-matching. The LLMs failed to correctly answer an addition problem on kiwis after irrelevant information (about kiwi size) was added to the question.