Engineer notes the sheer scale of modern language models has not, disappointingly, solved several genuinely basic use cases
HANGZHOU A local developer testing Alibaba’s newly launched large language model, boasting a reported 2.4 trillion parameters, says he remains unable to get the system to reliably summarize his own weekly grocery list without occasionally inventing items he never actually purchased.
The Specific Failure Case, As Documented
Developer Chen Wei fed the model a straightforward seven-item shopping list and received back, on three separate attempts, a summary that confidently included “a dozen eggs” despite eggs never once appearing in the original input.
Why This Bothers Him More Than It Probably Should
“Two point four trillion parameters and it’s inventing eggs,” he said, genuinely more baffled by this specific failure than impressed by the model’s considerable, well-documented capability on far more complex reasoning tasks elsewhere.
What the Model Actually Excels At, For Context
Chen confirms the system performs genuinely impressively on complex coding tasks, nuanced translation, and sophisticated reasoning problems, making the persistent grocery list hallucination “almost funnier because of how good it is at literally everything else.”
Whether This Represents a Broader Pattern
Several AI researchers interviewed note that scale doesn’t necessarily eliminate hallucination on simple, low-stakes tasks the way it improves performance on complex ones, a genuinely counterintuitive quirk of how these systems actually function under the hood.
What Chen Plans to Do About His Eggs Problem
He’s switched to manually reviewing any grocery-related output before trusting it, a workaround he finds “genuinely a little funny, that this is where my caution ended up being necessary despite the model’s genuinely enormous scale.” Further Asia tech coverage continues at bohiney.com.
SOURCE: https://bohiney.com
