AI Still Fails at Common Sense Vision: Inside the HSS Benchmark That Humans Ace at 93% and GPT 6 Astra Barely Passes

The gap between seeing and understanding has never been more measurable. A new benchmark from AI startups Elorian and Scale AI, called Humanity’s Sixth Sense (HSS), puts hard numbers on a problem the field has long acknowledged in the abstract: vision-language models can describe what is in a picture, but they struggle badly when asked what that picture implies. Humans clear the bar at 93.1 percent. The best AI system tested, GPT 6 Astra, manages 53.6 percent. That is not a rounding error. It is a different kind of capability.

What the HSS Benchmark Actually Measures

HSS is designed to test reasoning about information a visual scene implies but never explicitly shows. The dataset comprises 522 questions spanning 288 images and 234 short video clips, totaling 17.6 hours of footage across those clips. The questions are not about object detection or captioning. They probe inference.

Two examples from the benchmark illustrate the flavor. Given a photograph of a bookcase, a model must judge whether it can fit more books. Given an image of horses pulling a plow, it must determine whether the chains bear the plow’s drag weight. Neither question can be answered by pattern-matching on surface features. Both require a model to reconstruct physical and causal structure that the image only hints at.

The benchmark organizes its questions into four categories:

• Temporal and causal dynamics: What happened before the scene, and what happens next.
• Physical and spatial logic: What fits, what is in reach, and what someone else can see from where they are standing.
• Social understanding: Intent, attention, and belonging.
• Abstract and contextual inference: The patterns and rules a scene implies.

Each category targets a distinct failure mode. Together they form a stress test for the kind of common-sense reasoning that humans apply without conscious effort.

The Scoreboard: A Wide and Uncomfortable Gap

The human baseline on HSS sits at 93.1 pass@1 overall, with image performance at 94.1 and video at 91.9. Broken down by category, humans score 97 on temporal questions and 94.9 on spatial ones.

GPT 6 Astra, the strongest AI system evaluated, reaches 53.6 pass@1 overall. Its image score is 55.6, video 51.1, temporal 58.2, and spatial 53.6. That is roughly 40 points below the human baseline across every dimension.

The open-weight models fare considerably worse. GLM-5.3-flash, from Zhipu AI, scores 26.8 pass@1, and Kimi K3, from Moonshot AI, reaches 25.5. These are not marginal systems, yet they perform at roughly half the level of the already-struggling frontier model. The comparison that matters here is open-weight versus closed frontier, not where the developers are headquartered.

The consistency of the gap matters. This is not a case where one model has a bad day on one category. Every system tested fails across the board. The deltas between human and model performance are comparable across categories, in the high 30s to low 40s, which points to a broad capability shortfall rather than a single weak spot.

Why Models Fail: Missing Details and Misidentified Roles

The benchmark’s error analysis identifies two dominant failure modes. The most common single mistake, accounting for 21 percent of failures, is missing the one detail that decides the answer. A model might correctly parse most of a scene but overlook the specific element that tips the inference one way or another.

The second most common error, at 20 percent, is misidentifying an object, person, or role. This is a more fundamental failure: the model does not correctly establish what is in the scene before attempting to reason about it.

Together, these two categories explain over 40 percent of failures. The pattern suggests that current vision-language models are not simply bad at reasoning. They are often reasoning from a flawed or incomplete representation of the scene itself. The pipeline from perception to inference leaks at multiple points.

The World-Model Debate Gets Complicated

One of the most consequential claims in the HSS release is interpretive. The benchmark’s authors argue that HSS undercuts the assumption that true visual reasoning requires spatially aware 2D or 3D world models. The reasoning is straightforward: if frontier LLMs, which are not explicitly built around spatial world models, can reach 53.6 percent on tasks that demand physical and spatial inference, then the world-model hypothesis may be overstated.

This connects to a broader conversation about what world models are and where they help. In biology, for instance, the term carries a specific meaning: a system that can represent the state of a biological system, predict how that state evolves in response to interventions, and connect models to experimental data. Work like Microsoft Research’s Quine: A World Model for Biology That Connects Models, Labs, and Literature shows how domain-specific world models can be rigorously defined and evaluated.

HSS suggests that in vision, the relationship between world models and reasoning is less direct than some have assumed. A model can perform meaningful spatial inference without an explicit 3D representation, and adding such a representation may not close the gap. The benchmark does not settle the debate, but it reframes it. The question is no longer whether world models are necessary, but what specific architectural or training changes would move the score from 53.6 to something closer to 93.

What HSS Reveals About the Limits of Vision-Language Models

HSS shows that vision-language models, despite impressive gains on captioning, VQA, and multimodal benchmarks, still fail at the kind of inference that a human child performs effortlessly. The 93.1 to 53.6 gap is not a reason to dismiss current systems. It is a reason to be precise about what they can and cannot do. A model that scores 53.6 on HSS can still drive a rover, inspect a solar array, or coordinate a docking maneuver. It cannot reliably tell you whether a bookcase has room for one more book.

That distinction matters as AI moves into domains where the cost of a wrong inference is measured in hardware, not just in benchmark points. The world-model debate will continue, and HSS will be one of the instruments used to settle it. The more useful reading of the benchmark is diagnostic rather than declarative. The failures cluster at the perception stage, before reasoning even begins, which suggests the next round of progress may come less from bigger reasoning models than from better scene representation. A model that cannot correctly identify who is holding the rope will not benefit from a longer chain of thought about what the rope is for.