Visual Reasoning Benchmark

Each model answers the same reasoning prompts. An LLM judge grades every answer against the ground truth, and the score is the percentage graded correct. The strict-match column shows the percentage that matched the ground truth exactly, which penalizes models that explain their answer.

Strict accuracy counts an answer only when it matches the ground truth exactly after normalization. LLM-judged accuracy asks a second model acting as judge, Gemini 3.5 Flash at temperature zero, to compare the answer with the ground truth and decide whether it is correct, so a right answer still counts when it arrives with an explanation or in different words. The judge is the headline score because it measures whether the model got it right rather than whether it was terse; the strict score sits beside it on each task page, and the gap between the two shows how verbose a model is.

Every task is run at two levels of reasoning effort. The low row is the model answering with minimal deliberation, the high row is the same model on the same questions allowed to think longer. Both rows are ranked together so you can see whether the extra thinking is worth its cost and latency, which the Est. cost and Speed columns show, and the effort filter above the table shows one tier at a time. Models benchmarked before the current protocol have a high-effort row for Reasoning only until they are re-run. Every other table on this site, including the overall Average, uses the low-effort pass so all models are compared on the same amount of compute.

Low and high are benchmark tiers mapped to each model's native reasoning mechanism, because no two labs expose the same control. Claude, GPT, and Grok take low or high as a reasoning effort parameter. Gemini runs at its low and high thinking levels (a minimal versus large thinking budget for the older Gemini 2.5 Pro). Qwen, GLM, and Kimi only expose an on/off thinking mode, so low runs them with thinking disabled and high with thinking at high effort. Muse and Qwen3.8 Max cannot switch reasoning off, so their low pass uses the lowest native setting. All five tasks and the Average use the low pass.

Under the current protocol each task is run three times at each effort tier, and the score shown is the mean over those runs. The ± is half the range between the lowest and highest run, so it shows how much the score moved between runs; it is a range, not a confidence interval. Hover the ± for the run count and range. A score without one comes from a single run; models benchmarked before the protocol are being re-run under it. OCR results are currently a single preliminary run per model, so they carry no ±.

The answer is never written directly in the image. The model has to combine several visual facts, such as reading two values and comparing them, judging spatial relationships, or applying logic to what it sees.

Most models in this leaderboard link to their Playground page. Click the model name to open it, then upload your own image and run it. A few models are benchmarked for comparison only and do not have a Playground page yet.