Visual Reasoning Benchmark
The Reasoning task asks each model questions that require thinking beyond simply reading or spotting something. It covers arithmetic, comparisons, spatial relationships, logical deduction, and multi-step inference that combines several visual facts. The answer is never written directly in the image; the model must interpret the scene and reason about it.
Evals updated July 10, 2026Pricing updated July 17, 2026
| 1 | 91.3% | 1.9K | $0.011 | 9.0s | ||
| 2 | 87.0% | 1.7K | $0.0064 | 3.4s | ||
| 2 | 87.0% | 1.6K | $0.0026 | 3.6s | ||
| 2 | 87.0% | 1.5K | $0.017 | 5.7s | ||
| 5 | 82.6% | 1.5K | $0.0053 | 3.4s | ||
| 5 | 82.6% | 1.5K | $0.012 | 5.8s | ||
| 7 | 78.3% | 1.5K | $0.0017 | 3.6s | ||
| 7 | 78.3% | 1.5K | $0.0016 | 4.5s | ||
| 9 | 73.9% | 1.5K | $0.010 | 4.1s | ||
| 10 | 60.9% | 1.5K | $0.0077 | 2.2s | ||
| 10 | 60.9% | 405 | $0.0013 | 2.9s | ||
| 12 | 56.5% | 1.5K | $0.0031 | 2.4s | ||
| 13 | Qwen 3.7 Plus | 43.5% | 1.1K | $0.0004 | 2.0s | |
| 13 | 43.5% | 1.1K | $0.0002 | 2.9s | ||
| 13 | GLM 5V Turbo | 43.5% | 1.4K | $0.0017 | 5.0s | |
| 16 | Kimi K2.6 | 30.4% | 1.7K | $0.0018 | 7.8s |
Score vs. cost
Reasoning score (Accuracy) against estimated cost per sample. Upper-left is the sweet spot: high quality at low cost.
16 models on the current benchmark · Reasoning task only
Example Reasoning benchmark tasks
Real samples from the benchmark: the image each model sees, the question it is asked, and the ground-truth answer it is scored against.
The models are asked
Are there any abnormal or defective candies visible in the image? Answer strictly with one lowercase word (e.g., yes).
Ground truth
no
The models are asked
Based on the dosage per pill printed on the left pack and the total number of pills visible on the right pack, what is the total milligrams of medication in one full blister pack? Output the final amount as a single integer representing the total milligrams.
Ground truth
30
The models are asked
If the truck bed's grid was completely full without any empty spaces, what is the total number of gas cylinders it could hold? Output the final count as a single integer.
Ground truth
24
The models are asked
What is the difference between the original total number of events printed in the third paragraph and the handwritten revised number written directly above it? Output the final difference as a single integer (e.g., 5).
Ground truth
20
How Reasoning is scored
Each answer is graded against the ground truth. The leaderboard score is the percentage of samples answered correctly.
Every model runs the same 23 samples in a single evaluation pass. Token usage is measured from each provider’s API response, and cost per sample is that usage multiplied by the model’s published pricing. See the full methodology.
Frequently Asked Questions
Each model answers the same reasoning prompts. Answers are graded against the ground truth, and the score is the percentage answered correctly.
The answer is never written directly in the image. The model has to combine several visual facts, such as reading two values and comparing them, judging spatial relationships, or applying logic to what it sees.
Most models in this leaderboard link to their Playground page. Click the model name to open it, then upload your own image and run it. A few models are benchmarked for comparison only and do not have a Playground page yet.