Object Counting Benchmark
The Counting task asks each model to report exactly how many of a specified thing appear in an image. The model must find every matching object, cope with clutter, overlap, and partial occlusion, avoid double-counting, and return a single number. Targets range from people and products to pills, checkmarks, and even empty slots.
Evals updated July 10, 2026Pricing updated July 17, 2026
| 1 | 81.1% | 1.8K | $0.0075 | 3.9s | ||
| 2 | 73.0% | 1.6K | $0.013 | 5.8s | ||
| 3 | 71.6% | 1.5K | $0.0071 | 6.2s | ||
| 4 | 67.6% | 1.3K | $0.0012 | 2.9s | ||
| 4 | 67.6% | 1.6K | $0.0075 | 5.2s | ||
| 6 | 66.2% | 1.7K | $0.0033 | 4.2s | ||
| 7 | 64.9% | 1.6K | $0.015 | 10.0s | ||
| 8 | 63.5% | 1.5K | $0.017 | 6.0s | ||
| 9 | 60.8% | 1.6K | $0.0019 | 4.0s | ||
| 10 | 56.8% | 1.5K | $0.0030 | 3.0s | ||
| 11 | 52.7% | 1.5K | $0.0076 | 2.3s | ||
| 11 | 52.7% | 390 | $0.0012 | 3.1s | ||
| 13 | Qwen 3.7 Plus | 50.0% | 1.1K | $0.0004 | 2.3s | |
| 13 | Kimi K2.6 | 50.0% | 1.7K | $0.0019 | 10.9s | |
| 15 | GLM 5V Turbo | 48.6% | 1.4K | $0.0017 | 4.3s | |
| 16 | 47.3% | 1.1K | $0.0002 | 3.9s |
Score vs. cost
Counting score (Accuracy) against estimated cost per sample. Upper-left is the sweet spot: high quality at low cost.
16 models on the current benchmark · Counting task only
Example Counting benchmark tasks
Real samples from the benchmark: the image each model sees, the question it is asked, and the ground-truth answer it is scored against.
The models are asked
How many workers are standing or kneeling directly on the rebar grid inside the excavated pit? Output the final count as a single integer (e.g., 3).
Ground truth
8
The models are asked
How many intact capsules remain in the blister pack across all rows? Output the final count as a single integer (e.g., 15).
Ground truth
22
The models are asked
How many individual screws are visibly resting inside the plastic bin labeled "DIN 912 M8 x 30"? Output the final count as a single integer.
Ground truth
6
The models are asked
How many M&M's packages are visible in the machine? Output the final count as a single integer (e.g., 3).
Ground truth
22
The models are asked
How many shrink-wrapped packs of coke bottles with blue caps are visible on the pallet? Output the final count as a single integer (e.g., 3).
Ground truth
4
How Counting is scored
A count is correct only when it exactly matches the ground truth. The leaderboard score is the percentage of samples counted correctly.
Every model runs the same 74 samples in a single evaluation pass. Token usage is measured from each provider’s API response, and cost per sample is that usage multiplied by the model’s published pricing. See the full methodology.
Frequently Asked Questions
Each model answers the same counting prompts and returns a single number per image. A count is correct only when it exactly matches the ground truth, and the score is the percentage of exact matches.
Counting requires finding every instance of the target, including partially hidden or overlapping ones, without double-counting. Cluttered scenes, small objects, and look-alike distractors all cause miscounts, which makes this one of the most failure-prone tasks for current models.
Most models in this leaderboard link to their Playground page. Click the model name to open it, then upload your own image and run it. A few models are benchmarked for comparison only and do not have a Playground page yet.