Object Detection Benchmark
The Object Detection task asks each model to locate and classify objects by returning a bounding box and a class label for every one, in the requested output format. Unlike counting or identification, the model must report both what each object is and precisely where it sits in the image. Each prompt restricts the model to a fixed list of class labels.
Evals updated August 14, 2026Pricing updated August 18, 2026
| 1 | 77.1% | 61.4% | 60.3% | 3.2K | $0.013 | 30.49s | ||
| 2 | 69.4% | 59.8% | 56.2% | 2.2K | $0.0024 | 8.68s | ||
| 3 | 68.7% | 55.8% | 53.0% | 2.8K | $0.016 | 8.17s | ||
| 4 | 68.2% | 46.6% | 45.2% | 3.2K | $0.022 | 21.44s | ||
| 5 | 67.4% | 55.8% | 52.4% | 1.9K | $0.010 | 8.76s | ||
| 6 | 60.7% | 40.4% | 39.6% | 2.9K | $0.014 | 9.86s | ||
| 7 | 60.2% | 43.5% | 40.7% | 3.8K | $0.0094 | 8.52s | ||
| 8 | Qwen 3.7 Plus | 60.1% | 48.4% | 45.1% | 2.2K | $0.0013 | 13.21s | |
| 9 | 59.9% | 41.4% | 41.1% | 3.0K | $0.0015 | 10.29s | ||
| 10 | 58.8% | 46.4% | 45.1% | 2.6K | $0.0013 | 14.71s | ||
| 11 | 58.4% | 40.7% | 38.6% | 4.1K | $0.010 | 16.39s | ||
| 12 | 57.5% | 46.6% | 44.5% | 2.0K | $0.0023 | 4.09s | ||
| 13 | GLM 5V Turbo | 56.5% | 39.9% | 39.2% | 3.3K | $0.0052 | 8.59s | |
| 14 | 56.4% | 31.0% | 31.4% | 3.2K | $0.059 | 12.78s | ||
| 15 | 56.0% | 46.4% | 44.5% | 2.1K | $0.0041 | 6.33s | ||
| 16 | Qwen3.8 27BNEW | 54.5% | 43.2% | 41.2% | 3.8K | $0.0036 | 15.64s | |
| 17 | 54.4% | 30.2% | 30.5% | 3.1K | $0.027 | 11.20s | ||
| 18 | 52.1% | 42.9% | 40.3% | 3.1K | $0.0014 | 18.26s | ||
| 19 | 51.9% | 41.5% | 40.4% | 4.3K | $0.020 | 23.23s | ||
| 20 | 42.8% | 32.9% | 31.1% | 2.2K | $0.0001 | 9.54s | ||
| 21 | 41.7% | 21.1% | 22.2% | 2.9K | $0.034 | 13.17s | ||
| 22 | 41.0% | 22.3% | 23.0% | 3.1K | $0.0020 | 15.03s | ||
| 23 | 38.6% | 29.2% | 27.9% | 2.0K | $0.0031 | 5.86s | ||
| 24 | 38.6% | 13.3% | 16.8% | 3.0K | $0.026 | 8.34s | ||
| 25 | 36.1% | 15.8% | 18.1% | 3.1K | $0.011 | 7.19s | ||
| 26 | Kimi K2.6 | 35.7% | 26.7% | 26.8% | 3.5K | $0.0059 | 64.70s | |
| 27 | 33.7% | 25.1% | 24.7% | 1.4K | $0.010 | 10.06s | ||
| 28 | Grok 4.6NEW | 20.2% | 4.6% | 8.0% | 2.3K | $0.0068 | 6.47s | |
| 29 | 18.0% | 4.6% | 7.5% | 3.0K | $0.0100 | 18.73s | ||
| 30 | 16.1% | 3.5% | 5.9% | 2.5K | $0.0044 | 7.69s |
Score vs. cost
Object Detection score (mAP@50) against estimated cost per sample. Upper-left is the sweet spot: high quality at low cost.
30 models on the current benchmark · Object Detection task only
Example Object Detection benchmark tasks
Real samples from the benchmark: the image each model sees, the question it is asked, and the ground-truth answer it is scored against.
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: ball in basket, basket rim, basketball, jersey number, player, player doing layup dunk, referee
Ground truth
19 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: solar panel hot spot
Ground truth
15 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: banana tree
Ground truth
43 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: soda bottle cap
Ground truth
24 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: pill
Ground truth
50 objects
How Object Detection is scored
Quality is scored by how well the predicted boxes overlap the ground truth, using mean Average Precision. The headline metric is mAP@50 (a prediction counts when its box overlaps the truth by at least 50%), with stricter mAP@75 and mAP@50:95 also reported.
Every model runs the same sample set in a single evaluation pass. Token usage is measured from each provider’s API response, and cost per sample is that usage multiplied by the model’s published pricing. See the full methodology.
Frequently Asked Questions
Mean Average Precision measures how well predicted boxes match ground-truth boxes. The number is the overlap (IoU) threshold a prediction must clear to count: mAP@50 requires 50% overlap, mAP@75 requires 75%, and mAP@50:95 averages across thresholds from 50% to 95%. Higher thresholds demand more precise boxes, so scores drop as the threshold rises.
These are general-purpose vision language models prompted to output boxes, with no task-specific training. Dedicated detectors trained on a specific dataset still score far higher on it. The value here is zero-shot flexibility: the same model can detect arbitrary classes described in plain text.
Most models in this leaderboard link to their Playground page. Click the model name to open it, then upload your own image and run it. A few models are benchmarked for comparison only and do not have a Playground page yet.