Object Detection Benchmark
The Object Detection task asks each model to locate and classify objects by returning a bounding box and a class label for every one, in the requested output format. Unlike counting or identification, the model must report both what each object is and precisely where it sits in the image. Each prompt restricts the model to a fixed list of class labels.
Evals updated July 10, 2026Pricing updated July 17, 2026
| 1 | 61.7% | 37.9% | 37.6% | 2.1K | $0.0096 | 5.9s | ||
| 2 | 56.9% | 35.3% | 33.8% | 1.5K | $0.0065 | 5.7s | ||
| 3 | Qwen 3.7 Plus | 49.2% | 31.0% | 29.2% | 1.7K | $0.0009 | 8.2s | |
| 4 | GLM 5V Turbo | 48.2% | 25.4% | 25.7% | 2.3K | $0.0035 | 8.6s | |
| 5 | 46.2% | 20.9% | 23.3% | 3.2K | $0.034 | 14.5s | ||
| 6 | 44.7% | 14.6% | 19.4% | 3.0K | $0.014 | 7.4s | ||
| 7 | 43.3% | 16.7% | 20.2% | 3.1K | $0.0061 | 6.8s | ||
| 8 | 42.3% | 24.8% | 24.6% | 2.2K | $0.0011 | 12.3s | ||
| 9 | 40.6% | 10.3% | 15.7% | 2.4K | $0.040 | 10.6s | ||
| 10 | 39.3% | 20.1% | 21.6% | 1.7K | $0.0022 | 5.1s | ||
| 11 | Kimi K2.6 | 32.2% | 16.6% | 17.3% | 2.2K | $0.0025 | 10.9s | |
| 12 | 26.4% | 13.6% | 14.0% | 874 | $0.0055 | 6.2s | ||
| 13 | 18.6% | 2.8% | 6.3% | 2.3K | $0.017 | 5.2s | ||
| 14 | 18.0% | 4.4% | 6.7% | 2.3K | $0.0071 | 4.7s | ||
| 15 | 13.8% | 2.4% | 4.8% | 2.9K | $0.029 | 14.1s | ||
| 16 | 3.9% | 0.4% | 1.1% | 2.0K | $0.0032 | 5.3s |
Score vs. cost
Object Detection score (mAP@50) against estimated cost per sample. Upper-left is the sweet spot: high quality at low cost.
16 models on the current benchmark · Object Detection task only
Example Object Detection benchmark tasks
Real samples from the benchmark: the image each model sees, the question it is asked, and the ground-truth answer it is scored against.
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: ball in basket, basket rim, basketball, jersey number, player, referee
Ground truth
23 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: car, pedestrian, truck
Ground truth
5 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: minor scratch
Ground truth
1 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: bone fracture
Ground truth
3 objects
The models are asked
Detect all objects in this image. Output a JSON list where each entry contains the 2D bounding box in the key "box_2d" and the text label in the key "label". The "box_2d" value must be [y_min, x_min, y_max, x_max]: integers between 0 and 1000, normalized to the image height and width. Return only the JSON list, with no extra text. Only use these labels: dc current source, dc voltage source, ground, resistor
Ground truth
6 objects
How Object Detection is scored
Quality is scored by how well the predicted boxes overlap the ground truth, using mean Average Precision. The headline metric is mAP@50 (a prediction counts when its box overlaps the truth by at least 50%), with stricter mAP@75 and mAP@50:95 also reported.
Every model runs the same 250 samples in a single evaluation pass. Token usage is measured from each provider’s API response, and cost per sample is that usage multiplied by the model’s published pricing. See the full methodology.
Frequently Asked Questions
Mean Average Precision measures how well predicted boxes match ground-truth boxes. The number is the overlap (IoU) threshold a prediction must clear to count: mAP@50 requires 50% overlap, mAP@75 requires 75%, and mAP@50:95 averages across thresholds from 50% to 95%. Higher thresholds demand more precise boxes, so scores drop as the threshold rises.
These are general-purpose vision language models prompted to output boxes, with no task-specific training. Dedicated detectors trained on a specific dataset still score far higher on it. The value here is zero-shot flexibility: the same model can detect arbitrary classes described in plain text.
Most models in this leaderboard link to their Playground page. Click the model name to open it, then upload your own image and run it. A few models are benchmarked for comparison only and do not have a Playground page yet.