Vision Evals
Gemini 4 Carbon leads Roboflow's Vision Evals at 89.4% averaged across five tasks, ahead of Gemini 4 Argon (87.5%) and Claude Fable 5.2 (86.5%). It ranks first on counting, OCR and reasoning. The best open-weight model is Qwen3.8 27B at 69.5%, scored on 4 of the 5 tasks.
Evals updated November 10, 2026Pricing updated November 11, 2026
Overall ranking
| 1 | 89.4% | 1.6K | $0.0061 | 8.71s | ||
| 2 | 87.5% | 1.6K | $0.0058 | 13.40s | ||
| 3 | 86.5% | 2.5K | $0.031 | 9.33s | ||
| 4 | 85.8% | 2.1K | $0.0033 | 10.05s | ||
| 5 | 83.4% | 2.1K | $0.034 | 9.94s | ||
| 6 | 82.0% | 2.5K | $0.015 | 7.63s | ||
| 7 | 81.8% | 2.4K | $0.012 | 11.52s | ||
| 8 | 81.7% | 2.1K | $0.0068 | 12.82s | ||
| 9 | 80.1% | 1.8K | $0.0033 | 11.04s | ||
| 10 | 79.8% | 2.1K | $0.0045 | 12.76s | ||
| 11 | 79.3% | 2.3K | $0.0080 | 18.95s | ||
| 12 | 78.8% | 3.3K | $0.034 | 11.68s | ||
| 13 | 77.6% | 2.5K | $0.0073 | 8.33s | ||
| 14 | 77.1% | 1.8K | $0.0033 | 10.43s | ||
| 15 | 76.9% | 2.2K | $0.0074 | 10.20s | ||
| 16 | 76.8% | 1.8K | $0.010 | 8.44s | ||
| 17 | Muse EmberNEW | 76.6% | 3.4K | $0.0009 | 6.08s | |
| 18 | 74.7% | 2.5K | $0.039 | 8.35s | ||
| 19 | 74.4% | 3.2K | $0.0075 | 7.15s | ||
| 20 | 73.9% | 3.4K | $0.0084 | 25.50s | ||
| 21 | 73.8% | 3.4K | $0.0083 | 11.51s | ||
| 22 | 72.4% | 2.4K | $0.019 | 12.16s | ||
| 23 | 71.7% | 2.6K | $0.039 | 9.54s | ||
| 24 | 70.9% | 2.6K | $0.020 | 8.21s | ||
| 25 | 69.7% | 2.3K | $0.0004 | 10.21s | ||
| 26 | 69.6% | 2.9K | $0.0006 | 11.01s | ||
| 27 | 69.5%* | 2.8K | ~$0.0009 | 17.99s | ||
| 28 | 69.4% | 2.9K | $0.0059 | 7.37s | ||
| 29 | 67.1%* | 4.1K | ~$0.0021 | 42.09s | ||
| 30 | 66.0% | 2.4K | $0.025 | 10.10s | ||
| 31 | 66.0% | 2.4K | $0.010 | 9.52s | ||
| 32 | 65.7% | 1.8K | $0.0024 | 5.83s | ||
| 33 | 64.8%* | 4.6K | ~$0.0012 | 27.10s | ||
| 34 | 64.7% | 2.4K | $0.0011 | 8.60s | ||
| 35 | 64.1%* | 5.0K | ~$0.0043 | 80.37s | ||
| 36 | 63.5% | 5.1K | $0.020 | 33.31s | ||
| 37 | 62.5%* | 5.6K | ~$0.0016 | 31.88s | ||
| 38 | 61.7% | 1.6K | $0.0014 | 3.92s | ||
| 39 | 61.4% | 2.6K | $0.0013 | 10.60s | ||
| 40 | 59.7% | 1.8K | $0.0004 | 7.34s | ||
| 41 | 59.1% | 3.1K | $0.011 | 19.32s | ||
| 42 | 58.9% | 1.7K | $0.0008 | 7.77s | ||
| 43 | 58.3% | 2.6K | $0.021 | 7.66s | ||
| 44 | 57.7%* | 1.9K | ~$0.0015 | 34.36s | ||
| 45 | 57.6% | 848 | $0.0051 | 6.83s | ||
| 46 | 56.8% | 2.6K | $0.0033 | 31.23s | ||
| 47 | 56.0%* | 6.3K | ~$0.0021 | 41.36s | ||
| 48 | 56.0% | 3.7K | $0.0052 | 9.86s | ||
| 49 | 55.8% | 3.1K | $0.0001 | 9.29s | ||
| 50 | 55.3%* | 2.2K | — | 38.37s | ||
| 51 | 55.3% | 2.5K | $0.0078 | 16.41s | ||
| 52 | 55.3% | 2.5K | $0.0074 | 5.86s | ||
| 53 | 55.2% | 2.5K | $0.0011 | 15.87s | ||
| 54 | 54.6% | 2.6K | $0.0037 | 5.89s | ||
| 55 | 54.1%* | 3.2K | ~$0.0019 | 27.84s | ||
| 56 | 53.2% | 2.2K | $0.0034 | 5.51s | ||
| 57 | Kimi K2.6 | 52.8% | 2.5K | $0.0031 | 21.67s | |
| 58 | 52.4% | 1.7K | $0.0001 | 5.77s | ||
| 59 | 50.3% | 2.1K | $0.0011 | 9.45s | ||
| 60 | 49.2% | 2.1K | $0.0003 | 16.34s | ||
| 61 | 48.8%* | 2.9K | ~$0.0005 | 11.49s | ||
| 62 | 48.3%* | 6.8K | ~$0.0018 | 34.78s | ||
| 63 | DeepSeek V4 Flash Vision Exp | 47.2% | 3.1K | $0.0036 | 18.20s | |
| 64 | 37.8%* | 7.7K | ~$0.0014 | 30.60s | ||
| 65 | 33.4%* | 1.7K | ~$0.0004 | 8.89s | ||
| 66 | 33.2%* | 955 | ~$0.0002 | 4.85s | ||
| 67 | 29.9%* | 969 | ~$0.0002 | 3.60s | ||
| 68 | 28.3%* | 1.6K | ~<$0.0001 | 1.17s |
Score vs. cost
Overall benchmark score against estimated cost per sample, on a log scale. Upper-left is the sweet spot: high quality at low cost.
67 of 68 models plotted (the rest have no measured figure for this axis) · scores and efficiency pooled across all five tasks at low effort
Results by task
Object Detection
Locate and classify objects with a bounding box and label for each one.
- 1GPT-6.1 Terra84.7%
- 2Gemini 4 Carbon84.5%
- 3Gemini 4 Argon82.5%
Counting
Report exactly how many of a specified thing appear in an image.
- 1Gemini 4 Carbon90.1%
- 2Claude Fable 5.286.5%
- 3Gemini 4 Argon86.0%
Identification
Recognize and name a specific entity in an image from visual evidence.
- 1Claude Fable 5100.0%
- 1Gemini 3.1 Pro100.0%
- 3Gemini 3.5 Flash99.0%
OCR
Read text from images: single values, full transcriptions, structured JSON, and located text.
- 1Gemini 4 Carbon87.2%
- 2Gemini 4 Argon85.7%
- 3GPT-6.1 Terra81.9%
Reasoning
Answer questions that require thinking beyond reading or spotting something.
- 1Gemini 4 Carbon91.4%
- 2Gemini 4 Argon89.4%
- 3Claude Fable 5.288.1%
Every model’s score on every task
| Model | Object Detection | Counting | Identification | OCR | Reasoning |
|---|---|---|---|---|---|
| 84.5 | 90.1 | 93.8 | 87.2 | 91.4 | |
| 82.5 | 86.0 | 93.8 | 85.7 | 89.4 | |
| 81.6 | 86.5 | 96.9 | 79.5 | 88.1 | |
| 84.7 | 80.2 | 96.9 | 81.9 | 85.4 | |
| 82.1 | 80.2 | 89.6 | 77.8 | 87.2 | |
| 74.5 | 80.6 | 93.8 | 78.3 | 83.0 | |
| 70.6 | 80.6 | 99.0 | 76.7 | 82.1 | |
| 80.8 | 78.8 | 89.6 | 75.8 | 83.7 | |
| 70.5 | 78.4 | 96.9 | 73.9 | 80.8 | |
| 68.1 | 78.8 | 97.9 | 72.8 | 81.2 | |
| 76.7 | 81.1 | 88.5 | 74.0 | 75.9 | |
| 65.1 | 76.6 | 92.7 | 74.3 | 85.2 | |
| 74.3 | 79.3 | 91.7 | 66.3 | 76.4 | |
| 57.1 | 80.2 | 99.0 | 71.8 | 77.7 | |
| 74.7 | 76.6 | 93.8 | 66.7 | 72.6 | |
| 67.4 | 71.6 | 100.0 | 72.6 | 72.2 | |
Muse EmberNEW | 63.8 | 79.7 | 91.7 | 69.5 | 78.2 |
| 61.4 | 69.4 | 97.9 | 72.9 | 72.0 | |
| 60.6 | 76.6 | 91.7 | 68.8 | 74.2 | |
| 58.6 | 74.3 | 92.7 | 70.4 | 73.3 | |
| 59.0 | 76.6 | 89.6 | 68.7 | 75.1 | |
| 68.4 | 74.3 | 89.6 | 63.6 | 66.0 | |
| 56.4 | 63.5 | 100.0 | 72.3 | 66.2 | |
| 54.4 | 70.3 | 90.6 | 68.0 | 71.5 | |
| 65.5 | 71.6 | 87.5 | 59.8 | 64.2 | |
| 65.8 | 68.9 | 83.3 | 61.2 | 68.9 | |
| 65.7 | 64.9 | 85.4 | – | 62.0 | |
| 57.8 | 75.7 | 82.3 | 67.6 | 63.8 | |
| 59.7 | 67.1 | 82.3 | – | 59.2 | |
| 43.6 | 68.0 | 89.6 | 58.4 | 70.6 | |
| 60.6 | 65.8 | 86.5 | 56.3 | 60.9 | |
| 38.6 | 67.6 | 93.8 | 63.9 | 64.9 | |
| 57.0 | 65.3 | 82.3 | – | 54.8 | |
| 61.0 | 67.1 | 83.3 | 51.8 | 60.5 | |
| 50.5 | 67.6 | 80.2 | – | 58.1 | |
| 40.4 | 61.7 | 87.5 | 63.6 | 64.2 | |
| 52.9 | 62.6 | 80.2 | – | 54.1 | |
| 57.5 | 52.7 | 84.4 | 65.8 | 48.3 | |
| 41.0 | 66.2 | 81.3 | 60.7 | 57.6 | |
| 59.8 | 56.3 | 88.5 | 58.9 | 35.1 | |
| 23.8 | 65.8 | 84.4 | 60.6 | 61.1 | |
| 60.1 | 50.0 | 84.4 | 60.3 | 39.7 | |
| 38.6 | 54.0 | 84.4 | 61.5 | 53.0 | |
| 47.5 | 51.4 | 79.2 | – | 52.8 | |
| 33.7 | 52.7 | 93.8 | 65.2 | 42.4 | |
| 59.3 | 54.5 | 85.4 | 46.1 | 38.9 | |
| 38.1 | 56.8 | 83.3 | – | 45.9 | |
| 51.9 | 46.0 | 81.3 | 58.6 | 42.4 | |
| 33.1 | 55.4 | 84.4 | 55.4 | 51.0 | |
| 54.3 | 51.4 | 81.3 | – | 34.4 | |
| 19.0 | 59.5 | 83.3 | 57.1 | 57.6 | |
| 36.1 | 56.8 | 81.3 | 59.4 | 43.0 | |
| 52.1 | 47.3 | 90.6 | 56.4 | 29.8 | |
| 56.5 | 48.6 | 84.4 | 51.5 | 31.8 | |
| 44.2 | 43.2 | 81.3 | – | 47.7 | |
| 15.8 | 58.6 | 83.3 | 51.1 | 57.0 | |
Kimi K2.6 | 35.7 | 56.8 | 78.1 | 54.4 | 39.1 |
| 42.8 | 46.0 | 84.4 | 54.3 | 34.4 | |
| 42.0 | 50.0 | 76.0 | 48.6 | 35.1 | |
| 37.8 | 49.5 | 76.0 | 49.7 | 33.1 | |
| 35.0 | 42.8 | 76.0 | – | 41.5 | |
| 38.1 | 39.2 | 77.1 | – | 38.6 | |
DeepSeek V4 Flash Vision Exp | 45.3 | 46.0 | 68.8 | 44.0 | 31.8 |
| 31.5 | 28.8 | 63.5 | – | 27.1 | |
| 0.0 | 33.8 | 71.9 | – | 27.8 | |
| 23.8 | 22.1 | 54.2 | – | 32.7 | |
| 19.0 | 14.4 | 53.1 | – | 33.1 | |
| 2.5 | 26.6 | 59.4 | – | 24.7 |
Cell color follows the absolute score, on the same scale as the score key above. The column leader is bold. Hover a cell for the full breakdown.
What are Vision Evals?
Vision Evals measures what vision language models can actually do with real images. Every model receives the same samples across five tasks: Object Detection, Counting, Identification, OCR, and Reasoning. Answers are scored against ground truth, so the scores are directly comparable. No human votes, no subjective judgment.
Methodology
Every model runs the full sample set for every task. Under the current benchmark protocol each task is run at two reasoning effort tiers, low and high, and each of those configurations is run three times; the reported score is the mean over the runs, and the ± after a score is half the range across them. Models benchmarked before the protocol ran once per task at low effort, plus once at high effort for Reasoning, and are being re-run under it; until then their scores are single runs and carry no ±. Every table on this site other than a task leaderboard, including the Average column, uses the low-effort pass so all models are compared on the same amount of compute. Low and high are benchmark-wide tiers, not provider settings: each model's tier is mapped to its native reasoning mechanism, whether that is an effort parameter (Claude, GPT, Grok), a thinking level (Gemini), or a thinking mode that low disables and high enables (Qwen, GLM, Kimi).
Object Detection is scored by mean Average Precision (mAP@50 as the headline, with mAP@75 and mAP@50:95 reported on the task page) and OCR by deterministic scores against the ground truth (exact match for single values, similarity for transcriptions, field matching for JSON, and box overlap plus exact text for located text), averaged over all of its questions. Counting, Identification, and Reasoning are scored by a second model acting as judge, Gemini 3.5 Flash at temperature zero, which compares every answer with the ground truth and decides whether it is correct, so a right answer still counts when the model wraps it in explanation; a strict accuracy, counting only answers that match the ground truth exactly, is reported beside it on each task page, and the gap between the two is a measure of verbosity. The overall Average column is the unweighted mean of the five task scores. OCR results are currently a single preliminary run per model, so they carry no ±.
Token usage is measured directly from each provider’s API response, and output counts include reasoning tokens. Estimated cost per sample multiplies that measured input and output usage by the model’s published per-1M pricing at the time of our last price sync, so prices stay current without re-running the benchmark; it is an estimate on this benchmark, not a universal model cost. Speed is wall-clock inference time per sample.
Last evaluated: November 10, 2026
Looking for the previous benchmark?
The previous version of Vision Evals is preserved with its full leaderboards. It covers many more models, including deprecated ones that can no longer be re-run. See Vision Evals (legacy)
Frequently Asked Questions
Each of the five tasks produces one headline score: mAP@50 for Object Detection, the mean score over all OCR questions for OCR, and LLM-judged accuracy for the rest. The Average is the unweighted mean of those five scores, so every task counts equally. A few self-hosted models do not have OCR results yet; their Average covers the four tasks they have and is marked as such.
Strict accuracy counts an answer only when it matches the ground truth exactly after normalization. LLM-judged accuracy asks a second model acting as judge, Gemini 3.5 Flash at temperature zero, to compare the answer with the ground truth and decide whether it is correct, so a right answer still counts when it arrives with an explanation or in different words. The judge is the headline score because it measures whether the model got it right rather than whether it was terse; the strict score sits beside it on each task page, and the gap between the two shows how verbose a model is.
Under the current protocol each task is run three times at each effort tier, and the score shown is the mean over those runs. The ± is half the range between the lowest and highest run, so it shows how much the score moved between runs; it is a range, not a confidence interval. Hover the ± for the run count and range. A score without one comes from a single run; models benchmarked before the protocol are being re-run under it. OCR results are currently a single preliminary run per model, so they carry no ±.
Most models in this leaderboard link to their Playground page. Click the model name to open it, then upload your own image and run it. A few models are benchmarked for comparison only and do not have a Playground page yet.