Best LLMs with Vision Capabilities

LLMs with vision capabilities (also called vision language models or VLMs) are large language models extended with an image encoder, so they accept images as input alongside text. The encoder converts the image into token embeddings the language model reads the same way it reads words, which lets one model describe scenes, answer questions about images, read and transcribe text, and return structured output like JSON from a visual input. They inherit the strengths and weaknesses of their language side: broad world knowledge and instruction following, but also a tendency to state wrong visual details fluently, and looser spatial precision than dedicated detectors. This page lists 80 vision LLMs, including 31 open-weight options you can self-host; 75 of them run live in the Playground so you can test them on your own images.

Every vision LLM is multimodal, but not every multimodal model is an LLM. Multimodal is the broader property of handling more than one input type. CLIP is multimodal because it aligns images and text in one embedding space, yet it generates no language at all. A vision LLM is specifically a language model that can see: multimodal input, conversational text output.

On our vision benchmark across six tasks (53 models tested, updated Sep 5, 2026), GPT-6 Astra by OpenAI currently scores highest at 86.6%. The full ranking is on our evals page. For fixed categories in production, a model fine-tuned on your own data still often wins.

Yes. 31 of the 80 models here are open-weight (for example GLM 5.3 Flash, Qwen3.8 27B, and Muse Glimmer 30B), free to self-host under their licenses (MIT, Apache 2.0, and Modified MIT).

Yes. You can run 75 of them in the Roboflow Playground for free. Upload an image and compare the models' output side by side, no setup required.