LLaVA-1.5 vs Qwen-VL

LLaVA-1.5 is an open-source large multimodal model released in October 2023 by researchers at the University of Wisconsin-Madison and Microsoft Research. It builds on the original LLaVA architecture by introducing targeted refinements: switching the vision encoder to CLIP-ViT-L at 336-pixel resolution, replacing the projection layer with a two-layer MLP, and adding academic-task-oriented visual question answering data with response formatting prompts during training. These modifications achieve state-of-the-art performance across 11 benchmarks at release, with training completing in approximately one day on a single 8-A100 node.

The model accepts an image paired with a text prompt and generates natural language responses, supporting visual question answering, image captioning, and open-ended visual conversation. LLaVA-1.5 is available in 7B and 13B parameter variants built on the Vicuna language model, and is distributed under the Llama 2 Community License due to its Llama-2-based foundation. The original LLaVA paper was presented as an oral at NeurIPS 2023. Subsequent releases in the series (LLaVA-NeXT (LLaVA-1.6), LLaVA-NeXT-Video, and LLaVA-OneVision) are separate models with their own release pages and build on this foundation with expanded OCR, video, and multi-image capabilities.

Qwen-VL is a large vision-language model released in August 2023 by the Qwen team at Alibaba Cloud. Built on the 7-billion-parameter Qwen language model with an added visual receptor based on Openclip ViT-bigG, the model accepts images, text, and bounding box coordinates as inputs, and can produce both text and bounding boxes as outputs. Qwen-VL processes images at 448×448 resolution, higher than the 224×224 input used by many contemporaneous vision-language models, which supports finer-grained visual recognition and text-heavy tasks such as OCR. This design supports a range of multimodal tasks in a single model, including image captioning, visual question answering, visual grounding, text recognition, and image-conditioned dialogue, with native support for English, Chinese, and multilingual conversation.

At release, Qwen-VL achieved competitive results against contemporaneous vision-language models across zero-shot captioning, general VQA, text-oriented VQA, and referring expression comprehension benchmarks. A chat-tuned variant, Qwen-VL-Chat, is optimized for interactive use with instruction-following and multi-turn conversation. The model is distributed under the Tongyi Qianwen License, a custom license from Alibaba Cloud with specific terms that should be reviewed prior to commercial use. Qwen-VL is the first generation of Alibaba's open multimodal series and precedes the later Qwen2-VL and Qwen2.5-VL releases.