Microsoft Vision Models

Compare all 5 Microsoft vision models we track, all of them open-weight. Try 1 of them on your own images, free in the Roboflow Playground.

5 models · 5 open-weight · 1 free to try · Updated Jun 2025

About Microsoft Vision Models

Every Microsoft vision model in our catalog is open-weight. All 5 publish downloadable weights, so you can self-host under their own licenses instead of paying per request. The open-weight side runs on MIT and Custom licenses. The newest additions are Florence-2 (Jun 2025) and LLaVA-1.5 (Oct 2023).

Between them, the Microsoft models we list cover 10 distinct vision tasks. The widest coverage is object detection (2 models), OCR (2), and captioning (1). For object detection, start with Florence-2 or Faster R-CNN.

On the open-weight tier, LLaVA-1.5 is the largest at 7B, 13B parameters and ResNet-50 the smallest at 25.6M, small enough to run on a single GPU or on-device. 4 of the 5 open Microsoft models carry a permissive license, so commercial use is straightforward; the other 1 ship under terms worth reading before you deploy.

With every model here open-weight, the real trade is size against generality: the smaller checkpoints fine-tune and deploy cheaply on your own hardware, while the larger ones cover more ground out of the box. 1 of the 5 Microsoft vision models run live in the Roboflow Playground, so you can run the same image through several of them and compare the answers before committing to one.

Which Microsoft Model Should You Use?

What each of the 5 Microsoft vision models in our catalog is built for, and how you run it.

Florence-2: Best for: Instance Segmentation and Open Vocabulary Object Detection. Open weights: MIT license, 230M parameters. Self-host it or run it here. Small enough for on-device and edge deployment. Runnable in the Playground.
LLaVA-1.5: Best for: Visual Question Answering and Vision Language. Open weights: Custom license, 7B, 13B parameters. Self-host it or run it here.
TrOCR: Best for: OCR. Open weights: MIT license, 61.4M-600M parameters. Self-host it or run it here. Small enough for on-device and edge deployment.
ResNet-50: Best for: Classification. Open weights: MIT license, 25.6M parameters. Self-host it or run it here. Small enough for on-device and edge deployment.
Faster R-CNN: Best for: Object Detection. Open weights: MIT license, 41.8M parameters. Self-host it or run it here. Small enough for on-device and edge deployment.

Open-Source Microsoft Models

5 models with downloadable weights you can self-host under their licenses (MIT and Custom). 1 run live in the Playground through hosted APIs, so self-hosting is optional.

Florence-2

Florence-2, introduced by Microsoft Research at CVPR 2024, is an open-source vision-language foundation model designed to unify diverse computer vision tasks within a single sequence-to-sequence framework. Unlike traditional models that specialize in specific tasks, Florence-2 accepts both images and text prompts and outputs text for tasks such as captioning, object detection, segmentation, OCR, and region-based grounding. It comes in two sizes—Florence-2-base (~230M parameters) and Florence-2-large (~770M parameters)—and is trained on FLD-5B, a large dataset of ~126M images with ~5.4B annotations.The model demonstrates strong zero-shot and fine-tuned performance, often rivaling larger vision-language systems while remaining lightweight and efficient. Released under the MIT license, all weights are publicly available, making it accessible for fine-tuning and deployment in applications like VQA, content tagging, accessibility, and research. Florence-2’s compact design, versatility, and openness position it as a practical alternative to larger proprietary multimodal models.

LLaVA-1.5

LLaVA-1.5 is an open-source large multimodal model released in October 2023 by researchers at the University of Wisconsin-Madison and Microsoft Research. It builds on the original LLaVA architecture by introducing targeted refinements: switching the vision encoder to CLIP-ViT-L at 336-pixel resolution, replacing the projection layer with a two-layer MLP, and adding academic-task-oriented visual question answering data with response formatting prompts during training. These modifications achieve state-of-the-art performance across 11 benchmarks at release, with training completing in approximately one day on a single 8-A100 node.The model accepts an image paired with a text prompt and generates natural language responses, supporting visual question answering, image captioning, and open-ended visual conversation. LLaVA-1.5 is available in 7B and 13B parameter variants built on the Vicuna language model, and is distributed under the Llama 2 Community License due to its Llama-2-based foundation. The original LLaVA paper was presented as an oral at NeurIPS 2023. Subsequent releases in the series (LLaVA-NeXT (LLaVA-1.6), LLaVA-NeXT-Video, and LLaVA-OneVision) are separate models with their own release pages and build on this foundation with expanded OCR, video, and multi-image capabilities.

TrOCR

TrOCR (Transformer-based Optical Character Recognition) is an end-to-end OCR model released in September 2021 by Microsoft Research. It departs from the traditional two-stage OCR pipeline — which typically combines a CNN-based feature extractor with an RNN-based sequence decoder — by using a pure Transformer architecture composed of a pretrained image Transformer encoder and a pretrained text Transformer decoder, an approach that later became standardized as the VisionEncoderDecoder pattern in Hugging Face Transformers.TrOCR takes a cropped text line image as input and produces a sequence of output tokens, supporting printed, handwritten, and scene text recognition. The model is designed for use downstream of a separate text detection stage — TrOCR recognizes text in pre-cropped regions rather than detecting text locations in a full page. Microsoft released three size variants: TrOCR-small (62M parameters, DeiT-small encoder + MiniLM decoder), TrOCR-base (334M parameters, BEiT-base encoder + RoBERTa-large decoder), and TrOCR-large (558M parameters, BEiT-large encoder + RoBERTa-large decoder). Pretrained and fine-tuned checkpoints are available for printed text (on SROIE), handwritten text (on IAM), and scene text (on the standard scene text benchmarks) under the MIT license, distributed through the Microsoft unilm repository and Hugging Face. At release, TrOCR achieved state-of-the-art results across all three benchmark categories, and the model continues to be used as a baseline for handwritten text recognition.

ResNet-50

ResNet-50 is a deep convolutional neural network architecture introduced in the 2015 paper "Deep Residual Learning for Image Recognition" by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun at Microsoft Research. It is part of the ResNet (Residual Network) family, which introduced residual connections — shortcut paths that allow gradients to bypass layers during training — solving the degradation problem that had previously limited the practical training of very deep networks. ResNet-50 specifically refers to a 50-layer variant with approximately 25.6 million parameters, structured as a sequence of bottleneck residual blocks consisting of 1×1, 3×3, and 1×1 convolutions.ResNet-50 was trained on the ImageNet classification benchmark and achieved leading top-1 accuracy at release. Beyond classification, it became a widely used backbone feature extractor for downstream tasks including object detection (as the base network in Faster R-CNN, Mask R-CNN, and RetinaNet) and semantic and instance segmentation. Most current implementations in PyTorch torchvision, TensorFlow, and NVIDIA NGC use the ResNet-50 v1.5 variant, which relocates the stride-2 downsampling from the first 1×1 convolution to the 3×3 convolution within each bottleneck block, yielding approximately 0.5% higher top-1 accuracy than the original v1 formulation at a small throughput cost. ResNet-50 remains a common reference architecture in computer vision benchmarks and a standard backbone choice in detection and segmentation frameworks. The original Microsoft Research code is released under the MIT license.

Faster R-CNN

Faster R-CNN is an object detection model introduced by Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun at Microsoft Research, published at NIPS in June 2015. It advances upon Fast R-CNN and R-CNN by introducing the Region Proposal Network (RPN), a fully convolutional network that shares features with the detection network and generates object proposals at negligible additional cost. This makes Faster R-CNN the first near-real-time deep learning object detector based on region proposals.Faster R-CNN achieves strong detection accuracy on PASCAL VOC and MS COCO at the time of release. It remains a widely referenced architecture in computer vision research and is available through Meta's Detectron2 framework as a maintained PyTorch implementation. It is most appropriate for offline or server-side inference tasks where accuracy is prioritized over latency, as its two-stage pipeline carries higher inference cost than single-stage detectors.

Frequently Asked Questions About Microsoft Vision Models

Which Microsoft models can do object detection?

2 of the 5 Microsoft vision models we track handle object detection: Florence-2 and Faster R-CNN. Each model page lists its full task coverage, license, and specs.

Which Microsoft models can do OCR?

2 of the 5 Microsoft vision models we track handle OCR: Florence-2 and TrOCR. Each model page lists its full task coverage, license, and specs.

Are Microsoft vision models open source?

Yes. All 5 Microsoft vision models we track publish downloadable weights you can self-host under their licenses (MIT and Custom).

What is the best Microsoft model for object detection?

We do not publish a Microsoft-only ranking, so pick on constraints rather than a label. 2 Microsoft models handle object detection; the most recent is Florence-2 (Jun 2025). Measured scores across every lab we test are on our object detection benchmark, linked at the bottom of this page. For fixed categories in production, a model fine-tuned on your own data typically beats any general-purpose model.

How many Microsoft vision models are on Roboflow Playground?

We track 5 live Microsoft vision models. The most recent addition is Florence-2, released Jun 2025.

Can I try Microsoft vision models for free?

Yes. 1 of the 5 Microsoft models run live in the Roboflow Playground. Upload your own image, run several models on it at once, and compare the outputs side by side. No setup and no account required.

This page lists all 5 Microsoft vision models in the Roboflow Playground catalog, all of them open-weight and free to self-host. They cover object detection, OCR, and captioning, among other tasks. 1 of them run live in the Roboflow Playground on your own images. Compare licenses, parameters, prices, and release dates side by side, or open any model page for full details.