Microsoft

Florence-2, introduced by Microsoft Research at CVPR 2024, is an open-source vision-language foundation model designed to unify diverse computer vision tasks within a single sequence-to-sequence framework. Unlike traditional models that specialize in specific tasks, Florence-2 accepts both images and text prompts and outputs text for tasks such as captioning, object detection, segmentation, OCR, and region-based grounding. It comes in two sizes—Florence-2-base (~230M parameters) and Florence-2-large (~770M parameters)—and is trained on FLD-5B, a large dataset of ~126M images with ~5.4B annotations.The model demonstrates strong zero-shot and fine-tuned performance, often rivaling larger vision-language systems while remaining lightweight and efficient. Released under the MIT license, all weights are publicly available, making it accessible for fine-tuning and deployment in applications like VQA, content tagging, accessibility, and research. Florence-2’s compact design, versatility, and openness position it as a practical alternative to larger proprietary multimodal models.

LLaVA-1.5 is an open-source large multimodal model released in October 2023 by researchers at the University of Wisconsin-Madison and Microsoft Research. It builds on the original LLaVA architecture by introducing targeted refinements: switching the vision encoder to CLIP-ViT-L at 336-pixel resolution, replacing the projection layer with a two-layer MLP, and adding academic-task-oriented visual question answering data with response formatting prompts during training. These modifications achieve state-of-the-art performance across 11 benchmarks at release, with training completing in approximately one day on a single 8-A100 node.The model accepts an image paired with a text prompt and generates natural language responses, supporting visual question answering, image captioning, and open-ended visual conversation. LLaVA-1.5 is available in 7B and 13B parameter variants built on the Vicuna language model, and is distributed under the Llama 2 Community License due to its Llama-2-based foundation. The original LLaVA paper was presented as an oral at NeurIPS 2023. Subsequent releases in the series (LLaVA-NeXT (LLaVA-1.6), LLaVA-NeXT-Video, and LLaVA-OneVision) are separate models with their own release pages and build on this foundation with expanded OCR, video, and multi-image capabilities.

TrOCR (Transformer-based Optical Character Recognition) is an end-to-end OCR model released in September 2021 by Microsoft Research. It departs from the traditional two-stage OCR pipeline — which typically combines a CNN-based feature extractor with an RNN-based sequence decoder — by using a pure Transformer architecture composed of a pretrained image Transformer encoder and a pretrained text Transformer decoder, an approach that later became standardized as the VisionEncoderDecoder pattern in Hugging Face Transformers.TrOCR takes a cropped text line image as input and produces a sequence of output tokens, supporting printed, handwritten, and scene text recognition. The model is designed for use downstream of a separate text detection stage — TrOCR recognizes text in pre-cropped regions rather than detecting text locations in a full page. Microsoft released three size variants: TrOCR-small (62M parameters, DeiT-small encoder + MiniLM decoder), TrOCR-base (334M parameters, BEiT-base encoder + RoBERTa-large decoder), and TrOCR-large (558M parameters, BEiT-large encoder + RoBERTa-large decoder). Pretrained and fine-tuned checkpoints are available for printed text (on SROIE), handwritten text (on IAM), and scene text (on the standard scene text benchmarks) under the MIT license, distributed through the Microsoft unilm repository and Hugging Face. At release, TrOCR achieved state-of-the-art results across all three benchmark categories, and the model continues to be used as a baseline for handwritten text recognition.

ResNet-50 is a deep convolutional neural network architecture introduced in the 2015 paper "Deep Residual Learning for Image Recognition" by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun at Microsoft Research. It is part of the ResNet (Residual Network) family, which introduced residual connections — shortcut paths that allow gradients to bypass layers during training — solving the degradation problem that had previously limited the practical training of very deep networks. ResNet-50 specifically refers to a 50-layer variant with approximately 25.6 million parameters, structured as a sequence of bottleneck residual blocks consisting of 1×1, 3×3, and 1×1 convolutions.ResNet-50 was trained on the ImageNet classification benchmark and achieved leading top-1 accuracy at release. Beyond classification, it became a widely used backbone feature extractor for downstream tasks including object detection (as the base network in Faster R-CNN, Mask R-CNN, and RetinaNet) and semantic and instance segmentation. Most current implementations in PyTorch torchvision, TensorFlow, and NVIDIA NGC use the ResNet-50 v1.5 variant, which relocates the stride-2 downsampling from the first 1×1 convolution to the 3×3 convolution within each bottleneck block, yielding approximately 0.5% higher top-1 accuracy than the original v1 formulation at a small throughput cost. ResNet-50 remains a common reference architecture in computer vision benchmarks and a standard backbone choice in detection and segmentation frameworks. The original Microsoft Research code is released under the MIT license.

Faster R-CNN is an object detection model introduced by Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun at Microsoft Research, published at NIPS in June 2015. It advances upon Fast R-CNN and R-CNN by introducing the Region Proposal Network (RPN), a fully convolutional network that shares features with the detection network and generates object proposals at negligible additional cost. This makes Faster R-CNN the first near-real-time deep learning object detector based on region proposals.Faster R-CNN achieves strong detection accuracy on PASCAL VOC and MS COCO at the time of release. It remains a widely referenced architecture in computer vision research and is available through Meta's Detectron2 framework as a maintained PyTorch implementation. It is most appropriate for offline or server-side inference tasks where accuracy is prioritized over latency, as its two-stage pipeline carries higher inference cost than single-stage detectors.