Roboflow
Azure

Microsoft: Florence-2

Florence-2 Overview

Florence-2, introduced by Microsoft Research at CVPR 2024, is an open-source vision-language foundation model designed to unify diverse computer vision tasks within a single sequence-to-sequence framework. Unlike traditional models that specialize in specific tasks, Florence-2 accepts both images and text prompts and outputs text for tasks such as captioning, object detection, segmentation, OCR, and region-based grounding. It comes in two sizes—Florence-2-base (~230M parameters) and Florence-2-large (~770M parameters)—and is trained on FLD-5B, a large dataset of ~126M images with ~5.4B annotations.

The model demonstrates strong zero-shot and fine-tuned performance, often rivaling larger vision-language systems while remaining lightweight and efficient. Released under the MIT license, all weights are publicly available, making it accessible for fine-tuning and deployment in applications like VQA, content tagging, accessibility, and research. Florence-2’s compact design, versatility, and openness position it as a practical alternative to larger proprietary multimodal models.

Florence-2 Interactive Demo

Florence-2 Details & Performance

Details

Resources

PaperColabTutorial

Vision Tasks

CaptioningInstance SegmentationOCRObject DetectionOpen Vocabulary Object DetectionPhrase GroundingRegion Proposal

Features

Foundation VisionMultimodal VisionZero-shot Detection

Usage

Past 30 Days

Performance

Avg. Latency

Arena Rankings

Alternatives to Florence-2

Other models worth comparing for similar use cases.

Google
PaliGemma 2
PaliGemma 2 is a vision-language model released in December 2024 by Google DeepMind. It pairs the SigLIP-So400m vision encoder with the Gemma 2 language model family, extending the original PaliGemma architecture with stronger language capabilities and a wider set of transfer benchmarks. The model is designed primarily as a fine-tuning base rather than a chat-optimized assistant. Google releases pretrained "PT" checkpoints intended for task-specific adaptation rather than direct out-of-the-box use.PaliGemma 2 accepts an image paired with a text prompt and generates natural language output, supporting image captioning, visual question answering, optical character recognition, document understanding, object detection and segmentation (with appropriate fine-tuning), and a range of specialized vision-language tasks. The model is released at three parameter sizes (3B, 10B, and 28B), built on the Gemma 2 2B, 9B, and 27B language backbones. Each size is available at three input resolutions: 224, 448, and 896 pixels. Alongside the base PT checkpoints, Google released PaliGemma 2 Mix variants that have been tuned on a mixture of downstream tasks to provide stronger out-of-the-box performance for common applications such as OCR and document parsing. PaliGemma 2 is distributed under the Gemma license, a custom license from Google that permits commercial use subject to the terms of the Gemma Prohibited Use Policy.
IDEA Research
Grounding DINO
Grounding DINO is an open-vocabulary object detection model developed by IDEA Research, released in March 2023 under the Apache 2.0 license. It extends the DINO transformer-based detector with grounded pre-training, enabling it to detect arbitrary objects described by free-form text queries rather than a fixed set of predefined categories. The model integrates a text encoder with a visual backbone through a feature fusion module that aligns language and visual representations at multiple scales.Grounding DINO achieves strong zero-shot detection performance on COCO, LVIS, and ODinW benchmarks, and supports referring expression comprehension tasks. It is widely used as a foundation for open-vocabulary detection pipelines and as the detection backbone in systems such as Grounded-SAM. The model is particularly suited for applications requiring flexible, text-driven object localization across diverse domains.
Tencent
YOLO World
YOLO-World v2 Small (YOLO-World-S-v2) is the smallest variant of Tencent AI Lab’s YOLO-World v2 family, released around February 2024 under GPL-v3. With ~13 million parameters, it adopts a prompt-then-detect paradigm using offline vocabularies and is pretrained on large-scale datasets such as Objects365 and GoldG. The model processes image inputs at 640×640 or 1280×1280 resolutions and supports zero-shot open-vocabulary object detection, enabling recognition of novel categories from text prompts without retraining.Evaluations show competitive results across benchmarks like LVIS and COCO, while maintaining real-time efficiency. On an NVIDIA V100, the small variant reaches ~74 FPS at standard resolutions. Together with larger YOLO-World v2 models, it provides a scalable framework for efficient, open-vocabulary detection across diverse deployment settings.
Moondream 2
Moondream 2 is a small open-source vision-language model from Moondream, the company founded by Vikhyat Korrapati. It was first released in early 2024 and updated through mid-2025. At approximately 1.9 billion parameters, it is designed to run efficiently on consumer hardware such as laptops and edge devices while supporting a practical range of multimodal tasks. Moondream 2 combines a vision encoder based on SigLIP with a compact language backbone, trained for image understanding tasks rather than as a general chat model.The model accepts an image paired with a natural language prompt and produces text responses, supporting visual question answering, image captioning, and image-conditioned dialogue. Later Moondream 2 releases added object localization through a point API that returns coordinates for queried objects, along with improvements to OCR, counting, and document understanding. Moondream 2 is distributed under the Apache 2.0 license and is available through Hugging Face and the maintainer's distribution. Because the model is updated frequently, production deployments should pin to a specific revision rather than tracking the latest release. A successor model, Moondream 3 (Preview), was released in September 2025 with a 9B mixture-of-experts architecture and 2B active parameters, offering substantially stronger visual reasoning than Moondream 2 while retaining the efficiency-focused design. A referring expression segmentation extension to Moondream 3 was released in March 2026.
HuggingFace
SmolVLM2
SmolVLM2 is a compact multimodal vision-language model developed by the Hugging Face TB Research team, released in February 2025 under the Apache 2.0 license. It is designed for efficient image and video understanding on resource-constrained hardware, with model variants ranging from 256M to 2.2B parameters. SmolVLM2 processes images, multi-image inputs, and video alongside text queries to generate text outputs for tasks including visual question answering, image captioning, and OCR.SmolVLM2 is designed for on-device and edge deployment, requiring substantially less GPU memory than comparable multimodal models. It supports standard fine-tuning pipelines via the Hugging Face transformers library and quantization through bitsandbytes. SmolVLM2 is suited for applications where a capable vision-language model is needed without full server-scale infrastructure.
Qwen
Qwen2.5 VL 7B Instruct
Qwen2.5-VL-7B-Instruct is a 7-billion parameter vision-language model from Alibaba’s QwenLM team, released on January 26, 2025 under the Apache 2.0 license. It is the instruction-tuned variant of the 7B scale in the Qwen2.5-VL family, designed to process multimodal inputs such as text, images, charts, documents, and video. The model enables structured outputs—including JSON for structured content and bounding boxes for visual localization. Weights are publicly available on Hugging Face and GitHub, making it suitable for both research and applied multimodal use.

Deploy Florence-2 with an API

Florence-2 runs as a hosted REST endpoint through Roboflow Workflows. Pick a task, then hand the prompt to your coding agent or copy the code. Forking the workflow into a free Roboflow workspace replaces the your-workspace and YOUR_API_KEY placeholders with your own.

Connect your agent to Roboflow (once)

Add the Roboflow MCP server

claude mcp add --transport http roboflow https://mcp.roboflow.com/mcp

Run /mcp and authorize Roboflow in your browser when the OAuth flow opens.

Start a new Claude Code session so the MCP loads, then paste the prompt below (it works the same in any agent).

Fork this workflow to your Roboflow workspace to use it.

Integrate the Roboflow "Florence-2" workflow into my app.

- Endpoint: POST https://serverless.roboflow.com/<your-workspace>/workflows/playground-florence-2-c
- Auth: send my Roboflow API key as `api_key` in the request body, read from the ROBOFLOW_API_KEY env var (never hardcode).
- Body: { "api_key": ..., "inputs": { `image`: { type: "url" | "base64", value } } }.

With the Roboflow MCP connected, call `workflows_get` on "playground-florence-2-c" to read the exact input schema (the source of truth), then `workflows_run` on a sample image to confirm the output shape before writing code (the MCP is authenticated, so this needs no key). Without the MCP, use the contract above.

Before running the app, set up these keys so it does not error at runtime:
- `ROBOFLOW_API_KEY` (sent as `api_key`) from https://app.roboflow.com/settings/api
Create a .gitignore'd .env with these variables, using placeholder values for any I haven't given you. Then pause and tell me directly, in your reply: the full path to the .env file, exactly which keys I need to paste in, and the link to get each one. Wait for me to confirm I've added them before you run anything. Do not run the app until I confirm.

Then add the integration to my codebase: match my project's language, framework, and conventions; read every key from environment variables (never hardcode); add basic error handling; and include a small runnable example. If you can't tell what language my project uses, ask me.
Installpip install inference-sdk

Fork this workflow to your Roboflow workspace to use it.

# 1. Import the library
from inference_sdk import InferenceHTTPClient

# 2. Connect to your workflow
client = InferenceHTTPClient(
  api_url="https://serverless.roboflow.com",
  api_key="YOUR_API_KEY"
)

# 3. Run your workflow on an image
result = client.run_workflow(
  workspace_name="your-workspace",
  workflow_id="playground-florence-2-c",
  images={
    "image": "YOUR_IMAGE.jpg"  # Path to your image file
  },
  use_cache=True  # cache workflow definition for 15 minutes
)

# 4. Get your results
print(result)

Fork this workflow to your Roboflow workspace to use it.

const response = await fetch('https://serverless.roboflow.com/your-workspace/workflows/playground-florence-2-c', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    api_key: 'YOUR_API_KEY',
    inputs: {
      "image": {"type": "url", "value": "IMAGE_URL"}
    }
  })
});

const result = await response.json();
console.log(result);

Fork this workflow to your Roboflow workspace to use it.

curl --location 'https://serverless.roboflow.com/your-workspace/workflows/playground-florence-2-c' \
--header 'Content-Type: application/json' \
--data '{
  "api_key": "YOUR_API_KEY",
  "inputs": {
    "image": {"type": "url", "value": "IMAGE_URL"}
  }
}'

Florence-2 License

MIT

License terms and commercial-use guidance for Florence-2.

This model is released under the MIT License, a short and permissive open-source license that allows commercial use, modification, and redistribution.

Read the full MIT license ↗

Yes. Under the terms of the MIT license, you can freely use this model for commercial purposes. You must retain the copyright notice and license text when redistributing.

License information is provided as a guide and is not legal advice.