Roboflow

Best Zero Shot Segmentation Models

Compare the best 3 zero shot segmentation models and try 1 of them on your own images, free in the Roboflow Playground. 3 are open-weight, so you can self-host them for free under their licenses.

3 models · 3 open-weight · 1 free to try · prices synced Aug 26, 2026

Open-Source Zero Shot Segmentation Models

3 models with downloadable weights you can self-host under their licenses (Custom and Apache 2.0). 1 run live in the Playground through hosted APIs, so self-hosting is optional.

Meta
SAM 3
Released on November 19th, 2025, Segment Anything 3 (SAM 3) is a zero-shot image segmentation model that “detects, segments, and tracks objects in images and videos based on concept prompts.” This model was developed by Meta as the third model in the Segment Anything series.

Unlike its previous SAM models (Segment Anything and Segment Anything 2), you can provide SAM 3 with the prompt “shipping container” and it will generate precise segmentation masks for all shipping containers in an image. SAM 3 generates segmentation masks that correspond to the location of the objects found with a text prompt.
IDEA Research
Grounded SAM
Grounded SAM is an open-vocabulary image segmentation model developed by IDEA Research, released in January 2024 under the Apache 2.0 license. It combines Grounding DINO, a zero-shot open-vocabulary object detector, with the Segment Anything Model to produce precise segmentation masks for objects identified through free-form text prompts. The two models are used sequentially: Grounding DINO localizes objects from a text query, and SAM generates the corresponding segmentation masks.Grounded SAM enables zero-shot instance segmentation without task-specific training data, making it applicable to domains where labeled segmentation data is scarce. It supports arbitrary text queries and can segment objects not represented in standard training sets. The model is commonly used in automated labeling pipelines, robotic perception, and domain-specific vision applications requiring open-vocabulary segmentation.
SAM-CLIP
SAM-CLIP is a unified vision foundation model introduced by researchers at Apple and the University of Illinois Urbana-Champaign in October 2023. It merges two popular vision foundation models — Meta's Segment Anything Model (SAM) and OpenAI's CLIP — into a single shared Vision Transformer backbone through a combination of multi-task learning, continual learning, and teacher-student distillation. The method requires only a small fraction of the original pretraining datasets and demonstrates that complementary capabilities from distinct foundation models can be consolidated without retraining from scratch, reducing the storage and compute cost of running both models in inference.The resulting model retains SAM's zero-shot segmentation ability and CLIP's zero-shot classification and image-text retrieval, while introducing new capabilities the individual models lacked. SAM-CLIP establishes state-of-the-art results on zero-shot semantic segmentation across five benchmarks, improving mean IoU by 6.8 points on Pascal VOC and 5.9 points on COCO-Stuff over prior specialized models. The paper was accepted at the UniReps Workshop at NeurIPS 2023 and the eLVM Workshop at CVPR 2024. Apple has published the research but has not released model weights or inference code publicly.

How to Choose a Zero-Shot Segmentation Model

Zero-shot segmenters produce masks for things they were never trained on, guided by prompts. The practical choice is between prompt types and speed tiers within the SAM family and its combinations.

Click and box prompts (classic SAM) fit interactive use: a human indicates the object, the model returns the mask. Text-driven combinations like Grounded SAM chain an open-vocabulary detector into SAM so "every forklift" becomes masks with no clicks, which is what automated pipelines need. Newer concept-promptable models fold that into one step.

The highest-value production use of zero-shot segmentation is building datasets: masks that took minutes each by hand take seconds with a promptable model, and a compact trained segmentation model then serves production traffic at low latency. Using the big zero-shot model directly in production is usually the expensive path.

The bottom line: Interactive cutouts: click-prompt SAM. Automated pipelines: text-driven combinations. Either way, its biggest payoff is labeling data to train a faster production model.

Frequently Asked Questions About Zero Shot Segmentation Models

Zero-shot segmentation is the task of producing object masks for things the model was never specifically trained to segment, guided by a prompt such as a click, a box, or a text phrase. The SAM family established the recipe: a heavy image encoder computes features once, then a light prompt encoder and mask decoder turn each prompt into a mask interactively. Because the model learned a general notion of object boundaries from enormous mask datasets, it transfers to new domains without fine-tuning. It is used to accelerate dataset labeling, power click-to-cut-out image editing, and provide masks in open-world settings where classes are not known in advance. This page lists 3 zero shot segmentation models, including 3 open-weight options you can self-host; 1 of them run live in the Playground so you can test them on your own images.

It depends on your task and constraints. For fixed categories in production, a model fine-tuned on your own data typically beats any general-purpose model. Compare the zero shot segmentation models on this page and try them on your own images to see which fits.

Yes. 3 of the 3 zero shot segmentation models here are open-weight (for example SAM 3, Grounded SAM, and SAM-CLIP), free to self-host under their licenses (Custom and Apache 2.0).

Yes. You can run 1 of them in the Roboflow Playground for free. Upload an image and compare the models' output side by side, no setup required.

This page lists all 3 zero shot segmentation models in the Roboflow Playground catalog, all of them open-weight and free to self-host; 1 of them run live in the Roboflow Playground on your own images. Compare licenses, parameters, and release dates side by side, or open any model page for full details.