Roboflow
OpenAI

OpenAI: CLIP

CLIP Overview

OpenAI CLIP (Contrastive Language-Image Pretraining) is a vision-language model released in January 2021 by OpenAI. It jointly trains an image encoder and a text encoder to produce matching embeddings for image-caption pairs, using a contrastive objective over WebImageText (WIT), a dataset of 400 million image-text pairs collected from the public web. By learning to associate images with free-form text rather than a fixed set of class labels, CLIP produces a shared embedding space that enables zero-shot classification with arbitrary vocabularies at inference time.

CLIP supports zero-shot image classification by embedding candidate class labels as text and selecting the label whose embedding is closest to a given image's embedding. It is also widely used for image-text retrieval, as a frozen backbone in downstream vision-language models, and as a building block for content moderation, similarity search, and generative model guidance — notably as the text conditioning mechanism in early versions of Stable Diffusion. OpenAI released several CLIP variants built on different vision encoders, including ResNet and Vision Transformer backbones at multiple sizes and input resolutions, with ViT-L/14 at 336 pixels being the largest and most widely adopted. CLIP is distributed under the MIT license. The model has been widely influential as the basis for subsequent vision-language work — including SigLIP, OpenCLIP, and MetaCLIP — and remains a common reference baseline despite being released in 2021 and surpassed on many benchmarks by later models.

CLIP Details & Performance

Details

Resources

GitHubPaper

Vision Tasks

ClassificationImage EmbeddingImage SimilarityImage Tagging

Features

Foundation VisionMultimodal Vision

Usage

Past 30 Days

Not available

Not in Playground

Performance

Avg. Latency

Arena Rankings

Not yet ranked in arena

Alternatives to CLIP

Other models worth comparing for similar use cases.

Google
SigLIP
SigLIP is a vision-language model released in March 2023 by researchers at Google DeepMind. It adapts the CLIP image-text pretraining approach by replacing CLIP's softmax-based contrastive loss with a pairwise sigmoid loss, which operates independently on each image-text pair rather than requiring a global view of all pairs in a batch. This change decouples the loss from batch size, enabling more memory-efficient training and improved performance at smaller batch sizes, a regime where softmax contrastive learning typically struggles. Despite this simplification, SigLIP matches or exceeds CLIP-style models on zero-shot image classification and image-text retrieval benchmarks when trained on comparable data.SigLIP is distributed as an image encoder plus aligned text encoder, supporting zero-shot classification with arbitrary class vocabularies, image-text retrieval, and use as a frozen backbone in downstream vision-language models. Pretrained models are available at multiple Vision Transformer sizes and input resolutions, including 224, 256, 384, and 512 pixel inputs. SigLIP is released under the Apache 2.0 license by Google and is used as the vision encoder in Google's PaliGemma and PaliGemma 2. A successor, SigLIP 2, was released in February 2025 with multilingual support across 109 languages, improvements to localization and dense prediction, and two resolution handling variants (FixRes for backward-compatible fixed resolutions and NaFlex for native aspect ratio with variable sequence length).
Meta
DINOv2
DINOv2 is a self-supervised vision foundation model released in April 2023 by Meta AI's FAIR lab. It produces general-purpose visual features that transfer to a wide range of downstream tasks (including image classification, semantic segmentation, depth estimation, and image retrieval) without requiring task-specific fine-tuning. DINOv2 is trained on a curated dataset of 142 million images using a self-supervised objective combining student-teacher distillation, masked image modeling, and an image-level contrastive loss, extending the approach introduced in the original DINO.The model family spans Vision Transformer sizes from ViT-S (21M parameters) to ViT-g (1.1B parameters), with the larger variants setting state-of-the-art results on linear-probing benchmarks for classification, segmentation, and dense prediction tasks at release. DINOv2 features can be used directly as frozen backbones, reducing the need for labeled training data in downstream applications. The model is primarily used as an image encoder rather than as a complete task-specific model, making it a common backbone choice for custom vision pipelines. DINOv2 code and pretrained weights are released under the Apache 2.0 license, which was adopted after an initial CC-BY-NC 4.0 release in response to community requests for commercial compatibility. A successor model, DINOv3, was released in August 2025 with further scaling and a new training technique called Gram anchoring.
Google
Vision Transformer (ViT)
Vision Transformer is an image classification model developed by Google Research, first published in October 2020. It applies the transformer architecture directly to sequences of image patches without convolutional layers. Each image is divided into fixed-size patches, linearly projected into embeddings, and processed by a standard transformer encoder with multi-head self-attention. A classification token prepended to the patch sequence aggregates global image information for the final prediction.When pre-trained on large datasets such as JFT-300M and fine-tuned on ImageNet, ViT achieves competitive accuracy with state-of-the-art CNNs of the period. It performs best when pre-training data is abundant, as the lack of convolutional inductive biases makes it less data-efficient than CNN-based classifiers on smaller datasets. ViT established the foundation for transformer-based vision architectures and has influenced a broad range of subsequent models.
SAM-CLIP
SAM-CLIP is a unified vision foundation model introduced by researchers at Apple and the University of Illinois Urbana-Champaign in October 2023. It merges two popular vision foundation models — Meta's Segment Anything Model (SAM) and OpenAI's CLIP — into a single shared Vision Transformer backbone through a combination of multi-task learning, continual learning, and teacher-student distillation. The method requires only a small fraction of the original pretraining datasets and demonstrates that complementary capabilities from distinct foundation models can be consolidated without retraining from scratch, reducing the storage and compute cost of running both models in inference.The resulting model retains SAM's zero-shot segmentation ability and CLIP's zero-shot classification and image-text retrieval, while introducing new capabilities the individual models lacked. SAM-CLIP establishes state-of-the-art results on zero-shot semantic segmentation across five benchmarks, improving mean IoU by 6.8 points on Pascal VOC and 5.9 points on COCO-Stuff over prior specialized models. The paper was accepted at the UniReps Workshop at NeurIPS 2023 and the eLVM Workshop at CVPR 2024. Apple has published the research but has not released model weights or inference code publicly.

CLIP License

MIT · Permissive license

CLIP is released under MIT, a permissive license. The CLIP license lets you use, modify, and sell work built on the model, with the copyright notice as the only real obligation and no requirement to open-source related code changes.

Commercial use
Permitted with no separate commercial license. No usage caps, revenue thresholds, or field-of-use limits apply to CLIP.
Modification
Permitted. You can fine-tune or rewrite CLIP and keep the result closed-source.
Redistribution
Permitted. Include the original copyright and permission notice in copies or substantial portions of the work.

MIT grants no explicit patent license and disclaims all warranties. If patent exposure is a concern for your deployment, review it with counsel before launch.

Read the full MIT license ↗

Do I need a commercial license for CLIP?

No commercial license is needed for CLIP: permissive terms let you keep related code private while deploying commercially.

Do not hesitate to reach out with questions for your commercial project — our team will help you start solving business problems on the first call. See Roboflow commercial licensing for the models included in each plan.

Talk to sales

This model is released under the MIT License, a short and permissive open-source license that allows commercial use, modification, and redistribution.

Yes. Under the terms of the MIT license, you can freely use this model for commercial purposes. You must retain the copyright notice and license text when redistributing.

License information is provided as a guide and is not legal advice.