Top Foundation Vision Models

Claude Fable 5 is Anthropic's first generally available Mythos-class large language model, released on June 9, 2026. It is built for long-horizon, asynchronous, and agentic tasks that prior Claude generations could not sustain, including multi-day autonomous coding sessions, complex knowledge work, and document-heavy analysis. The model supports a 1 million token context window with up to 128,000 output tokens per request and uses adaptive thinking as its sole reasoning mode, where the effort level is adjustable but raw chain-of-thought is never returned. Vision capabilities allow the model to parse diagrams, charts, and tables embedded in files and PDFs, and to use visual feedback to evaluate its own coding outputs against design goals. On benchmarks such as SWE-Bench Pro, the model scores 80.3% compared to 69.2% for Claude Opus 4.8, and it leads on CursorBench 3.1 for autonomous coding workflows.Claude Fable 5 shares the same underlying model weights as Claude Mythos 5, but is deployed with safety classifiers that automatically reroute queries in high-risk domains — including cybersecurity, biology, and chemistry — to Claude Opus 4.8. These classifiers trigger in fewer than 5% of sessions on average. As a designated Covered Model, all traffic is subject to mandatory 30-day data retention to support safety monitoring. The model is available via the Claude API, Amazon Bedrock, Vertex AI, and Microsoft Foundry. Anthropic has not publicly disclosed parameter count, architecture details, or training data composition for this model.

Claude Opus 4.8 is Anthropic's most capable generally available large language model, released on May 28, 2026 as an incremental upgrade to Claude Opus 4.7. The model accepts text and image inputs and produces text outputs, with a 1 million token context window on the Claude API, Amazon Bedrock, and Google Cloud Vertex AI (200k tokens on Microsoft Foundry) and up to 128k max output tokens. It uses adaptive thinking and supports adjustable effort tiers — high by default, with extra and max tiers available for more demanding tasks. A fast mode operates at approximately 2.5x standard speed. The model is described by Anthropic as a hybrid reasoning model designed for advanced coding, agentic workflows, long-context reasoning, and professional knowledge work.Key behavioral improvements over Opus 4.7 include substantially reduced rates of unreported code flaws, improved honesty in self-assessment, and better tool-calling reliability. On Anthropic's Super-Agent benchmark, Opus 4.8 completes every case end-to-end, and it scores 84% on Online-Mind2Web for computer-use and browser-agent tasks. It achieves 88.6% on SWE-bench Verified and 69.2% on SWE-bench Pro. Alongside the model, Anthropic launched Dynamic Workflows in Claude Code (research preview), which enables Claude to orchestrate hundreds of parallel subagents for codebase-scale tasks such as large migrations. The Messages API was also updated to accept mid-task system messages without breaking prompt caching, improving support for long-running agentic pipelines.

Qwen3.6-27B is a dense 27-billion-parameter multimodal language model developed by Alibaba's Qwen team and released on April 22, 2026. It combines a causal language model with an integrated vision encoder, supporting text, image, and video inputs natively. The architecture employs a hybrid attention design that interleaves Gated DeltaNet linear attention blocks with standard Gated Attention layers across 64 transformer layers with a hidden dimension of 5,120. Unlike Mixture-of-Experts variants in the Qwen3.6 family, all 27 billion parameters are active on every inference pass, simplifying deployment and quantization. The model supports a native context window of 262,144 tokens, extensible to approximately 1,010,000 tokens via YaRN scaling. It is released under the Apache 2.0 license with open weights available on Hugging Face and ModelScope.The model introduces two notable capabilities relative to prior Qwen releases: enhanced agentic coding support covering frontend workflows and repository-level reasoning, and a Thinking Preservation mechanism that retains chain-of-thought reasoning context across multi-turn conversation history to reduce redundant token generation in iterative agent sessions. It supports both a thinking mode for multi-step reasoning and a non-thinking mode for faster responses within a single model. On coding benchmarks, Qwen reports scores of 77.2 on SWE-bench Verified, 59.3 on Terminal-Bench 2.0, and 48.2 on SkillsBench. Vision capabilities include chart understanding (CharXiv RQ: 78.4), OCR (CC-OCR: 81.2), and video understanding (VideoMME with subtitles: 87.7).

Claude Opus 4.7 is a proprietary multimodal language model developed by Anthropic, released on April 16, 2026. It is designed for agentic coding, long-horizon task execution, and enterprise knowledge work. The model supports text and vision inputs and operates with a context window of up to 1,000,000 tokens. It introduces adaptive thinking, which dynamically allocates reasoning based on task complexity, along with configurable effort controls including a new xhigh setting that sits between the existing high and max levels. It achieves 87.6% on SWE-bench Verified and 78.0% on OSWorld-Verified, reflecting strong performance on autonomous software engineering and computer use tasks respectively.Compared to Claude Opus 4.6, version 4.7 shows improved instruction following and higher reliability in extended agentic tasks. Vision capabilities now support high-resolution inputs up to 2,576px on the long edge (~3.75 megapixels), more than three times the resolution of prior Claude models, enabling finer interpretation of dense diagrams, UI screenshots, and document layouts. These improvements, combined with self-verification on long-running tasks and a new task budget system for controlling agentic loops, make it well-suited for complex software engineering, technical analysis, and multimodal vision workflows.

Qwen3.6-35B-A3B is a sparse Mixture-of-Experts (MoE) multimodal language model developed by the Qwen team at Alibaba Group. It carries 35 billion total parameters but activates only approximately 3 billion per forward pass via a learned routing mechanism, giving it the representational capacity of a large dense model at a fraction of the inference compute. The model is natively multimodal, processing images, documents, and video alongside text as a core architectural capability rather than an add-on. It supports a native context window of 262,144 tokens, extensible up to 1,010,000 tokens via YaRN. A key design feature is the unified thinking/non-thinking mode framework: users can switch between deliberate chain-of-thought reasoning and fast direct responses within a single model, and a "thinking preservation" option retains reasoning context across multi-turn agentic workflows to reduce redundant computation.The model is specifically optimized for agentic coding tasks, including repository-level reasoning, frontend workflow generation, multi-step tool use, and MCP (Model Context Protocol) integration. On SWE-bench Verified it scores 73.4%, on Terminal-Bench 2.0 it scores 51.5%, and on MCPMark it scores 37.0%. For vision-language tasks it achieves 92.0 on RefCOCO, 89.9 on OmniDocBench 1.5, and 83.7 on VideoMMMU. The model also supports Multi-Token Prediction (MTP) for speculative decoding. All Qwen3.6 open-weight models are released under the Apache 2.0 license.

GPT-5.4 mini is a fast, cost-efficient model developed by OpenAI and released on March 17, 2026, optimized for high-throughput workloads and subagent orchestration. It supports text and image inputs within a 400,000-token context window, making it ideal for processing extensive visual datasets and large codebases in a single request. Designed for low-latency production environments, the model integrates with key API features including function calling, web search, and tool-based computer use, allowing it to assist in automated workflows that require navigating digital interfaces.Compared to the previous GPT-5 mini, this version runs more than twice as fast while approaching the performance levels of the flagship GPT-5.4 on reasoning and coding benchmarks. While the larger GPT-5.4 introduces native, state-of-the-art computer-use capabilities, GPT-5.4 mini provides a scalable alternative for interpreting screenshots and reasoning over dense UI layouts. For vision tasks on Playground, it excels at extracting structured information from visual documents and assisting in agentic tasks that involve real-time interpretation of software interfaces alongside text.

GPT-5.4 nano is a high-throughput model developed by OpenAI and released on March 17, 2026, as the efficiency-optimized entry in the GPT-5.4 family. Engineered for cost-sensitive production environments and latency-critical workloads, it features an expanded 400,000-token context window that enables the processing of large document batches or extensive logs in a single pass. The model is primarily optimized for text-heavy operations, serving as a premier engine for high-volume classification, data extraction, ranking, and the orchestration of lightweight sub-agents where speed and low per-token costs are the primary requirements.While it supports text and image inputs, GPT-5.4 nano is designed as a text-first worker rather than a specialized visual reasoning tool. In multi-model architectures, it is best utilized for structured text tasks and simple coding sub-tasks, leaving intensive vision reasoning and UI navigation to its sibling, GPT-5.4 mini. Compared to the previous GPT-5 nano, this version provides a significant leap in reliability for structured outputs and tool calling, making it a dependable and economical choice for developers building scalable, automated pipelines that require rapid execution at the edge of the GPT-5.4 ecosystem.

GPT-5.4 is a proprietary multimodal large language model developed by OpenAI and released on March 5, 2026. It is designed for professional workloads such as advanced software development, research, and agentic automation. The model combines the general reasoning capabilities of the GPT-5 series with software engineering improvements derived from GPT-5.3-Codex. In the API and Codex environments it supports context windows of up to 1 million tokens, enabling long-context reasoning and large-scale code or document workflows.Compared with GPT-5.2, GPT-5.4 reduces false individual claims by 33% and lowers overall response errors by 18%, improving factual reliability across complex tasks. It is also the first general-purpose OpenAI release with native computer-use capabilities, allowing agents to interact with desktops, browsers, and external applications to complete multi-step workflows. The model family includes three variants: GPT-5.4 (standard), GPT-5.4 Pro for higher-performance workloads, and GPT-5.4 Thinking, a reasoning-oriented version in ChatGPT that presents an upfront plan before generating its response. The API also introduces a Tool Search system that allows models to retrieve tool definitions dynamically, reducing token usage in tool-heavy integrations.

Gemini 3.1 Pro is a proprietary multimodal model from Google’s Gemini 3 series, released in early 2026 and designed for advanced reasoning across large multimodal datasets. It accepts text, images, audio, video, and documents, supporting up to a 1-million-token input context with up to 64k output tokens. Compared with Gemini 3 Pro, it improves long-context synthesis and multi-step reasoning, enabling more reliable analysis of large documents, datasets, and software codebases.The model also advances visual understanding and grounding, allowing it to interpret UI screenshots, diagrams, and real-world scenes while referencing specific regions within images or video. These capabilities make Gemini 3.1 Pro well suited for multimodal workflows involving document processing, interface analysis, robotics research, and complex visual reasoning.

Claude Sonnet 4.6 is Anthropic's mid-tier large language model, released February 17, 2026, designed to balance performance, cost, and versatility for professional and developer use. It supports text and vision-based tasks with advanced reasoning, agentic capabilities, and Adaptive Thinking — a mode where the model dynamically scales its internal reasoning depth. A beta context window of up to 1,000,000 tokens (200K standard) enables processing of entire codebases or document collections in a single request. Parameters are undisclosed.Optimized for coding, computer use, long-context reasoning, agent planning, and knowledge work, Sonnet 4.6 delivers a full generational upgrade over Sonnet 4.5 and approaches Opus 4.5-level performance across many benchmarks at a fraction of the cost. It is the default model on Claude.ai, Claude Cowork, and is available via API and major cloud platforms — making it well suited for production workloads requiring strong reasoning without flagship pricing.

Claude Opus 4.6 is the flagship large language model from Anthropic, released on 2026-02-05 for advanced reasoning, complex coding, and enterprise agent workflows. It supports text and image inputs via API, offers a 200K-token standard context window with a 1M-token beta option, and enables outputs up to 128K tokens, with adaptive reasoning and context compaction for sustained tasks.As of 2026-02-17, Anthropic also released Claude Sonnet 4.6, extending the 1M-token context window to a broader tier. Opus remains positioned for maximum depth and benchmark performance, while Sonnet 4.6 brings long-context capability to more cost- and latency-sensitive production use cases.

Gemini 3 Flash is a proprietary multimodal large language model developed by Google through Google DeepMind, designed to deliver fast, cost-efficient reasoning across real-time products and developer workflows. Released in December 2025, it is the Flash-tier variant of the Gemini 3 family, balancing low latency with reasoning quality approaching Pro models.The model supports text, images, audio, and video, with an exceptionally large context window of roughly one million input tokens and outputs up to ~65k tokens. It emphasizes rapid responses for coding, summarization, analysis, and agentic tasks, and exposes configurable “thinking levels” via API to trade speed for deeper reasoning. Today, Gemini 3 Flash positions itself as a high-throughput, production-ready model, serving as the default in the Gemini app and Google Search’s AI Mode, optimized for scalable, interactive AI applications.

GPT-5.2 is OpenAI’s latest flagship large language model, released in December 2025. It is a proprietary, multimodal system supporting text and vision inputs, along with tool use, and features a 400,000-token context window designed for working with long documents, extended conversations, and complex workflows.Relative to GPT-5.1, GPT-5.2 is positioned by OpenAI as offering improved long-context reasoning, more capable tool use, and stronger performance on professional tasks such as writing, coding, spreadsheet work, and image interpretation. The model is available in multiple variants (including Instant, Thinking, and Pro) that balance speed, cost, and depth of reasoning, making GPT-5.2 a general-purpose model aimed at reliability and workflow robustness rather than minimal latency or lowest cost.

Claude Opus 4.5 is Anthropic’s most advanced large language model in the Claude Opus family, designed for high-end reasoning, coding, and autonomous agent workflows. Released in late 2025, it targets developers and enterprises that need reliable long-context understanding and strong multi-step problem solving in production environments.The model supports text and code natively, with reported multimodal capabilities for documents and images, and offers an exceptionally large context window of up to roughly 200,000 tokens. Claude Opus 4.5 emphasizes long-horizon task execution, complex code generation and refactoring, and sustained reasoning over large inputs. In the current landscape, it positions itself as a premium, accuracy- and reasoning-focused alternative to faster or cheaper peers, trading cost for depth and contextual fidelity. Typical applications include advanced coding assistants, research analysis, agentic automation, and enterprise knowledge workflows deployed via Anthropic’s API or major cloud platforms.

Released on November 19th, 2025, Segment Anything 3 (SAM 3) is a zero-shot image segmentation model that “detects, segments, and tracks objects in images and videos based on concept prompts.” This model was developed by Meta as the third model in the Segment Anything series.

Unlike its previous SAM models (Segment Anything and Segment Anything 2), you can provide SAM 3 with the prompt “shipping container” and it will generate precise segmentation masks for all shipping containers in an image. SAM 3 generates segmentation masks that correspond to the location of the objects found with a text prompt.

SAM 3D Objects is a 3D reconstruction model released on November 19, 2025 by Meta AI as part of the broader SAM 3 release. It extends the Segment Anything Model family from 2D segmentation into 3D object reconstruction, predicting geometry, texture, and spatial layout for individual objects from a single RGB image. Given an image together with a prompt identifying the object (a segmentation mask, point, or bounding box), the model outputs a full textured 3D mesh, without requiring multi-view captures, depth sensors, or known camera parameters.SAM 3D Objects uses a two-stage transformer architecture: a coarse stage that predicts 3D shape and object pose, followed by a refinement stage that adds texture and surface detail, with DINOv2 used to encode the input image. The model is trained via a human-and-model-in-the-loop data engine that combines synthetic 3D assets with real-image annotations, producing approximately 3.14 million mesh annotations across nearly 1 million images. Meta released SAM 3D Objects alongside SAM 3D Body, a companion model for single-image human mesh recovery, and SAM 3D Artist Objects (SA-3DAO), a new evaluation benchmark assembled with artist-created 3D ground truth. The model is designed as a companion to SAM 3 for downstream applications including augmented reality, robotics, content creation, and visual effects; it already powers the View in Room feature on Facebook Marketplace. SAM 3D Objects is released under the SAM 3 license; users should review the license terms prior to commercial use.

GPT-5.1 is an OpenAI frontier-grade model in the GPT-5 series, offering stronger general-purpose reasoning, clearer long-form responses, and improved instruction following. It introduces two variants—Instant and Thinking—that dynamically adjust computational depth. Instant focuses on fast, conversational replies, while Thinking provides deeper, more thorough reasoning for complex tasks. In ChatGPT, GPT-5.1 also powers an Auto mode that switches between these variants automatically based on task difficulty.The model supports significantly expanded context windows: up to 16K/32K/128K tokens for Instant (depending on tier) and up to 196K tokens for Thinking on paid tiers. GPT-5.1 is also compatible with ChatGPT tools such as web search, file and image analysis, and multi-step workflows.GPT-5.1 includes enhanced tone and style controls, allowing responses to be tailored using presets like Friendly, Professional, or Efficient, along with fine-grained adjustments for warmth, brevity, and emoji usage. Designed for broad applications in research assistance, coding, analysis, and conversational agents, GPT-5.1 serves as OpenAI’s primary full-capability successor to GPT-5 across ChatGPT and API integrations.

Claude Haiku 4.5 is Anthropic’s lightweight model in the Claude 4.5 series, released in October 2025 under a proprietary license. Designed for speed and cost efficiency, it delivers near-frontier performance while maintaining Anthropic’s AI Safety Level 2 standard. Haiku 4.5 supports both text and multimodal (text and image) inputs, integrates tool use and extended reasoning, and features a 200,000 token context window, making it adept at handling long or complex workflows. Though the parameter count remains undisclosed, it achieves about 73.3% on SWE-bench Verified, reflecting strong coding and reasoning ability. Haiku 4.5 is ideal for developers and researchers seeking rapid, cost-effective model calls for analysis, coding, or multimodal understanding.

Claude Sonnet 4.5, released by Anthropic in September 2025, is the company’s most advanced Sonnet-series model, built for high-performance reasoning, coding, and long-horizon agentic workflows. It is a multimodal system that accepts both text and images, with a 200,000-token context window designed for handling large documents and extended interactions. Anthropic highlights its improvements in reliability, reduced sycophancy, and alignment, making it suitable for sustained enterprise use.The model delivers strong results in coding and autonomous workflows, achieving 61.4% on the OSWorld benchmark and leading performance on SWE-bench Verified. It introduces infrastructure features such as a memory tool (beta), checkpointing for Claude Code, parallel tool use, and tighter integration with VS Code. Compared to Opus, which targets broader reasoning, Sonnet 4.5 is optimized for structured, long-duration tasks. Positioned against leading offerings from OpenAI and Google, it is aimed at enterprise automation, software engineering, and research-intensive applications.

GPT-5, released by OpenAI in August 2025, is a multimodal large language model that advances beyond the GPT-4 family with a new “unified system” architecture. This design allows the model to dynamically choose between fast responses and extended reasoning depending on task complexity. It supports text, code, and images, alongside stronger tool use and agentic workflows, making it more adaptable for real-world problem solving. While its exact context window size is not disclosed, GPT-5 is optimized for long-horizon reasoning and multi-step tool chaining, indicating substantially expanded capacity over its predecessors.The release introduced specialized variants: GPT-5 Pro, offering extended reasoning for complex workflows, and GPT-5 Codex, optimized for advanced coding tasks such as large-scale refactoring and code review. GPT-5 shows benchmark gains in coding, biomedical reasoning, multimodal analysis, and scientific tasks. Developers also gain new controls, such as verbosity and personalization parameters, for greater steerability. With these improvements, GPT-5 positions itself as OpenAI’s most capable and versatile model, suited for enterprise automation, research, healthcare, and sophisticated coding environments.

GPT-5 Mini, released by OpenAI on August 7, 2025, is a mid-tier variant of the GPT-5 family that balances cost, speed, and capability. It is multimodal, supporting both text and image inputs, and offers a substantial input context window of ~400,000 tokens with output lengths up to ~128,000 tokens. While less powerful than the full GPT-5, it inherits its safety tuning, instruction-following improvements, and multimodal reasoning, making it a practical choice for developers who need large context handling without the expense of premium models.GPT-5 Mini is optimized for affordability while retaining strong reasoning performance. Benchmarks show it outperforming earlier models such as GPT-4o on many multimodal and medical VQA tasks, though it lags behind GPT-5 on the most complex problems. Ideal use cases include prototyping, scalable content generation, document analysis, and mid-range reasoning tasks where efficiency and context capacity matter more than top-tier accuracy.

GPT-5 Nano, released by OpenAI on August 7, 2025, is the smallest and most cost-efficient model in the GPT-5 family. Like its larger counterparts, it is multimodal—accepting text and images, supporting tool use, structured outputs, and reasoning—but it is optimized for speed, low latency, and affordability. It features input and output token limits of roughly 272K and 128K tokens respectively, enabling large-context processing even at its compact scale. Its knowledge cutoff is around May 2024, slightly earlier than the full GPT-5 model.GPT-5 Nano is well-suited for high-volume or cost-sensitive deployments such as mobile apps, embedded AI systems, or rapid-response APIs. While it offers less depth on complex reasoning and coding tasks compared to GPT-5 Mini or Pro, it retains core multimodal and agentic capabilities, making it an attractive option where efficiency and scale matter more than maximum performance.

Claude 4.1 Opus, released by Anthropic in August 2025, is the upgraded flagship of the Claude 4 family, building on Opus 4 with stronger reasoning and agentic capabilities. Like its predecessor, it is multimodal and optimized for text, code, and tool use, with support for large context windows suited to multi-file codebases, technical workflows, and long-horizon problem solving. On benchmarks, Opus 4.1 improves coding performance, reaching ~74.5% on SWE-Bench Verified compared to Opus 4’s ~72.5%. It demonstrates more precise debugging, refactoring, and orchestration of agentic tasks while maintaining similar safety and alignment safeguards. It is best suited for enterprise-scale software development, research automation, and advanced reasoning workflows where reliability and depth of analysis are critical.

Gemini 2.5 Flash-Lite, released for general availability on July 22, 2025, is the most cost-efficient model in the Gemini 2.5 family, designed for high-volume and latency-sensitive tasks. It is multimodal, supporting text, images, video, audio, and PDFs as inputs, with text as its primary output. The model handles up to 1 million input tokens and generates outputs up to 64K tokens, making it suitable for large-scale document or media processing at low cost. It is built on a Sparse Mixture-of-Experts architecture with native multimodal support, though exact parameter counts are undisclosed.Flash-Lite offers the lowest usage cost among Gemini 2.5 models. It introduces developer controls for “thinking mode,” allowing fine-tuning of reasoning depth vs. efficiency. It also integrates native tools such as code execution, search grounding, and URL context. While strong on translation, classification, coding, and general multimodal reasoning, it lacks support for image or audio generation in its stable release and is less capable than Gemini 2.5 Flash or Pro on complex reasoning-heavy workflows.

Gemini 2.5 Flash, released on June 17, 2025, is Google DeepMind’s production-ready, efficiency-focused model in the Gemini 2.5 family. It is multimodal, accepting text, images, video, and audio as inputs, with text as the primary output format. The model supports 1 million input tokens and up to 65K output tokens, enabling it to process very large contexts such as books, long video transcripts, or extensive datasets. Its training knowledge extends to January 2025.Designed as a price-performance leader, Gemini 2.5 Flash balances speed and reasoning power, making it suitable for everyday enterprise and developer use cases without the higher latency and cost of Pro models. It supports advanced workflows like function calling, code execution, search grounding, URL context ingestion, and structured outputs. While efficient and scalable, output length is still limited compared to its input capacity, and multimodal outputs (e.g. image or audio generation) remain restricted to specialized or preview variants.

Grok 4, released by xAI on July 9, 2025, is the fourth-generation model in the Grok family and the most advanced to date. It is multimodal, supporting text, vision, tool use, and real-time web search, with a reported 256,000-token context window for long-form reasoning and document analysis. Its training data extends through November 2024, making it the most up-to-date Grok model at launch.The lineup includes Grok 4 Generalist for broad tasks, Grok 4 Heavy for higher-capacity reasoning, and Grok 4 Code optimized for programming and debugging. A notable feature is its always-on “Think” mode, designed for deeper multi-step reasoning. While xAI has not disclosed parameter counts, Grok 4 is positioned to compete with frontier models like GPT-5 and Claude 4, balancing real-time knowledge via web integration with structured tool use. It is best suited for coding, complex reasoning, and multimodal AI assistants.

Florence-2, introduced by Microsoft Research at CVPR 2024, is an open-source vision-language foundation model designed to unify diverse computer vision tasks within a single sequence-to-sequence framework. Unlike traditional models that specialize in specific tasks, Florence-2 accepts both images and text prompts and outputs text for tasks such as captioning, object detection, segmentation, OCR, and region-based grounding. It comes in two sizes—Florence-2-base (~230M parameters) and Florence-2-large (~770M parameters)—and is trained on FLD-5B, a large dataset of ~126M images with ~5.4B annotations.The model demonstrates strong zero-shot and fine-tuned performance, often rivaling larger vision-language systems while remaining lightweight and efficient. Released under the MIT license, all weights are publicly available, making it accessible for fine-tuning and deployment in applications like VQA, content tagging, accessibility, and research. Florence-2’s compact design, versatility, and openness position it as a practical alternative to larger proprietary multimodal models.

Gemini 2.5 Pro, released on June 17, 2025, is Google DeepMind’s most capable model in the Gemini 2.5 family, optimized for deep reasoning, coding, and complex multimodal tasks. It accepts text, images, audio, video, and PDFs as input and outputs text. The model supports 1 million input tokens with an output capacity of up to 65K tokens, enabling large-scale comprehension of datasets, codebases, and technical documents. Its training knowledge extends to January 2025.Pro outperforms earlier Gemini 2.0 models across benchmarks, including agentic coding tasks where it achieved ~63.8% on SWE-Bench Verified. It supports structured outputs, function calling, code execution, search grounding, and URL context, making it well-suited for enterprise, STEM, and developer workflows. However, it does not currently support image or audio generation in its stable release, and its higher computational cost and latency make it less efficient than Flash or Flash-Lite. It is available via the Gemini API, Google AI Studio, and Vertex AI.

Depth Anything V2 is a monocular depth estimation model released in June 2024 by researchers at the University of Hong Kong and TikTok. It predicts a dense depth map from a single RGB image, enabling 3D-aware applications without the need for stereo cameras, LiDAR, or multi-view inputs. The model improves on the original Depth Anything through three modifications: replacing real labeled images with 595K high-quality synthetic images during teacher training, scaling up teacher model capacity, and using the stronger teacher to generate pseudo-labels on 62 million unlabeled real images used to train the student models. This pipeline reduces the depth prediction artifacts that can occur in reflective, transparent, and texture-poor regions. Compared to diffusion-based depth models such as Marigold, Depth Anything V2 runs more than 10× faster while producing more accurate predictions.Depth Anything V2 is released in four sizes: Small (25M), Base (97M), Large (335M), and Giant (1.3B), and in two output modes: relative depth (normalized scene-level estimates) and metric depth (absolute distance in meters, produced by fine-tuning the relative-depth backbone on depth-annotated datasets). The Small, Base, and Large model weights are released under Apache 2.0, and the Giant variant under CC-BY-NC-4.0 for non-commercial use. A successor model, Depth Anything 3, was released in November 2025 by the ByteDance Seed team, extending the framework to multi-view depth estimation and camera pose recovery.

SAM-CLIP is a unified vision foundation model introduced by researchers at Apple and the University of Illinois Urbana-Champaign in October 2023. It merges two popular vision foundation models — Meta's Segment Anything Model (SAM) and OpenAI's CLIP — into a single shared Vision Transformer backbone through a combination of multi-task learning, continual learning, and teacher-student distillation. The method requires only a small fraction of the original pretraining datasets and demonstrates that complementary capabilities from distinct foundation models can be consolidated without retraining from scratch, reducing the storage and compute cost of running both models in inference.The resulting model retains SAM's zero-shot segmentation ability and CLIP's zero-shot classification and image-text retrieval, while introducing new capabilities the individual models lacked. SAM-CLIP establishes state-of-the-art results on zero-shot semantic segmentation across five benchmarks, improving mean IoU by 6.8 points on Pascal VOC and 5.9 points on COCO-Stuff over prior specialized models. The paper was accepted at the UniReps Workshop at NeurIPS 2023 and the eLVM Workshop at CVPR 2024. Apple has published the research but has not released model weights or inference code publicly.

DINOv2 is a self-supervised vision foundation model released in April 2023 by Meta AI's FAIR lab. It produces general-purpose visual features that transfer to a wide range of downstream tasks (including image classification, semantic segmentation, depth estimation, and image retrieval) without requiring task-specific fine-tuning. DINOv2 is trained on a curated dataset of 142 million images using a self-supervised objective combining student-teacher distillation, masked image modeling, and an image-level contrastive loss, extending the approach introduced in the original DINO.The model family spans Vision Transformer sizes from ViT-S (21M parameters) to ViT-g (1.1B parameters), with the larger variants setting state-of-the-art results on linear-probing benchmarks for classification, segmentation, and dense prediction tasks at release. DINOv2 features can be used directly as frozen backbones, reducing the need for labeled training data in downstream applications. The model is primarily used as an image encoder rather than as a complete task-specific model, making it a common backbone choice for custom vision pipelines. DINOv2 code and pretrained weights are released under the Apache 2.0 license, which was adopted after an initial CC-BY-NC 4.0 release in response to community requests for commercial compatibility. A successor model, DINOv3, was released in August 2025 with further scaling and a new training technique called Gram anchoring.

The Segment Anything Model is a promptable image segmentation foundation model developed by Meta AI, released in April 2023 under the Apache 2.0 license. It introduces a general-purpose segmentation architecture trained on SA-1B, a dataset of over 1 billion masks across 11 million images collected using a data engine that leveraged the model itself. SAM accepts point, bounding box, and mask prompts and generates high-quality segmentation masks for any object in an image, including objects not seen during training.SAM achieves strong zero-shot performance across a wide range of segmentation tasks and domains. Its promptable interface makes it suitable as a building block for automated annotation, interactive segmentation tools, and integration with detection models such as Grounding DINO. SAM has been extended by subsequent works including SAM 2, SAM 3, and Grounded-SAM.

SigLIP is a vision-language model released in March 2023 by researchers at Google DeepMind. It adapts the CLIP image-text pretraining approach by replacing CLIP's softmax-based contrastive loss with a pairwise sigmoid loss, which operates independently on each image-text pair rather than requiring a global view of all pairs in a batch. This change decouples the loss from batch size, enabling more memory-efficient training and improved performance at smaller batch sizes, a regime where softmax contrastive learning typically struggles. Despite this simplification, SigLIP matches or exceeds CLIP-style models on zero-shot image classification and image-text retrieval benchmarks when trained on comparable data.SigLIP is distributed as an image encoder plus aligned text encoder, supporting zero-shot classification with arbitrary class vocabularies, image-text retrieval, and use as a frozen backbone in downstream vision-language models. Pretrained models are available at multiple Vision Transformer sizes and input resolutions, including 224, 256, 384, and 512 pixel inputs. SigLIP is released under the Apache 2.0 license by Google and is used as the vision encoder in Google's PaliGemma and PaliGemma 2. A successor, SigLIP 2, was released in February 2025 with multilingual support across 109 languages, improvements to localization and dense prediction, and two resolution handling variants (FixRes for backward-compatible fixed resolutions and NaFlex for native aspect ratio with variable sequence length).

Grounding DINO is an open-vocabulary object detection model developed by IDEA Research, released in March 2023 under the Apache 2.0 license. It extends the DINO transformer-based detector with grounded pre-training, enabling it to detect arbitrary objects described by free-form text queries rather than a fixed set of predefined categories. The model integrates a text encoder with a visual backbone through a feature fusion module that aligns language and visual representations at multiple scales.Grounding DINO achieves strong zero-shot detection performance on COCO, LVIS, and ODinW benchmarks, and supports referring expression comprehension tasks. It is widely used as a foundation for open-vocabulary detection pipelines and as the detection backbone in systems such as Grounded-SAM. The model is particularly suited for applications requiring flexible, text-driven object localization across diverse domains.

OWL-ViT (Open-World Localization with Vision Transformers) is an open-vocabulary object detection model released in May 2022 by Google Research. It adapts a pretrained CLIP-style image-text model by removing the final pooling layer and attaching lightweight classification and box prediction heads to each Transformer output token, producing a detector capable of localizing arbitrary objects described by free-form text at inference time. Rather than being restricted to a fixed taxonomy such as the 80 categories in Microsoft COCO, OWL-ViT can detect object classes specified by a user's text query, including categories the model was never explicitly trained on.OWL-ViT accepts an image and a list of text queries as input, and produces bounding boxes with class assignments drawn from the supplied queries. It also supports one-shot image-conditioned detection, where a cropped image region is used as the query instead of text, allowing the model to find visually similar instances within a target scene. The model is released in multiple Vision Transformer sizes (ViT-B/32, ViT-B/16, ViT-L/14) and CLIP-pretrained variants, distributed through the Google Research scenic repository and Hugging Face under the Apache 2.0 license. A successor model, OWLv2, was released in June 2023, introducing the OWL-ST self-training recipe that scales training to over one billion pseudo-annotated examples and substantially improves detection performance on rare and long-tail categories while preserving the open-vocabulary interface.

OpenAI CLIP (Contrastive Language-Image Pretraining) is a vision-language model released in January 2021 by OpenAI. It jointly trains an image encoder and a text encoder to produce matching embeddings for image-caption pairs, using a contrastive objective over WebImageText (WIT), a dataset of 400 million image-text pairs collected from the public web. By learning to associate images with free-form text rather than a fixed set of class labels, CLIP produces a shared embedding space that enables zero-shot classification with arbitrary vocabularies at inference time.CLIP supports zero-shot image classification by embedding candidate class labels as text and selecting the label whose embedding is closest to a given image's embedding. It is also widely used for image-text retrieval, as a frozen backbone in downstream vision-language models, and as a building block for content moderation, similarity search, and generative model guidance — notably as the text conditioning mechanism in early versions of Stable Diffusion. OpenAI released several CLIP variants built on different vision encoders, including ResNet and Vision Transformer backbones at multiple sizes and input resolutions, with ViT-L/14 at 336 pixels being the largest and most widely adopted. CLIP is distributed under the MIT license. The model has been widely influential as the basis for subsequent vision-language work — including SigLIP, OpenCLIP, and MetaCLIP — and remains a common reference baseline despite being released in 2021 and surpassed on many benchmarks by later models.

DETR (Detection Transformer) is an end-to-end object detection model developed by Facebook Research (Meta), released in May 2020. It is one of the first models to eliminate hand-crafted components such as anchor generation and non-maximum suppression by framing object detection as a direct set prediction problem, solved with a transformer encoder-decoder architecture built on top of a CNN backbone.DETR achieves 42.0% AP on the COCO benchmark with a ResNet-50 backbone, performing comparably to a well-tuned Faster R-CNN at the time of release. Its attention-based design allows it to reason about global context and long-range dependencies within an image. DETR is primarily used as a research baseline and architectural reference, with subsequent works such as Deformable DETR and DINO building on its foundations to address its slower training convergence and limited small-object detection capability.

Detectron2 is a computer vision model library developed by Facebook AI Research (Meta), released in September 2019. It serves as a comprehensive platform for object detection, instance segmentation, panoptic segmentation, keypoint detection, and DensePose, implemented in PyTorch. It is the successor to the original Detectron framework, which was written in Caffe2, and offers a more modular and extensible codebase designed for both research and production use.Detectron2 includes implementations of Faster R-CNN, Mask R-CNN, RetinaNet, Cascade R-CNN, Panoptic FPN, and several other architectures. Its modular design allows components such as backbones, necks, and heads to be swapped independently, making it widely used as a baseline framework in academic research. It supports training on COCO-format datasets and integrates with standard distributed training setups.

Mask R-CNN is an instance segmentation model developed by Facebook AI Research (Meta), released in October 2017. It extends Faster R-CNN by adding a parallel branch that predicts binary segmentation masks for each detected object, independent of the classification and bounding box regression branches. A key contribution is RoIAlign, which replaces RoIPool with bilinear interpolation to preserve spatial correspondence between features and input pixels, significantly improving mask quality.Mask R-CNN achieves strong performance on the COCO instance segmentation benchmark and supports keypoint detection as an additional output head. It remains a foundational architecture in instance segmentation and is available through Meta's Detectron2 framework. The model is most appropriate for tasks requiring pixel-level object delineation, such as medical imaging, autonomous driving, and industrial inspection.