Roboflow

Ultralytics: YOLOv8 Instance Segmentation

YOLOv8 Instance Segmentation Overview

YOLOv8 Instance Segmentation is the segmentation variant of the YOLOv8 model developed by Ultralytics, released in January 2023 under the AGPL-3.0 license. It extends the standard YOLOv8 detection head with a mask prediction branch that generates pixel-level segmentation masks for each detected object using a prototype mask approach. This enables real-time instance segmentation within a single forward pass.

YOLOv8 Instance Segmentation shares the same backbone and neck architecture as the base detection model and is available in the same size range. It is deployable through Roboflow Inference and supports fine-tuning on custom COCO-format segmentation datasets. It is suited for applications requiring both object localization and precise mask prediction at real-time speeds.

YOLOv8 Instance Segmentation Interactive Demo

YOLOv8 Instance Segmentation Details & Performance

Details

Vision Tasks

Instance Segmentation

Features

Real-Time Vision

Usage

Past 30 Days

Performance

Avg. Latency

Arena Rankings

Not yet ranked in arena

Alternatives to YOLOv8 Instance Segmentation

Other models worth comparing for similar use cases.

YOLO26
YOLO26 is a real-time object detection model developed by Ultralytics, released in October 2025. It introduces a native end-to-end, NMS-free architecture that eliminates the Non-Maximum Suppression post-processing step, reducing CPU latency by up to 43% for the Nano variant compared to NMS-dependent versions. The model incorporates the MuSGD optimizer and ProgLoss with STAL for improved training stability and small-object detection, and removes Distribution Focal Loss to ensure maximum compatibility with ONNX and TensorRT export targets.YOLO26 supports object detection, instance segmentation, pose estimation, and oriented bounding box detection within a unified framework, with model sizes available from Nano to Extra Large. Its NMS-free design makes it particularly well suited for deployment scenarios where post-processing overhead is a bottleneck, such as embedded systems and real-time edge inference pipelines.
YOLO11
YOLO11 is an object detection and multi-task vision model developed by Ultralytics, released in September 2024 under the AGPL-3.0 license. It is the latest generation in the Ultralytics YOLO series and supports object detection, instance segmentation, image classification, pose estimation, and oriented bounding box detection within a single unified framework. YOLO11 introduces architectural refinements that improve accuracy while reducing parameter count compared to YOLOv8 at equivalent model sizes.YOLO11 is available in five model sizes from Nano to Extra Large and is deployable through the Ultralytics Python package, Roboflow Inference, and export formats including ONNX, TensorRT, and CoreML. It supports fine-tuning on custom datasets through the standard Ultralytics training API.
YOLOv9
YOLOv9 is a real-time object detection model developed by Chien-Yao Wang and Hong-Yuan Mark Liao at Academia Sinica, released in February 2024 under the GPL-3.0 license. It introduces Programmable Gradient Information (PGI), a mechanism that preserves complete input information through auxiliary reversible branches during training to address information loss in deep network layers. It also introduces the Generalized Efficient Layer Aggregation Network (GELAN), which achieves better parameter utilization compared to prior CSP-based designs.YOLOv9-C achieves 53.0% AP on COCO with 42% fewer parameters and 21% less computation than YOLOv8-C at comparable accuracy. YOLOv9-E achieves 55.6% AP. The model is deployable through Roboflow Inference and supports fine-tuning via the standard training pipeline in the official repository.
YOLOv12
YOLOv12 is an attention-centric real-time object detection model developed by researchers at Tsinghua University, with the arXiv paper published in February 2025 under the AGPL-3.0 license. It introduces an Area Attention module that partitions feature maps into regions and applies self-attention within each region, reducing the quadratic complexity of full self-attention while capturing long-range dependencies. It also incorporates R-ELAN for improved feature aggregation and scaled residual connections for training stability.YOLOv12-L achieves 54.0% AP on COCO, while the YOLOv12-N variant achieves 40.5% mAP at 1.62ms latency on an NVIDIA T4 GPU. The model is built on the Ultralytics codebase, supporting detection, segmentation, and other standard YOLO tasks at competitive real-time speeds.
Meta
Mask R-CNN
Mask R-CNN is an instance segmentation model developed by Facebook AI Research (Meta), released in October 2017. It extends Faster R-CNN by adding a parallel branch that predicts binary segmentation masks for each detected object, independent of the classification and bounding box regression branches. A key contribution is RoIAlign, which replaces RoIPool with bilinear interpolation to preserve spatial correspondence between features and input pixels, significantly improving mask quality.Mask R-CNN achieves strong performance on the COCO instance segmentation benchmark and supports keypoint detection as an additional output head. It remains a foundational architecture in instance segmentation and is available through Meta's Detectron2 framework. The model is most appropriate for tasks requiring pixel-level object delineation, such as medical imaging, autonomous driving, and industrial inspection.
RF-DETR Segmentation
RF-DETR Segmentation is a real-time instance segmentation model developed by Roboflow, with a preview base model released in October 2025 under the Apache 2.0 license and the full variant family — Nano through 2XL — released in January 2026. It extends the RF-DETR object detection architecture with a segmentation head inspired by MaskDINO, enabling pixel-level object delineation while maintaining the real-time performance characteristics of the base model. It is deployable through Roboflow Inference and the open-source rfdetr Python package.RF-DETR Segmentation supports fine-tuning on custom COCO- or YOLO-format instance segmentation datasets and is benchmarked on Microsoft COCO. It is suited for applications requiring both precise object masks and real-time inference, such as robotic manipulation, quality control, and augmented reality overlays.

YOLOv8 Instance Segmentation License

AGPL-3.0

License terms and commercial-use guidance for YOLOv8 Instance Segmentation.

This model is released under the GNU Affero General Public License v3.0 (AGPL-3.0), a strong copyleft license. Like GPL-3.0, derivative works must be released under the same license, and AGPL-3.0 extends this requirement to network deployment.

Read the full AGPL-3.0 license ↗

Commercial use is permitted under AGPL-3.0, but if you offer this model as part of a network service (such as a public API or web app), you must make the complete source code of your modified version available to all users of that service. Many commercial users prefer to acquire a separate license from the model authors to avoid this requirement.

AGPL-3.0 closes the "SaaS loophole" in GPL-3.0: even hosting the model behind an API counts as distribution and triggers the source-disclosure requirement.

To use YOLOv8 Instance Segmentation in a commercial project without the AGPL-3.0 conditions, you need a commercial license. As a paid Roboflow customer, you're automatically granted commercial-use rights for YOLOv8 Instance Segmentation models trained on or uploaded to our platform. See the Roboflow Licensing guide for the deployment-method by plan matrix.

If you're a free Roboflow customer, you can use YOLOv8 Instance Segmentation through our serverless hosted API at no cost. Self-hosted commercial use requires a paid plan.

License information is provided as a guide and is not legal advice.

Frequently Asked Questions About YOLOv8 Instance Segmentation

YOLOv8 Instance Segmentation is a pretrained computer vision model for instance segmentation. Unlike a general vision language model, it returns structured predictions for its task rather than free text.

YOLOv8 Instance Segmentation comes in 5 sizes: Nano (1280×1280), Small (1280×1280), Medium (1280×1280), Large (1280×1280), XL (1280×1280). Smaller variants run faster on constrained hardware; larger ones trade speed for accuracy. You can switch sizes in the demo to compare them on the same image.

Yes. The demo on this page runs YOLOv8 Instance Segmentation in the free Roboflow Playground: upload an image and see results in seconds. A free account unlocks unlimited runs.