Google

Google: MediaPipe

MediaPipe Overview

MediaPipe is an open-source framework developed by Google for building real-time machine learning pipelines across mobile, web, desktop, and edge platforms. First released in 2019, the framework uses a graph-based architecture where pre-built components called Calculators process streaming data such as images, video, and audio through configurable computation graphs. This design allows developers to compose perception pipelines from reusable building blocks without writing custom glue code between models. The current MediaPipe Tasks API replaces the earlier Solutions API and provides a unified cross-platform interface for vision, text, and audio.

Rather than providing a single model, MediaPipe ships a suite of ready-to-use Tasks that wrap trained models for specific problems. These include MediaPipe Pose Landmarker for 33-point body landmark detection, Hand Landmarker for 21-point hand tracking, Face Landmarker which extends the earlier 468-point Face Mesh with blendshape outputs for facial expression, Selfie Segmentation for person-background separation, and Holistic Landmarker for combined body, hand, and face tracking. The Tasks prioritize on-device inference with low latency and support GPU acceleration where available, making the framework a common choice for mobile augmented reality, fitness and wellness applications, gesture-based interfaces, and accessibility features such as sign language recognition.

MediaPipe Details & Performance

Details

Resources

GitHub

Vision Tasks

Object DetectionPose EstimationSemantic SegmentationKeypoint Detection

Features

Real-Time Vision

Usage

Past 30 Days

Not available

Not in Playground

Performance

Avg. Latency

Arena Rankings

Not yet ranked in arena

Alternatives to MediaPipe

Other models worth comparing for similar use cases.

YOLOv8 Pose Estimation
YOLOv8 Pose Estimation is the keypoint detection variant of the YOLOv8 model developed by Ultralytics, released in April 2023 under the AGPL-3.0 license. It extends the YOLOv8 detection head to predict keypoint locations and visibility scores alongside bounding boxes, using a decoupled head for joint localization and keypoint regression. By default it targets the 17-keypoint COCO human pose skeleton, but can be configured for custom keypoint sets.YOLOv8 Pose shares the same architecture and size variants as the base detection model and achieves competitive performance on the COCO keypoints benchmark at real-time inference speeds. The model is deployable through Roboflow Inference and is suited for applications including sports analytics, ergonomics monitoring, gesture recognition, and human activity detection.
Meta
Detectron2
Detectron2 is a computer vision model library developed by Facebook AI Research (Meta), released in September 2019. It serves as a comprehensive platform for object detection, instance segmentation, panoptic segmentation, keypoint detection, and DensePose, implemented in PyTorch. It is the successor to the original Detectron framework, which was written in Caffe2, and offers a more modular and extensible codebase designed for both research and production use.Detectron2 includes implementations of Faster R-CNN, Mask R-CNN, RetinaNet, Cascade R-CNN, Panoptic FPN, and several other architectures. Its modular design allows components such as backbones, necks, and heads to be swapped independently, making it widely used as a baseline framework in academic research. It supports training on COCO-format datasets and integrates with standard distributed training setups.
Meta
Mask R-CNN
Mask R-CNN is an instance segmentation model developed by Facebook AI Research (Meta), released in October 2017. It extends Faster R-CNN by adding a parallel branch that predicts binary segmentation masks for each detected object, independent of the classification and bounding box regression branches. A key contribution is RoIAlign, which replaces RoIPool with bilinear interpolation to preserve spatial correspondence between features and input pixels, significantly improving mask quality.Mask R-CNN achieves strong performance on the COCO instance segmentation benchmark and supports keypoint detection as an additional output head. It remains a foundational architecture in instance segmentation and is available through Meta's Detectron2 framework. The model is most appropriate for tasks requiring pixel-level object delineation, such as medical imaging, autonomous driving, and industrial inspection.
YOLO11
YOLO11 is an object detection and multi-task vision model developed by Ultralytics, released in September 2024 under the AGPL-3.0 license. It is the latest generation in the Ultralytics YOLO series and supports object detection, instance segmentation, image classification, pose estimation, and oriented bounding box detection within a single unified framework. YOLO11 introduces architectural refinements that improve accuracy while reducing parameter count compared to YOLOv8 at equivalent model sizes.YOLO11 is available in five model sizes from Nano to Extra Large and is deployable through the Ultralytics Python package, Roboflow Inference, and export formats including ONNX, TensorRT, and CoreML. It supports fine-tuning on custom datasets through the standard Ultralytics training API.
YOLOv8 Instance Segmentation
YOLOv8 Instance Segmentation is the segmentation variant of the YOLOv8 model developed by Ultralytics, released in January 2023 under the AGPL-3.0 license. It extends the standard YOLOv8 detection head with a mask prediction branch that generates pixel-level segmentation masks for each detected object using a prototype mask approach. This enables real-time instance segmentation within a single forward pass.YOLOv8 Instance Segmentation shares the same backbone and neck architecture as the base detection model and is available in the same size range. It is deployable through Roboflow Inference and supports fine-tuning on custom COCO-format segmentation datasets. It is suited for applications requiring both object localization and precise mask prediction at real-time speeds.

MediaPipe License

Apache 2.0

License terms and commercial-use guidance for MediaPipe.

License information is provided as a guide and is not legal advice.