Faster R-CNN vs RT-DETR

Faster R-CNN is an object detection model introduced by Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun at Microsoft Research, published at NIPS in June 2015. It advances upon Fast R-CNN and R-CNN by introducing the Region Proposal Network (RPN), a fully convolutional network that shares features with the detection network and generates object proposals at negligible additional cost. This makes Faster R-CNN the first near-real-time deep learning object detector based on region proposals.

Faster R-CNN achieves strong detection accuracy on PASCAL VOC and MS COCO at the time of release. It remains a widely referenced architecture in computer vision research and is available through Meta's Detectron2 framework as a maintained PyTorch implementation. It is most appropriate for offline or server-side inference tasks where accuracy is prioritized over latency, as its two-stage pipeline carries higher inference cost than single-stage detectors.

RT-DETR (Real-Time Detection Transformer) is an object detection model developed by Baidu, released in April 2023 under the Apache 2.0 license. It is the first transformer-based real-time object detector, addressing the inference speed limitations of earlier DETR models through an efficient hybrid encoder that decouples intra-scale interaction and cross-scale fusion, enabling the model to process multi-scale features without the high computational overhead of standard transformer encoders.

RT-DETR achieves 53.1% AP on COCO at 108 FPS on an NVIDIA T4 GPU for the RT-DETR-L variant, outperforming comparably sized YOLO detectors at similar speeds. It maintains end-to-end inference without non-maximum suppression, simplifying deployment pipelines. RT-DETR established the baseline for real-time transformer detection and has been extended by subsequent works including RF-DETR and RT-DETRv2.