Computer VisionPRODUCTION

Autonomous Vessel Tracking & Visual Re-Identification System

Production maritime perception pipeline integrating YOLO vessel detection, OC-SORT tracking, and LoFTR + DINOv2 feature descriptors deployed on live RTSP streams via FastAPI.

CategoryComputer Vision

Statusproduction System

Core Tooling

YOLOOC-SORTDINOv2LoFTRFastAPIOpenCVPythonRTSP

01. Problem & Operational Context

Real-time vessel tracking on live maritime RTSP video feeds suffers from false-positive track fragmentation, wake interference, wave backscatter, and severe visual appearance changes as vessels change bearing.

02. Technical Approach

Engineered an end-to-end maritime vision pipeline combining YOLO detection, OC-SORT with motion-gating and per-track centroid history, coupled with a deep feature-matching identification stage using DINOv2 patch descriptors and LoFTR cross-attention.

03. System Architecture & Pipeline

End-to-End System Pipeline & Computation Flow

Continuous RTSP ingestion pipeline decoupled into fast temporal tracklet association and quality-adaptive visual descriptor matching.

STEP 01OpenCV / RTSP

Live RTSP Frame Ingestion

Multi-threaded RTSP video capture with ring-buffer frame queue to prevent packet dropping during inference spikes.

STEP 02YOLO / PyTorch

YOLO Vessel Detection

Real-time bounding box extraction with confidence thresholding tuned for maritime watercraft classes.

STEP 03OC-SORT / Kalman

OC-SORT Motion-Gated Tracking

Observation-Centric SORT tracking with momentum direction gating and centroid history to suppress false tracklets.

STEP 04LoFTR / DINOv2 / FastAPI

LoFTR + DINOv2 Feature Matching

Dense descriptor extraction and RANSAC geometric verification for vessel re-identification across bearing shifts.

04. Key Implementation Details

Engineered a vessel detection and tracking pipeline running continuously on live RTSP feeds, implementing motion-gating to suppress wave-induced false alarms.
Extracted visual descriptors using DINOv2 vision transformer features combined with LoFTR keypoint matching for non-rigid vessel identification.
Incorporated quality-adaptive thresholds and RANSAC inlier filtering to eliminate incorrect feature pairings in low-contrast conditions.
Deployed inference backend as a modular FastAPI service with operator review endpoints.

05. Architectural Tradeoffs & Decisions

Decision: Selected OC-SORT over standard SORT/DeepSORT for maritime tracking.

Justification

OC-SORT prevents error accumulation during nonlinear vessel turns and temporary target occlusions by leveraging observation trajectories.

Alternative Considered

Standard DeepSORT which frequently lost tracking lock during sharp course changes.

06. Engineering Challenges & Failures

Challenge: Water reflections and dynamic ship wakes generating spurious false-positive detections.

Solution: Implemented motion-gating requiring consistent temporal trajectory vectors across consecutive frames before track initialization.

Challenge: Vessel aspect ratio and lighting variations between approach and departure angles.

Solution: Integrated DINOv2 patch-level visual descriptors that retain structural contours across radical perspective shifts.

07. Evaluation & Observations

Evaluation Methodology:

Evaluated tracking persistence, ID switch reduction, and false-positive suppression on continuous live RTSP maritime video feeds.

Key Empirical Finding:

LoFTR cross-attention matching established robust vessel identity correspondence even in low-contrast fog and glare conditions.

Verified Artifacts

architectureRTSP Perception Pipeline Topology

End-to-end stream ingestion, OC-SORT motion gating, and FastAPI service architecture.

pipelineLoFTR + DINOv2 Matching Flow

Quality-adaptive thresholding and RANSAC geometric inlier filtering pipeline.

Project LinksView Repository All Projects