Differential Drive Mobile Robot Positioning System and Target Tracking

Image Processing Course

In this project, we implement a differential drive mobile robot positioning system and target tracking using image processing techniques. The system is designed to localize a robot in indoor environments using:

  • a ceiling-mounted camera for real-time tracking
  • an ESP32 microcontroller
  • a Python program on a PC for image processing and control decisions
  • Aruco markers for target identification
  • a differential-wheeled robot

To process real-time video feed from the camera, we use OpenCV and Python. Operations such as intensity transformations, geometric adjustments, and Aruco marker detection are performed to facilitate accurate target tracking. The closed-loop feedback mechanism ensures responsive motor control based on real-time image analysis.

Check codes for Image processing and control along with ESP32 codes in this repo.

Exhibition fully working System
Data flow and Workflow Diagrams
Hardware Design
Circuit Diagram
Different scenarios for the robot where two aruco markers represent the robot front and back and one marker for the target. The scenarios show different positions and orientation of the robot and the target.
Prices and availability table
(Ram: RAM Electronics shop, Bab Ellouq, near Tahreer square, Cairo, Egypt)
Demonstration of different scenarios

Team members

Course Instructors

References

[1] Kloock, Maximilian, Patrick Scheffe, Isabelle Tülleners, Janis Maczijewski, Stefan Kowalewski, and Bassam Alrifaee. “Vision-based real-time indoor positioning system for multiple vehicles.” IFAC-PapersOnLine 53, no. 2 (2020): 15446-15453.

[2] Yi, Jingru, et al. “Oriented object detection in aerial images with box boundary-aware vectors.” Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021.

[3] Yang, Xue, and Junchi Yan. “Arbitrary-oriented object detection with circular smooth label.” Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16. Springer International Publishing, 2020.

[4] Benjdira, Bilel, et al. “Car detection using unmanned aerial vehicles: Comparison between faster r-cnn and yolov3.” 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS). IEEE, 2019.

[5] Braun, João, et al. “Object Detection for Indoor Localization System.” Optimization, Learning Algorithms and Applications: Second International Conference, OL2A 2022, Póvoa de Varzim, Portugal, October 24-25, 2022, Proceedings. Cham: Springer International Publishing, 2023.