Ball Tracking Robot
The ball tracking robot uses object tracking to follow the ball. With the use of the Python Library, OpenCV, the camera can recognize faces, detect objects, and much more. This project uses a Raspberry Pi, Pi Camera, Motor Driver to control the motors, DC Motors to control the wheels, and a Ultrasonic Sensor to avoid object collision.
| Engineer | School | Area of Interest | Grade |
|---|---|---|---|
| Zachary Yeung | Lynbrook High School | Engineering | Incoming Sophmore |
.jpg)
Third Milestone
Summary
For my third milestone, I incorporated the ultrasonic sensor into my project. When the robot gets to a certain distance from an object, it will stop moving until the object moves further away.
Challenges
I wanted to utilize the two other ultrasonic sensors in my project, but when I used three ultrasonic sensors, the frames for my PiCamera dropped from 30 fps to around 5 fps. This made it pretty hard for my robot to track the ball, so I only used one. I tried using threading which is running multiple functions or codes at the same time. This method succeeded in keeping the frame rates high, however I realized I will have to cover the right and left ultrasonic sensor for my modification.
What’s Next?
I plan on adding a way for the robot to take the ball and bring it back to me by using servos to take the ball and implementing PID control.
Second Milestone
Summary
My second milestone is to code the ball tracking for my robot. The main components I used for the second milestone is the Raspberry Pi, PiCamera, and the motors. The code I wrote first creates a contour around the ball with x and y as the center of the contour. The x value is the width of the frame. Based on this, if the x value is on the right side of the frame, then the car moves to the right. If the x value is on the left side of the frame, then the car moves left. If the ball is around the middle of the frame, the car moves forward. Lastly, when no contours are detected, the car will turn constantly turn right until it finds anything red.
Challenges
A challenge I faced was coding because I didn’t really know how to code. So I used many different tutorials to create my ball tracking code. Another challenge I had was when I finished coding the ball tracking, the car would only move left and right to get to the ball, which was fixed when I added a middle range where it could move forward. The last challenge I had was setting up VNC so that I could run the Raspberry Pi without connecting it to my laptop. It only worked once for me, but now it doesn’t work anymore.
What’s Next?
For the next milestone I am going to use the ultrasonic sensors to avoid colliding into object. I also might try to set up the VNC again.
First Milestone
Summary
My first milestone is wiring up all the components of the robot and making sure that it work. The components I used are the Raspberry Pi, DC motors, a motor driver, ultrasonic sensors, and a PiCamera. The Raspberry Pi controls all the components, the DC motors spin in certain directions, the motor driver amplifies current to the DC motors, the ultrasonic sensors calculates distance, and the PiCamera is a camera. Right now, the motors spin, the PiCamera works fine, and the ultrasonic sensors work. Later on I will write code so that it will track a ball using the PiCamera and if the ball moves, the robot will follow it. I will also use the ultrasonic sensors to avoid crashing into obstacles like the ball.
Challenges
One challenge I faced during the first milestone was setting up the Raspberry Pi. I was confused because not only did you have to set up the MicroSD card but you also had to download certain softwares to display the Raspberry Pi on a device. I also had a hard time setting up the SSH for the Raspberry Pi, which is a way to connect the command promt on your computer to the Raspberry Pi terminal. Everytime I tried connecting to the Raspberry Pi through SSH, it would say that it cannot find the Raspberry Pi.
What’s Next?
For the next milestone, I will code ball tracking, and making sure that the robot follows if the ball moves.
Starter Project
Summary
For my starter project, I created the game console. I chose this project because it seemed coolest out of all the starter projects. This game console includes Tetris, Snake, Racing, Space Invaders, and Slots. How this game console works is whenever a button is pressed, it sends a signal to the chip, which is then displayed on the LED dot matrix. The red switch is used to turn on the console, the yellow button is used pause the game, the green button starts the game, and the blue buttons controls the LEDs.
Challenges
A challenge when creating this starter project is soldering. I accidentally burnt some parts with the soldering iron, but my game console still works. To avoid this, pay attention to what your soldering iron is touching.
Code
PID control
import RPi.GPIO as GPIO
import time
import math
GPIO.setmode(GPIO.BCM)
# Motor pins
ENA = 10 # Left motor enable
ENB = 9 # Right motor enable
in1,in2,in3,in4 = 2,3,17,27
#in1,in2 right motor
#in3,in4 left motor robot facing forward
# Set up GPIO
encoder_right = 13
encoder_left = 26
GPIO.setmode(GPIO.BCM)
GPIO.setup(encoder_right, GPIO.IN)
GPIO.setup(encoder_left, GPIO.IN)
# Calculations
wheel_radius = 0.0325
wheelbase = 0.157
meters_per_tick = (2 * math.pi * wheel_radius) / 20
previous_right_value = 1
right_ticks = 0
previous_right_ticks = 0
previous_left_value = 1
left_ticks = 0
previous_left_ticks = 0
theta = 0
x = 0
y = 0
x_goal = 1
y_goal = 1
# PID gains
Kp = 1.5
Ki = 0.01
Kd = 0.5
# Initialize GPIO
GPIO.setup([ENA,ENB,in1,in2,in3,in4],GPIO.OUT)
# Initialize PWM for motor speed control
pwm_left = GPIO.PWM(ENA, 1000)
pwm_right = GPIO.PWM(ENB, 1000)
pwm_left.start(0)
pwm_right.start(0)
# Set initial motor speeds
target_speed = 100 # Desired speed (0-100)
pwm_left.ChangeDutyCycle(target_speed)
pwm_right.ChangeDutyCycle(target_speed)
# Initialize PID variables
prev_error = 0
integral = 0
# Main loop
try:
while True:
# Get current position (x, y, theta) from sensors
# Read the sensor's output
right_encoder_value = GPIO.input(encoder_right)
left_encoder_value = GPIO.input(encoder_left)
if previous_right_value == 0 and right_encoder_value == 1:
right_ticks += 1
print("right_ticks: ",right_ticks)
previous_right_value = right_encoder_value
if previous_left_value == 0 and left_encoder_value == 1:
left_ticks += 1
print("left_ticks: ",left_ticks)
previous_left_value = left_encoder_value
time.sleep(0.005)
delta_ticks_right = right_ticks - previous_right_ticks
delta_ticks_left = left_ticks - previous_left_ticks
previous_left_ticks = left_ticks
previous_right_ticks = right_ticks
distance_L = meters_per_tick * delta_ticks_left
distance_R = meters_per_tick * delta_ticks_right
distance_average = (distance_R + distance_L) / 2
x_change = distance_average * math.cos(theta)
y_change = distance_average * math.sin(theta)
theta_change = (distance_R - distance_L) / wheelbase
x = x + x_change
y = y + y_change
theta = theta + theta_change
theta_range = (theta - (2 * math.pi * math.floor((theta + math.pi)/(2*math.pi)))) #Keeps theta in the interval (0, 360)
theta_range = theta * 180/math.pi
print(x,y,theta_range)
# Calculate error with respect to origin (0,0)
x_error = x - x_goal # current x position - desired x position
y_error = y - y_goal # current y position - desired y position
# Compute PID terms
error = Kp * (x_error + y_error)
integral += Ki * error
derivative = Kd * (error - prev_error)
# Compute control output (adjust motor speeds)
left_speed = target_speed + error + integral + derivative
right_speed = target_speed - error - integral - derivative
print(left_speed)
print(right_speed)
# Apply control output to motors
pwm_left.ChangeDutyCycle(left_speed)
pwm_right.ChangeDutyCycle(right_speed)
# Update previous error
prev_error = error
# Sleep for a short time (loop rate)
time.sleep(0.1)
except KeyboardInterrupt:
GPIO.cleanup()
pwm_left.stop()
pwm_right.stop()
Ball Tracking
import RPi.GPIO as GPIO
import time
import cv2
from picamera2 import Picamera2
import numpy as np
from gpiozero import Servo
GPIO.setwarnings(False)
# To use Broadcom GPIO numbers instead of using board pi numbers
GPIO.setmode(GPIO.BCM)
servo_right = Servo(16)
servo_left = Servo(12)
servo_right.detach()
servo_left.detach()
in1,in2,in3,in4 = 2,3,17,27
TRIG_FRONT = 23
ECHO_FRONT = 24
# Front Sensor
GPIO.setup(TRIG_FRONT, GPIO.OUT)
GPIO.setup(ECHO_FRONT, GPIO.IN)
# Set the trig pin to low
GPIO.output(TRIG_FRONT,False)
# Setup the ouput pins
GPIO.setup([in1,in2,in3,in4],GPIO.OUT)
GPIO.output([in1,in2,in3,in4],GPIO.LOW)
# Create a Picamera2 instance
picam2 = Picamera2()
# Configure camera settings
config = picam2.create_preview_configuration(main={"format": "XRGB8888", "size": (640, 100)})
picam2.configure(config)
picam2.start()
def move_forward():
GPIO.output(in1,GPIO.HIGH)
GPIO.output(in2,GPIO.LOW)
GPIO.output(in3,GPIO.LOW)
GPIO.output(in4,GPIO.HIGH)
def turn_left():
GPIO.output(in1,GPIO.HIGH)
GPIO.output(in2,GPIO.LOW)
GPIO.output(in3,GPIO.LOW)
GPIO.output(in4,GPIO.LOW)
def turn_right():
GPIO.output(in1,GPIO.LOW)
GPIO.output(in2,GPIO.LOW)
GPIO.output(in3,GPIO.LOW)
GPIO.output(in4,GPIO.HIGH)
def sensor():
# Wait 50 miliseconds and set trig to high for a new ultrasonic pulse transmission
time.sleep(0.05)
GPIO.output(TRIG_FRONT,True)
# Set it back to low after this pulse to wait for the pulse to bounce back
time.sleep(0.00001)
GPIO.output(TRIG_FRONT,False)
# If no pulse returns ECHO is 0 and it records the time. When there is a pulse ECHO becomes 1 and it also records the time using time.time()
while GPIO.input(ECHO_FRONT) == 0:
pulse_start = time.time()
while GPIO.input(ECHO_FRONT) == 1:
pulse_end = time.time()
pulse_duration = pulse_end - pulse_start
distance = (pulse_duration * 34300)/2 # distance = (time * speed of sound [34300 cm/sec]) / 2
return round(distance,2) # rounds the distance by two decimal places
def motor(x, distance):
timer_started = False
start_time = 0
if distance > 5:
if x >= 430: # turn right
turn_right()
elif x <= 210: # turn left
turn_left()
elif 210 < x < 430: # move forward
move_forward()
elif len(contours) == 0: # spin right in a circle when no red object found
turn_right()
else:
GPIO.output([in1,in2,in3,in4],GPIO.LOW)
print("stop")
servo_right.max()
servo_left.min()
time.sleep(1)
servo_right.detach()
servo_left.detach()
while True:
# Capture a frame from the camera
frame = picam2.capture_array()
hsv_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
# Detect the ball
lower_color = np.array([150, 140, 1])
upper_color = np.array([190, 255, 255])
mask = cv2.inRange(hsv_frame, lower_color, upper_color)
mask = cv2.erode(mask, None, iterations=2)
mask = cv2.dilate(mask, None, iterations=2)
# Find contours and track the ball
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
if contours:
ball_center = max(contours, key=cv2.contourArea)
((x, y), radius) = cv2.minEnclosingCircle(ball_center)
if radius > 10:
cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2)
distance = sensor()
print(distance)
motor(x, distance)
# Display the frame
cv2.imshow("Ball Tracking", frame)
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break
# Clean up
cv2.destroyAllWindows()
picam2.stop()
GPIO.cleanup()
Bill of Materials
| Part | Note | Price | Link |
|---|---|---|---|
| Canakit | Contains Raspberry Pi 4 Model B, USB-C power supply, USB MicroSD card reader, MicroSD card, Micro HDMI to HDMI Cable, USB-C PiSwitch, and a case to hold the Raspberry Pi | $119.99 | Link |
| 4 L298n motor drivers | Controls the DC motor | $9.99 | Link |
| Robot Car Kit | Kit with a robot car chassis, DC motors, and tires | $12.99 | Link |
| Ultra Sonic Sensors | 3 Ultrasonic Sensors with 3 acryllic mounts, female to female wires, and male to female wires | $7.99 | Link |
| Pi Camera | Contains a Pi Camera and a acryllic case | $12.99 | Link |
| Soldering Kit | Contains a soldering iron and solder wire | $12.67 | Link |
| Logitech wireless keyboard and mouse | Wireless keyboard and mouse | $16.49 | Link |
| Basic connections components kit | Contains breadboards, resistors, LEDs, and jumper wires | $11.99 | Link |