Method that uses PySerial and Arduino’s Serial library to send data from the Python code to the Arduino
Welcome to pySerial's documentation - pySerial 3.4 documentation
serial_port = serial.Serial('COM4', 9600, timeout=.1)
def write_angle(ser : serial.Serial, angle : int, motor : int):
ser.write(bytes([255]))
ser.write(bytes([motor]))
ser.write(bytes([angle]))
This function takes advantage of the Arduino’s USB connection to the “COM4” serial port. It instantiates an object of PySerial’s serial class at a baud rate of 9600 bits per second and uses bytes() to write the starting byte (see “Understanding Serial Communication” below), which lets the Arduino code know that this is the data that specifies servomotor input, the motor, and an angle to the serial port. Its parameters are as follows:
ser: the serial port being communicated through
angle: the angle being requested
motor: the specific motor being repositioned
import servocontroller
#...
ser = serial.Serial('COM4', 9600, timeout=.1)
# ...
def angle_change(ser : serial.Serial, hand_location : list):
hand_x = 1 if hand_location.x > 1 else hand_location.x
hand_y = 1 if hand_location.y > 1 else hand_location.y
# hand_z = abs(hand_location.z) {NOT WORKING}
normalized_location = {
hand_x : 9,
hand_y : 10
# hand_z : 11 {NOT WORKING}
}
for key in normalized_location:
angle = round(180 * key)
motor = normalized_location[key]
servocontroller.write_angle(ser, angle, motor)
# ...
def main():
while True:
# ...
coords = findHandPos(frame, cv2.COLOR_BGR2RGB)
if coords != None:
landmark_loc = coords[0].landmark[mpHands.HandLandmark.MIDDLE_FINGER_MCP]
angle_change(ser, landmark_loc)
This excerpt of the code is responsible for establishing a connection with the serial port, finding the location of a landmark (see “Hand Detection”), normalizing the individual coordinate components, and passing an angle assigned to a motor to the Arduino.
Because landmark locations are represented through a ratio of a position to the dimensions of the screen from 0 to 1, they can be treated as a modifier, or scalar, to some value. In this case, it is used to return an angle that can be directly inputted into a servomotor.
Coordinate components must be normalized because of what is believed to be floating-point errors in MediaPipe’s hand tracking model. Since the servomotor is strictly expecting values between 0 to 180, the coordinate components are explicitly bound from 0 to 1 through a check in the assignment of each corresponding variable.
#include <Servo.h>
Servo servo9;
Servo servo10;
Servo servo11;
// ...
int pythonData[3];
int startbyte;
int angle;
int motor;
// ...
void setup() {
Serial.begin(9600)
// ...
}
void loop() {
if (Serial.available() < 2) {
return;
}
startbyte = Serial.read();
if (startbyte != 255) {
return;
}
for (int i = 0; i < 2; i++) {
pythonData[i] = Serial.read();
}
motor = pythonData[0];
angle = pythonData[1];
switch (motor) {
case 9:
servo9.write(angle);
break;
case 10:
servo10.write(angle);
break;
case 11:
servo11.write(angle);
break;
}
}
This code establishes a connection to the serial port at the same baud rate as the connection made in the Python code, checks if the data in the port is the data specifying servomotor input, then uses the remaining data to reorient a specified servomotor.
There are two guard clauses in the main loop, one that checks if there are 3 bytes in the serial buffer (see “Understanding Serial Communication” below) and another that checks if the first byte in the buffer matches the starting byte specified in the Python transmission.
Serial communication is a method of sending data from one system to another by breaking down requests into single-bit packets, passing them through a channel such as a computer bus, and reading them onto the receiving system sequentially.