James L's Senior Project Blog

Project Title: Predicting Human Stupidity: Enhancing Safety With Machine Vision Based Motion Prediction BASIS Advisor: Debby Hermann Internship Location: University of Texas at San Antonio -Department of Electrical and Computer Engineering, Autonomous Control Engineering Lab Onsite Mentor: Yufang Jin, Ph.D, Professor, Department of Electrical and Computer Engineering

Project Abstract

In a world increasingly dependent on technology, ensuring human safety has become one of the most pressing challenges, particularly as our environments grow more automated. As industries adopt robotics and automation to enhance efficiency, the interaction between humans and machines introduces risks that must be mitigated to prevent accidents. This project aims to address these concerns by using machine vision techniques to predict and account for human error, ultimately protecting human lives in high-risk environments. The UTSA Autonomous Control Engineering Lab has been working on a foundational project that utilizes camera vision to monitor the surroundings of manufacturing robots. My project will expand on that groundwork by identifying potential dangerous human activities around the robot and implementing proper response mechanisms. To achieve this, the project will integrate Kalman filters which isolates human shapes from outside noise, OpenPose which tracks key points on a human body, and machine learning to analyze human poses and movements. The algorithm will be trained to recognize specific behaviors and postures commonly associated with distracted or risky actions, such as looking down at a phone or eating while walking. By identifying these actions in real time, the system will alert robots to slow down or adjust their trajectories to prevent collisions with humans. Through this research, we will be able to predict human movement 2-3 seconds in advance at a 90+ percent accuracy and limit human injuries from autonomous machines which could be applied on autonomous vehicles and manufacturing.