EyeSpeak: Smart Eye Tracking System for Assistive Communication

Abstract

Recent developments in eye tracking technology and human–computer interaction have facilitated the creation of communication systems for people with severe physical and speech disabilities. This literature survey explores research works related to eye tracking, eye gaze estimation, blink detection, facial landmark detection, virtual keyboard, and multimodal interaction for hands-free communication. The authors of these works have implemented various approaches and used different technologies to improve the accuracy of eye movement detection, typing speed, and overall communication efficiency. For example, computer vision with OpenCV, MediaPipe, Dlib, Eye Aspect Ratio (EAR), Convolutional Neural Networks (CNN), Machine Learning (ML), Large Language Models (LLMs), and text-to-speech APIs were employed to achieve high performance, reliability, and usability. Moreover, some researchers suggested using head pose estimation, voice commands, predictive text, and advanced interaction techniques to simplify the communication process and reduce physical interaction. The developed eye-tracking systems and communication tools were successfully applied to assistive communication, Augmentative and Alternative Communication (AAC), healthcare, rehabilitation, human–computer interaction, and accessibility domains and benefited people with Amyotrophic Lateral Sclerosis (ALS), cerebral palsy, quadriplegia, and other conditions. However, most solutions have some shortcomings, such as reduced accuracy due to lighting, calibration, and hardware constraints, extra weight, eye strain, the need for ocular tracking, slow typing, and the complexity of the underlying algorithms. To summarize, the papers mentioned in this literature survey offer a useful insight into current trends and approaches used to design effective communication systems and help build a solid foundation for creating low-cost, easy-to-use, and high-performance communication tools.