PENERAPAN CONVOLUTIONAL NEURAL NETWORK PADA PENGENALAN BAHASA ISYARAT INDONESIA SECARA REAL-TIME

Abstract

Sign language is a form of visual communication used by individuals who are deaf or speech-impaired. However, many people in the general public still lack understanding of sign language, which hinders communication between people with disabilities and their surroundings. This research aims to develop a real-time alphabet translator system for Indonesian Sign Language (BISINDO), implemented as an Android application. The system utilizes a Convolutional Neural Network (CNN) model based on the MobileNetV2 architecture, which is trained to recognize 26 alphabet letters from hand gesture images sized 128x128 pixels in RGB format. The dataset was collected and processed through augmentation and divided into training, validation, testing, and evaluation sets. The model was trained using transfer learning and fine-tuning methods and then converted into TensorFlow Lite (.tflite) format for deployment on Android devices. Evaluation results show that the model achieved an average accuracy of 93% on the evaluation dataset. Testing the Android application also demonstrated good real-time performance in recognizing hand gestures. This application is expected to help bridge communication between people with disabilities and the general public through practical and accessible technology.