Malaysian Sign Language Real-Time Tutorial using CNN Algorithm

AUTHORS

1. Meor Adib Zakwan Meor Ahmad Fauzi

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

2. Adnan Shafi

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

3. Toya Lazmin Khan

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

4. Nazihah Surati

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

5. Lele Mohammed

   Department of Computer Science, Federal Polytechnic Bauchi, 740102, Bauchi, Nigeria

6. Shakeef Ahmed Rakin

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

7. Nicholas Jia Chern Pang

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

8. Zuriahati Mohd Yunos

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

9. Sharifah Zarith Rahmah Syed Ahmad

   Faculty of Computing, Universiti Teknologi Malaysia, Skudai, 81310 Johor Bahru, Johor, Malaysia

ABSTRACT

The communication challenges faced by people with hearing and speech impairments in Malaysia are made worse by effective resources for learning Malaysian Sign Language (MSL). The Malaysian Sign Language Real Time Tutorial (MASRETT) is an instructional website designed to decrease the communication gap between both disabled and non-disabled. MASRETT is developed to assist in learning fundamental MSL virtually besides reducing the communication barriers which leads to a better and stronger social relation in Malaysia. Agile software development is the methodology of choice for creating this system since it facilitates the iterative identification and correction of mistakes. The artificial intelligence (AI) model which is convolutional Neural Network (CNN) algorithm is used to recognize the sign language input. The findings indicate that MASRETT significantly enhances accessibility and effectiveness of MSL learning particularly for individuals with lack access to traditional classes. However, the results showed that only 33% of the signatures are recognized using CNN algorithm for real time signs detection. It is observed that the results are not significant because of the limitation in data and inefficient model usage and can be improved using improved algorithm as in future work.