My Research

Malaysian Sign Language Real-Time Tutorial using CNN Algorithm

The communication challenges faced by people with hearing and speech impairments in Malaysia are made worse by effective resources for learning Malaysian Sign Language (MSL). The Malaysian Sign Language Real Time Tutorial (MASRETT) is an instructional website designed to decrease the communication gap between both disabled and non-disabled. MASRETT is developed to assist in learning fundamental MSL virtually besides reducing the communication barriers which leads to a better and stronger social relation in Malaysia. Agile software development is the methodology of choice for creating this system since it facilitates the iterative identification and correction of mistakes. The artificial intelligence (AI) model which is convolutional Neural Network (CNN) algorithm is used to recognize the sign language input. The findings indicate that MASRETT significantly enhances accessibility and effectiveness of MSL learning particularly for individuals with lack access to traditional classes. However, the results showed that only 33% of the signatures are recognized using CNN algorithm for real time signs detection. It is observed that the results are not significant because of the limitation in data and inefficient model usage and can be improved using improved algorithm as in future work.

• SEMARAK ILMU SDN BHD• December 2024

PUBLISHED

Optimizing American Sign Language Recognition with Binarized Neural Networks: A Comparative Study with Traditional Models

Sign language is crucial for communication among individuals with hearing or speech impairments. Automated recognition systems are essential for learning and translating different sign language variants. However, these systems often face high computational demands and large memory footprints, limiting their use in real-time and resource-constrained environments. This research develops an optimized pipeline for American Sign Language (ASL) recognition, comparing Binarized Neural Networks (BNNs) with traditional full-precision neural networks. Using Larq, a library for training binarized models, we leverage BNNs' reduced memory and computational needs, suitable for embedded systems and edge devices. The study uses a dataset of ASL alphabet images, applying data augmentation to address data imbalance and occlusions. Both binarized and traditional models are trained and evaluated on accuracy, precision, recall, F1-score, memory footprints, and inference times. Results show that BNNs offer competitive performance with significantly lower computational requirements, demonstrating their potential for efficient and accessible ASL recognition systems.

• BRAC University• May 2024

SUBMITTED TO EGU25-12648

HOLOS-IE: A System Model for Assessing Carbon Emissions and Balance in Agricultural Systems

Agriculture significantly contributes to greenhouse gas (GHG) emissions, mainly via enteric and manure methane (CH4) from livestock and fertilizer-induced nitrous oxide (N2O) from soils. Mitigation strategies include dietary changes, feed additives, and fertilisation with circularity approaches. Agroforestry further offsets GHGs through carbon sequestration (soil and biomass) while enhancing soil health and ecosystem services. Achieving carbon-neutral farms by 2050 requires sustainable agricultural transformation. System-based modelling is crucial for understanding agriculture, supporting informed decision-making, and balancing data needs. HOLOS-IE, evolving into HOLOS-EU, simplifies complex modelling for farmers and stakeholders, empowering them to reduce their environmental footprint and achieve sustainable production. The HOLOS-IE v3.0 (www.ucd.ie/holos-ie) utilises large datasets, evidence-based algorithms, GIS, Machine Learning, and C#.NET coding. The ongoing development focuses on refining model components (crops, grasses, livestock, agroforestry and farm infrastructure), and their sub-components. These components are driven by key soil, climate and relevant variables, which are automated or user-defined inputs. As a case study, HOLOS-IE was applied to a 30-hectare Irish dairy farm to explore agroforestry scenarios (silvopastoral systems with Oak and Sycamore hedgerows) by sparing 5% of land without reducing livestock density. The model predicted sectoral GHG emissions, carbon removal, and total/net carbon balance, quantifying soil and biomass carbon sequestration. This analysis highlighted the offsetting potential and provided insights into total and net carbon balances, guiding future land-use planning for climate change mitigation... [TENTATIVE]

• Copernicus Publications• January 2025

Ongoing

An optimized deep-learning based pipeline for recognition of sign language from low-resolution thermal imagery

Sign language is used mainly by handicapped and disabled individuals to communicate with people from within and outside their communities. Since effective communication is mandatory in this interconnected world, each country has its own variant of sign language, which acts as a line of communication between normal and handicapped individuals. Computer-based automated recognition of sign language is important since it can help us learn different sign languages and perform automated translation. There are quite a few existing sign language detection modules, but it is often difficult to introduce those in daily life activities as those are not prepared for all the difficult scenarios, such as during the night or in low-light conditions. In this research, we propose an optimized pipeline for sign language detection under low-light conditions using thermal images of sign language. The influence and capabilities of thermal imaging in the detection of heat patterns from hand gestures help extract features easily. We employ Larq, an open-source Python library for training binarized neural networks for optimization purposes, which is effective computationally and has a low cost. These factors are crucial in terms of real-time recognition and accessibility. For comparison, traditional full precision models under similar conditions are evaluated as well. Furthermore, the research is done on a dataset consisting of low-resolution thermal images of sign language digits. The experimental results reveal that the module proves to be competitive with traditional models in terms of performance metrics, while also maintaining low computational requirements. The results lay the groundwork for effective and accessible solutions for sign language recognition. Additionally, it sheds light on the research needed to advance binarized neural networks (BNNs).

• February 2024