当前位置: X-MOL 学术Expert Syst. Appl. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
An ultra-low-computation model for understanding sign languages
Expert Systems with Applications ( IF 8.5 ) Pub Date : 2024-03-21 , DOI: 10.1016/j.eswa.2024.123782
Mohammad K. Fallah , Mohammadreza Najafi , Saeid Gorgin , Jeong-A. Lee

In artificial intelligence applications, advanced computational models, such as deep learning, are employed to achieve high accuracy, often requiring the execution of numerous operations. Conversely, lightweight computational models are typically more resource-efficient, making them suitable for various devices, including smartphones, tablets, and wearable technology. This paper presents an ultra-low-computation solution for interpreting sign languages to assist deaf and hard-of-hearing individuals without needing specialized hardware or significant computational resources. The proposed approach initially performs data abstraction on the input data. During this process, the image is systematically scanned from various perspectives, and the collected information is then encoded into a one-dimensional vector. Subsequently, the abstracted information undergoes processing through a Fully Connected Neural Network (FCN), resulting in highly accurate output. We also introduced two abstraction methods, namely Opaque and Glass, inspired by the interaction of light with different types of objects. The proposed abstractions facilitate the comprehension of the hand gesture’s outer boundary as well as its row-wise and column-wise density of pixels. Our experiments on three datasets confirm the efficiency of the proposed method, achieving an accuracy of 99.4% in recognizing American Sign Language, 99.96% accuracy in recognizing Indian Sign Language, and 99.95% accuracy in recognizing Bangla Sign Language. Notably, the model size and the number of MAC operations are significantly smaller than state-of-the-art computational models trained on the same datasets.

中文翻译:

用于理解手语的超低计算模型

在人工智能应用中,采用深度学习等先进计算模型来实现高精度,通常需要执行大量操作。相反,轻量级计算模型通常更节省资源,使其适用于各种设备,包括智能手机、平板电脑和可穿戴技术。本文提出了一种超低计算量的解决方案,用于解释手语,以帮助聋哑人和听力困难的人,而无需专门的硬件或大量的计算资源。所提出的方法首先对输入数据执行数据抽象。在此过程中,从各个角度系统地扫描图像,然后将收集到的信息编码为一维向量。随后,抽象信息通过全连接神经网络(FCN)进行处理,从而产生高度准确的输出。我们还引入了两种抽象方法,即不透明和玻璃,其灵感来自于光与不同类型物体的相互作用。所提出的抽象有助于理解手势的外边界及其行方向和列方向的像素密度。我们在三个数据集上的实验证实了该方法的效率,识别美国手语的准确率达到 99.4%,识别印度手语的准确率达到 99.96%,识别孟加拉手语的准确率达到 99.95%。值得注意的是,模型大小和 MAC 操作数量明显小于在相同数据集上训练的最先进的计算模型。
更新日期:2024-03-21
down
wechat
bug